DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. DevOps and CI/CD
  4. DevOps Security at Scale (Part 4): Security as a First-Class Citizen

DevOps Security at Scale (Part 4): Security as a First-Class Citizen

Learn to optimize your code based on security, instead of resorting to last-minute fixes, in this series on building secure systems at scale.

Brian Kelly user avatar by
Brian Kelly
·
May. 16, 18 · Tutorial
Like (1)
Save
Tweet
Share
4.99K Views

Join the DZone community and get the full member experience.

Join For Free

This is the fourth blog post in a series discussing how high-performing DevOps teams build secure systems at scale.

Software teams are usually responsive to serious bugs, bad designs, or terrible performance of their systems. That’s because those are acute, obvious, in-your-face-bad kinds of issues that demand immediate attention.

On the opposite side of the coin are latent issues that, left untouched, will become bigger problems in the future. These are items that get addressed through preventative work before they become major visible problems. For example, technical debt is well understood by most software teams, and it usually gets periodic investment to prevent it spiraling out of control.

A sophisticated DevOps team will regard security issues as acute, immediate problems and not file them away in a growing backlog of To-Dos, never to be addressed. They elevate security from being a secondary concern to being a first-class citizen.

Less sophisticated teams think of vulnerabilities and security flaws the same way as technical debt, and try to chip away at them to prevent them from becoming a problem in the future. Unlike technical debt, vulnerabilities are problems right now. They haven’t yet been exploited. And once they’re exploited, it’s too late.

When a team puts security as a first-class citizen, they naturally adopt many good security practices as part of their development and release processes. Let’s examine some of those practices in detail.

Secure Management of Secrets

All secrets for productions systems (passwords, private keys, or any sensitive information that an attacker could use) are stored in a secure, highly-available vault and accessed at runtime only by the systems that have been granted access to them. The secrets are automatically changed on a frequent schedule, thereby limiting the amount of time they are valid even if they are breached. Human operators of the vault are not permitted access to the secret values held within.

Good Security Hygiene

There are a variety of “good hygiene” practices which security-focused teams will always take care to perform:

  • The principle of least privilege is applied everywhere possible: machines and humans only get access to the resources they absolutely deserve, and their permissions are promptly revoked once that privilege should be removed.
  • Vulnerability scans are automatically run on all dependent 3rd-party libraries. Vulnerable libraries are upgraded immediately if fixes are available. Upgrading 3rd-party libraries to newer compatible versions is not a complex process.
  • 3rd-party libraries are kept up to date, even if vulnerabilities aren’t yet detected in them. This helps avoid major compatibility issues if a dependent library needs to be immediately updated to address a vulnerability, but the patch is only available for the latest version of that library.
  • Penetration tests are conducted very regularly to scan the application (and its CI/CD environment) for potential attack vectors. These tests are sometimes even done as part of a CD pipeline using automated tools and augmented with regular human-driven, white-hat ethical hacks.
  • All developers are trained in application and information security as part of their new hire programs and in their ongoing education.
  • The development workflow for product code changes includes explicit checks for information exposure, attack vectors, and security policy violations. For example, if a team uses GitHub PRs, a PR template would be used that codified these expectations for the submitter and reviewer(s).

All Team Members Work to Enable Secure Velocity

Security teams in some companies have earned a negative reputation as “the people in the way of progress”. Typically this arises due to the security team hearing about a new system or initiative too late in its development cycle. Regardless of how free-flowing and speedy the development of that component was, the security team still have the job of ensuring that their business isn’t put at risk with the deployment of that new technology.

Security approvals can take time (especially with lots of concurrent projects going on) and can lead to the security team getting a reputation of being a blocker. Discontent mounts, emotions rise, and efforts are taken to work around their governance with “waivers” and “escalations.” In dysfunctional organizations, developers might even work to subvert the security team’s authority by covertly deploying applications without their approval, making matters even worse.

A high-performing organization will recognize that the root cause of the above problems come from the security team being in the wrong position of the development workflow.

Shifting Security Left

Sophisticated teams will directly involve security at its early stages of development of new applications, get architectures and technology choices reviewed by them as early as possible, and make an effort to “move security left” in the workflow. Then if the security team finds serious issues, they can be addressed much earlier, perhaps even before much code has been written. This leads to a much less contentious discussion between developers and security, and a far healthier relationship between those groups.

For teams that haven’t yet pushed security to the left, it’s probably natural for them to think that such a move would slow down their development process. If viewed within a short enough horizon, doing security reviews of early versions of new applications would seem like a costly and burdensome choice.

But if the entire development lifecycle is considered, those early security reviews prevent more costly rework later on when major architectural choices are proven to have vulnerabilities that have to be reworked. Early security reviews actually de-risk development projects and make much more economic sense than doing them as a later step in the release lifecycle.

Summary

Teams that work to put security concerns high on their list of priorities quickly learn to optimize around them. Instead of applying security fixes as band-aids when vulnerabilities are discovered, they modify their development flow to incorporate process and tools that help them avoid security gaps in the first place. This mindset leads these teams to be proactive in dealing with security issues, not reactive – and that is what it means to make security a first-class citizen in a DevOps organization.

DevOps Security at Scale Series

  • Part 1: Security Policy as Code
  • Part 2: Separation of Duties
  • Part 3: Focus on Flow and Velocity
  • Part 5: Automation
  • Part 6: Technology Adoption
security teams DevOps Citizen (app)

Published at DZone with permission of Brian Kelly, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Express Hibernate Queries as Type-Safe Java Streams
  • Stream Processing vs. Batch Processing: What to Know
  • A Brief Overview of the Spring Cloud Framework
  • How To Check Docker Images for Vulnerabilities

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: