Intellyx Brainblog: Weaving Security Into DevOps Atop Cloud Infrastructure

Reactive vs. Proactive Infrastructure Security Throughout the DevOps Lifecycle

An Intellyx BrainBlog by Jason English, for Bridgecrew
When looking at the early threads of software development, skill specialization was the name of the game. Developers coded new features. QA teams discovered bugs. IT Ops teams managed hardware. Security teams blocked threats. Project managers managed.

At some point in the last decade, these separate threads of specialized software disciplines gradually wove themselves together. First, agile methodologies pulled many aspects of requirements definition, testing, and delivery back into smaller development teams. Then, the DevOps movement caused multiple stakeholders from all disciplines to collaborate far more closely—from design to release.

When on-demand cloud infrastructure and fully automated infrastructure as code (IaC) started stitching together the underpinning fabric of DevOps practices, the responsibilities of developers started expanding to include much more than application code. In the context of cloud-native development, infrastructure specification, observability, and of course security must now be encompassed into a proactive DevSecOps lifecycle.

Short of expecting developers to become security experts, how can we weave security into the DevOps lifecycle, so teams can release with confidence without slowing down releases?

Threading secure dev practices into code

You’ve probably heard that old NIST report of the economic impact of fixing bugs by software phase. Whether you have or haven’t, it’s common sense that bugs caught in earlier stages of development require far less time and effort to fix and are thus exponentially less costly than bugs caught in a runtime environment.

We now shift automated testing left to catch bugs earlier with code linters and unit and integration testing, so developers can address coding errors and infrastructure specifications before they are fully baked into integrated code. So naturally, the same economic principle should apply to securing our software as early as possible. Security flaws are just another kind of software bug.

Still, security adds another dimension. It’s not just about the cost of labor—from both security and engineering teams—to remediate security flaws; it’s the risk of an exploit being introduced and escaping into production and the enormous potential costs of a resulting security breach.

To get ahead of this supply chain threat, we need to shift our security left, too, leveraging IaC’s machine readability to get security and compliance feedback as early as possible. Whether done in a developer’s command line or IDE, pre-commit IaC scanning gives developers a look-ahead security view of potential misconfigurations while they are actively coding.

Automated security scans and feedback on each commit, pull request, compile, and code check-in gives developers a real stake in the security of their software before it makes it into production.

Reactive vs. Proactive Infrastructure Security Throughout the DevOps Lifecycle

Download Infographic

Conventional application delivery practices encourage reactive security fixes after build and release time, which causes failures and issues to resolve in production. By proactively weaving automation into the DevSecOps lifecycle, developers can shift security left and discover misconfigurations and threats earlier.

Proactively braiding code and components together

Weaving code and components together in a distributed cloud and API-driven microservices world further amplifies the potential for configuration vulnerabilities.

This secure stitching was once managed with IT governance and security auditing tools atop the company’s application estate. Developers could run regression, smoke, and integration tests to validate that software would work properly in the environment, and the SecOps teams would review prior to deploying to production.

We can no longer risk leaving that until apps are deployed. The security of developed software and its configuration through IaC tools has become a critical part of agile delivery pipelines and security posture.
While IaC frameworks like Terraform and CloudFormation are superb for provisioning infrastructure scalably and predictably, most developers still have very little insight into what gaps might appear once their software gets pushed out into fast-changing, highly interdependent cloud environments.

Embedding security earlier becomes even more critical when infrastructure is involved—especially when open source components like Terraform modules and Helm charts are in the mix. Open source IaC templates are rarely secure by design so just like you would when adopting other OSS, it’s important to use scrutiny—ideally in an automated way.

By embedding security guardrails into CI/CD pipelines, you can ensure that all code being committed is free of misconfigurations and vulnerabilities. Additionally, adding guardrails throughout the test and delivery phases of the DevOps lifecycle is a great way to strengthen each release thread.

Tie off loose ends with real-time policy

With increased tightness, the combined tensile strength of woven fibers allows them to tolerate much higher weight and stress levels. With IaC scanning and policy enforcement threads already woven in, how can security further tighten software in production?

Running applications and storing data in elastically scaling cloud computing resources already upped the ante for security. With the shared responsibility model, we’re not concerned about whether the base layer of AWS itself or Google Cloud Platform would come with their own thorough perimeter security, DDoS prevention, and authentication controls. So long as no human shares the master account keys, the cloud service provider itself is pretty safe.

We are more concerned with the infrastructure and applications we deliver in the cloud. Today’s cloud-native production environments are subjected to unpredictable request volumes from remote services and the unexpected wind shear forces of traffic against a drastically increased attack surface.

The best-performing DevOps teams are pushing out releases up to exponentialy faster than traditional waterfall-style development shops. Unfortunately, at these breakneck speeds, unexpected variations in infrastructure can start to appear as misconfigurations in code and propagate quickly into multiple container images and Kubernetes clusters if left unchecked.

Because cloud delivery allows companies to deliver and decommission ephemeral microservices and data instances at breakneck speed, we must address these risks by tightening the gaps between dev and ops and shifting security left.

Just as importantly, who should be authorized to make a pull request, and who approves changes? How can SREs and security teams drop in to inspect and eliminate misconfigurations without seeing unauthorized private data or interrupting ongoing development and release processes?

Automated user access policies should step in here and act as a real-time ‘traffic cop’ governing individual user, team, and role-based read/write access to all resources: repos, release automation controls, and system analytics.

The Intellyx Take

Like silk or carbon fiber, once the threads of DevOps, infrastructure, and security are codified and woven together in proactive infrastructure DevSecOps lifecycles, our continuously changing cloud-based applications become much stronger and resilient than the sum of each individual thread.

To keep up with the fast pace of change inherent in today’s cloud applications, policy-based governance of infrastructure configuration and role-based access must be embedded throughout the DevOps lifecycle and reinforced in runtime with real-time monitoring, alerting, and automation capabilities.

Now is always the best time to tie up those loose ends to build a proactive DevSecOps lifecycle, to not only deliver software faster but more securely.

© 2021, Intellyx, LLC. Intellyx retains editorial control over the content of this blog. At the time of publishing, Bridgecrew (a Palo Alto Software company) is an Intellyx customer. Image credits: Intellyx (Infographic by Jason English).