Author: Guy Eisenkot

  • How Application Developer Turnover is Shaping Security

    How Application Developer Turnover is Shaping Security

    Observing the developer job market for the past two years reveals a surprising truth that has a profound impact on technology organizations. In the “new normal” of hybrid work, life moves fast and there’s no sign of slowing down. Since 2021, we’ve witnessed all-time record demand for software engineering developers. The emergence of privately held large and medium technology startups, fueled by a fear of missing out (FOMO) driven venture capital market, created vast growth opportunities for software engineer developers in particular. From developing and deploying new applications to scaling your business through multi-cloud environments, cloud and cloud-native companies are everywhere. Not only are cloud and cloud-native companies receiving enormous funding rounds, but worldwide spending on public cloud services is forecasted to grow 20.7% to a total of $591.8 billion in 2023.

    This dramatic cloud growth is primarily driven by the warp speed developers are operating. Thanks to developers, every company is now a digital company and every industry is reaping the benefits of a cloud presence. A retail organization can now build and deploy an application for the holiday shopping season within hours, and workforces can access applications from their kitchen, local coffee shop, or halfway across the world.

    However, as with any new, widely adopted technology, cybercriminals look to target and exploit organizations in their early cloud days as they begin to grasp and understand the technology.

    It was reported earlier this year that the recent exposure of roughly a terabyte of Pentagon emails was likely due to a cloud configuration error. Cloud attacks are on the rise, and there is a major disconnect about who is responsible for keeping the organization safe. Developers who are told to build with speed don’t want their applications slowed down or user experience to be altered by security protocols. Still, accelerating application velocity has security teams caught between remaining responsive to application teams and securing an increasingly complex cloud environment. A major problem is bubbling up, and if security and developers teams cannot come together, the result can leave organizations vulnerable.

    Developers and Turnover

    In a recent survey, 75% of respondents reported a higher-than-usual turnover rate in DevOps. Similar to the well-documented cybersecurity talent shortage, multiple workplace stressors appear to be taking a toll on developers. The same survey reported that 38% of enterprises deploy code to production or release to end users every day, with 17% deploying multiple times a day. To put this into perspective, just two decades ago, developers were shipping limited updates one to two times a year.

    Given the pressure to deliver new code every day, combined with security teams’ pressure to ensure that code is secure, it’s no surprise that we’re witnessing higher-than-normal developer attrition. Instead of pointing fingers at developers for unsecure code, security teams need to create tools and training that do not impede their number one priority: Shipping new code. We’re in a developer-friendly job market and organizations cannot afford to lose top talent because they are overburdening them with security tasks. Developers will find greener pastures.

    Shifting Left

    By now, everyone in the industry has heard of shifting security left—the practice of moving security to the earliest possible point in the development process to ensure code is secure and limit timely and costly remediation. In practice, this means bringing security to where the developers operate so vulnerabilities and misconfigurations are caught early when they are easy to fix.

    This approach has brought about a deeper level of engagement between application developers and security teams, especially in the design phase. However, while developers might now be embracing tools and processes that help them code secure applications, there’s still a limit to how much shift left responsibility they can and want to handle.

    Path Forward

    Shifting security left is certainly a recommended starting point, but simply equipping developers with DevSecOps tools isn’t a winning approach. Developers must be trained and have the desire to effectively use the tools. Like the old saying, give a man a fish, and you feed him for a day. Teach a man to fish, and you feed him for a lifetime. The same holds when shifting security left.

    To ensure developers and security teams understand their security responsibilities within the organization, consider implementing the following:

    1. Define a clear shift left strategy with developers and security teams
    2. Provision the necessary security tools that effectively protect the organization but don’t create roadblocks for developers. Additionally, more tools don’t necessarily mean more security, so be strategic.
    3. Train developers to be more mindful of cybersecurity issues. Most developers haven’t had formal cybersecurity training, so it’s up to organizations to make sure that particular skills gap is closed

    DevSecOps is as much about changing culture and approaches as it is about acquiring new tools. Developers will, unfortunately, come and go, but it should never be because security teams are overburdening them.

  • DevSecOps Will Cross the Chasm in 2022

    DevSecOps Will Cross the Chasm in 2022

    We’ve been talking about DevSecOps and shift-left security for years. Although this approach probably didn’t “cross the chasm” in 2021, we did see some very telling milestones. Cybersecurity VC funding surged to record heights with a focus on DevOps and cloud security and the need for DevSecOps became glaringly evident with the Log4j vulnerability. On a more personal note, our own open source security project Checkov has surpassed three million downloads.

    Much of this momentum was inevitable. Those in infrastructure and reliability circles have already been adopting and touting the benefits of shifting left and automating as much as possible, so in many ways, it’s expected that security would follow suit. Undoubtedly, the pandemic has also accelerated the need for developers to work more autonomously without having security personnel and processes acting as barriers to productivity and velocity.

    We’re confident that 2022 will see the silos between development and security teams continue to crumble as developer-led security practices become the norm for cloud-native organizations.

    DevSecOps finally crossing the chasm doesn’t mean that every enterprise and traditional organization will shift security left and adopt security best practices. It will, however, give them competitive advantages over those who don’t—both in decreased security costs and increased developer productivity and, thus, time-to-market.

    What else is in store for the future of DevSecOps?

    Rise of the DevSecOps Job Title

    Fewer than 5,000 people on LinkedIn currently have “DevSecOps” in their job title, yet there are over 20,000 current openings for DevSecOps roles. In 2022, expect to see more of those positions filled. This means security teams across industries will conduct fewer manual security audits, there will be a considerable influx of homegrown DevSecOps tooling and point solutions will likely begin to consolidate into single platforms. 

    Blurred Lines Between Application and Infrastructure Security

    Until recently, application security was a very well-defined (albeit fractured) space focusing on securing the custom code and open source packages that make up applications. However, with the rapid adoption of cloud-native applications, the lines between application and infrastructure security are blurring. We expect to see this trend continue as more engineers take on more infrastructure-related projects, vendors start catering to use cases outside of their core competencies (through acquisitions and in-house development) and the role of DevSecOps continues to expand within organizations.

    Infrastructure-as-Code: The Great Cloud Migration’s Next Chapter

    We’ve been talking about the great migration to the cloud for years. At this point, many companies are ready to move on to the next chapter: Infrastructure-as-code (IaC). As a result of this trend, DevSecOps will become much more important, as security needs to be baked earlier in development phases or risk being left behind. In addition, security teams will need to become more well-versed in development technologies and practices to provide the proper guidance for the new way applications are built and deployed. 

    More Software Supply Chain Attacks

    Hackers have been targeting retailers and security vendors for years, exploiting a minor weakness to gain access and move laterally into sensitive data. In the past year, however, software supply chain attacks stepped into the spotlight due to multiple supply chain attacks. We expect this trend to continue, which will, in turn, put more focus on securing supply chains. DevSecOps, which has focused mostly on tools and practices for securing the code and infrastructure, will expand to include the supply chain mechanism. 

    If 2021 was the year of hype for DevSecOps, we believe (and hope) that these best practices are embraced en masse across industries. The benefits of shift-left security are well documented: The number of high severity incidents is significantly reduced, the potential attack surface is minimized, compliance efforts are simplified and the time to remediation is lowered. Organizations also save money by catching misconfigurations and vulnerabilities earlier in the software development life cycle while at the same time gaining time back with tools, both open source and commercial, that are empowering developers to move fast and build applications that are more secure and reliable.

  • Building an IaC Security and Governance Program Step-by-Step

    Building an IaC Security and Governance Program Step-by-Step

    Infrastructure as code (IaC) is undoubtedly changing how engineers approach the cloud. IaC, when coupled with dev tools and automation, opens up new avenues for infrastructure performance, scalability and, now, security.

    By embedding IaC security and compliance controls into your version control systems and CI/CD pipelines, you can start identifying and fixing errors earlier. But, to do so without being disruptive, it’s vital to lay out a strategy to determine where and how to enforce security controls to meet your goals without slowing your developers down, from experimentation to governing your strategy.

    This post will share a technical process for approaching and building an internal IaC security strategy.

    What are Your Infrastructure Security Goals?

    The key to implementing the most sustainable infrastructure security program lies in leveraging the tools and workflows developers already use. For many, that may be easier said than done because security tooling traditionally lacks the access and context necessary to play nice with dev workflows. The IaC security approach is fundamentally different from the more reactive cloud security monitoring and alerting that happen independent of developer workflows.

    As you start paving the road ahead for continuous infrastructure security processes, start by defining your ideal state. That way, as you go through the process, you’ll have an idea as to how and where you want to enforce different security and compliance controls.

    What Policies Do You Want to Enforce, and How?

    Your infrastructure security program is highly dependent on your organizations’ security priorities and maturity, and you should read this guide with your unique needs in mind.

    If you want to improve your overall cloud security posture, you may want to look for ways to apply common security policy frameworks such as the Center for Internet Security. Several open source security scanners such as Checkov or Open Policy Agent (OPA) enforce policy-as-code at the IaC level.

    They’re usually standalone packaged containers defined to accept a file, directory, repository, branch, etc., and output scan results in a machine-readable format such as JSON or JUnit XML. The most significant advantage of using an open source scanner is that they pre-package valuable security testing logic that your developers won’t have to learn.

    The alternative to a scanner is a full-featured security product, preferably one with exposed APIs and webhook integration sources. Most developer-first security solutions have done this part of the heavy lifting, so you should be getting some of this out-of-the-box.

    If you need to enforce external controls such as industry benchmarks or regulatory compliance requirements, you’ll need a more structured approach that maps individual checks to desired outcomes. Many cloud providers include robust compliance reporting for standard benchmarks such as HIPAA, NIST, PCI-DSS, etc. Make sure your policy enforcement approach enables you to build your own policies so you can adopt those controls as well.

    Where Should You Enforce Infrastructure Policies?

    Each of your teams likely has its unique frameworks and processes in place, and it’s important to have a full lay of the land before diving in. Start with a simple mapping exercise and list all the configuration frameworks in use—from CloudFormation or Azure Resource Manager (ARM) to Terraform or Pulumi.

    Once you know what frameworks exist internally, the next step is to determine the best place to scan them for infrastructure security errors. With many frameworks in place, building out new processes from scratch would be an uphill battle. Depending on your organization’s DevOps maturity, you’ll likely find an existing foundation on which you can build.

    Understand your teams’ end-to-end code review process—from committing new infrastructure changes to CI/CD build pipelines and deployments. Your team likely has some automated tests—unit, acceptance, integration, etc.—in place to ensure updates work as intended and don’t introduce bugs. Use those processes as an example for inserting security scanning. To better understand those processes, you might want to ask questions such as:

    • What triggers a CI/CD build?
    • Do your current automated quality assurance efforts run continuously on every new pull request and commit?
    • Do more time-consuming tests such as integration tests run on less-than-continuous intervals such as nightly or only on branches following specific naming conventions?
    • Are developers testing their work locally before committing?
    • How many CI/CD pipelines are in use?

    How and where you implement your security scanning will depend on those processes and the tools used to do them. Based on your current setup, you’ll generally want to take one of two approaches to IaC security—leveraging CI/CD processes or the built-in functionality from your version control system (VCS).

    Approach #1: CI/CD Pipeline

    Building security controls as part of CI/CD pipelines is probably the easier of the two approaches because modern CI/CD tools have completely abstracted away the complexity behind orchestrating test logic.

    In many cases, scanning security in CI pipelines is the fastest way to implement an IaC security and governance program. If a modern CI tool is already in place, you can probably set it up in less than 10 minutes. If you find yourself going down this path, consider the residual results of the scan and spend time dry-running it with the team that’s going to be in charge of handling its output.

    For teams operating in a centralized infrastructure operation model, this is an optimal solution. Expertise and control can remain in the hands of a small few, and the continuous enhancements to this process will not affect a large population of developers. For everyone else—continue reading.

    Approach #2: Version control system

    Your version control system (VCS) of choice is also hiding some robust security governance internal controls. Apart from adding individual automation capabilities, all of the three platforms currently dominating the market (GitHub, GitLab and Bitbucket) offer embedded code review and authorization controls that can be utilized to better control who can change what and where.

    Through their native versioning and branching, developers can perform extensive testing without compromising any running production system and continue to collaborate and converse before making changes due to a test.

    By embedding infrastructure security here, there are some pros in terms of more fine-grained access and controls. Two of the more common methods to do that utilize the standard pull request review (PR) process:

    • PR annotations: Start simple by choosing the path of least resistance. Using a standard webhook or API interface, you can start by scanning every PR and annotating its results with concrete secure coding advice. The benefit of the annotation is that it’s frictionless. Developers can choose to adopt or ignore it—little frustration, but on the other hand, not a lot of governance.
    • Checks: With the addition of native CI to most git platforms, you can also always incorporate a standard CI pipeline in git. The main benefit is to include a CI-like scan as part of each individual PR. It surfaces the same kind of output in an earlier stage and enables developers to react before actually failing building in the pipeline.

    How Should You Enforce Security Output?

    Before you include intrusive controls and overwhelming security scanning results, it’s a good idea to set expectations for how security output should be actioned. Nobody wants to get bombarded by tens of failure indications with no clear path to justify their decisions. If your IaC strategy is noisy or becomes a blocker at the end of the day, your team won’t adopt it.

    Define how passive or active you want the output of your security controls to be. Should it block builds actively or provide passive feedback to address in some pre-defined SLA?

    Some scanners can even integrate further into pipelines and orchestrate more advanced actions, such as blocking a build from completing until testing is complete. Scanning in CI was generally meant to be passive; it wasn’t built for interactive testing and troubleshooting. Should a development pipeline hold the answers to complex questions that need to be answered when resolving a security error or compliance violations?

    Ask yourself who is going to get the notification when that build fails. Continue by imagining if your CI console is where you want to troubleshoot a misconfiguration. Will you have sufficient context to solve the kind of issues your scanner is going to raise? Will you be able to deduce the impact of a change you’ll make to pass a CI scan?

    In addition to the previously mentioned benefits of hinging your IaC security strategy on your VCS, there are more fine-grained controls in git such as:

    • Approval rules: With a VCS, you can define how many approvals, and a pull request needs before it can be merged. Some even let you select which specific users those should be.
    • Code owners: If you have admin access to your main repo, consider using CODEOWNERS parameters to define a stricter approval protocol for more sensitive files.

    Some of the more mature tools also support advanced functionalities such as importing custom rule files and applying suppression and skips as code annotations. Those should help you make sure that only relevant checks eventually crop up.

    Conclusion

    There are no one-size-fits-all strategies for infrastructure security, and we as an industry are still defining the best practices for embedding it into IaC. It’s up to every security team to work closely with their engineering and DevOps to make security consumable and, ultimately, to prevent risk as early in the development life cycle as possible.