devopsy.kinsta.cloud

Category: DevSecOps

  • Making SBOMs Actionable

    Making SBOMs Actionable

    A software bill of materials (SBOM) is a list of all the software components found in a given codebase or used in a given software build. Great. So, now what? Why do we even care about SBOMs?

    Those are great questions—because in and of itself, the SBOM doesn’t really do anything; it is simply a means to an end. That end being greater software security and a more secure software supply chain.

    Let’s take one step back to understand why there is such a sudden and urgent demand for SBOMs. The most recent ‘big bang’ attack on the software supply chain was the Apache Log4j flaw, discovered in December 2021. That vulnerability opened the doors to countless breaches and caused billions of dollars in damage. Before that, the high-profile hacking of SolarWinds caused similar damage.

    Now, will the SBOM prevent that from happening (again)? No. However, what an SBOM can do is tell you exactly where these kinds of vulnerabilities exist in software and enable you to quickly patch your systems or block exploits. Having a software bill of materials enables you to quickly assess the risks in your codebase and mitigate them as needed.

    It’s strongly recommended that you use dynamic SBOMs (versus static) that are automatically updated to the latest SBOM information and provide extensive search capabilities as part of an SBOM database. That allows for immediate identification of a specific dependency version, like the Log4j case. Some solutions also update the SBOM information based on the runtime (like Docker or Kubernetes). That way, you know not only if some component is used, but also where.

    SBOMs are an invaluable tool as they allow an audit of licenses, libraries, modules, applied patches and other components to look for weaknesses. However, for such an audit to reliably trace the components used to build the software, the SBOM must have certain attributes.

    • Unique identities for each software component
    • Separate identities (independent of those used internally in the organization) to identify each machine and user involved in the development process
    • Timestamps to facilitate traceability of each change or component incorporation

    Additionally, from a security point of view, a key element necessary for the reliability of the SBOM is to prevent unauthorized changes to it. And the most effective way to do this is by using an immutable ledger that records a history of when each change was made.

    Once the reliability of an SBOM is assured, DevSecOps teams can use it as part of their threat scanning toolbox and, therefore, improve software security.

    There is no doubt that SBOMs should be requested from your software vendors and that you should consider creating SBOMs along with your own developed software. It’s all about the proper storage of the SBOMs so you can be sure they’re recent, searchable and trustworthy and tamper-proof.

    The benefits and use cases for SBOMs are numerous; they vary across stakeholders who produce, choose and operate software and are amplified when combined. Use cases for SBOMs include better software development, supply chain management, vulnerability management, asset management and high assurance processes. The benefits include reducing cost, mitigating security risk, license risk and compliance risk.

    But the key is making the SBOM actionable.

    No developer, no software maintainer or DevOps engineer wants to manually collect the dependencies and produce SBOM documents. It needs to be fully automated within the software build and deployment pipeline and there needs to be a proactive check of where it’s currently running.

    The most efficient approach to work with a pervasive SBOM policy is to have them generated as a byproduct of a modern development CI/CD process.

    In today’s complex development environments, it is imperative that SBOMs can be shared frictionlessly between teams and organizations as a fundamental part of software management transparency. This transparency model is supported by the ISO association through its OpenChain Specification, and it is part of an effort to improve security in the digital industry.

    There are two SBOM specifications: The Software Package Data Exchange (SPDX) backed by the Linux Foundation and the CycloneDX specification managed by the CycloneDX Core working group. For more details, have a look here.

    Make sure to choose the SBOM standard that makes the most sense for you and stick to that for all of the software developed or used in your infrastructure. The test of whether your SBOM is actionable will come when the next Log4j moment arrives. You should be able to pinpoint the existence and the location of the component in question to mitigate risk effectively.

    To hear more about cloud-native topics, join the Cloud Native Computing Foundation and the cloud-native community at KubeCon+CloudNativeCon North America 2022 – October 24-28, 2022

  • JFrog Adds Module to Better Secure Software Supply Chains

    JFrog Adds Module to Better Secure Software Supply Chains

    JFrog today added a JFrog Advanced Security module to its Artifactory repository that enables DevOps teams to scan both binaries and source code for vulnerabilities and misconfigurations.

    Stephen Chin, vice president of developer relations for JFrog, said that approach will enable DevOps team to ensure applications are secure before they are deployed in a production environment.

    In addition to discovering vulnerabilities, the JFrog Advanced Security module also detects exposed secrets such as passwords, access tokens and private keys, said Chin. Leveraging existing scanning capabilities provided via the JFrog Xray software composition analysis tool, it can also be used to inspect the file used to provision infrastructure-as-code, he added.

    Most importantly, it provides development teams with the context needed to determine which components of an application are using containers that encapsulate code that is actually exploitable, noted Chin.

    One of the major issues DevOps teams encounter today is that they are often presented with a long list of vulnerabilities by cybersecurity teams—many of which don’t meaningfully impact applications that are not exposed to the Internet. DevOps teams then need to manually report why any given vulnerability doesn’t need to be remediated. The JFrog Advanced Security module helps automate that process, said Chin.

    In general, JFrog continues to pursue a binary-first approach to securing software supply chains, said Chin. While it’s important to address vulnerabilities in source code whenever possible, it is application binaries running in a production environment that cybercriminals are going to exploit, he noted.

    Cybercriminals have, in effect, opened a new front in their efforts to compromise IT environments by injecting malware into software components before they are deployed in a production environment. The hope is that, once deployed in those environments, malware can be activated at some later date. As a result, DevOps teams are now being required to improve cybersecurity hygiene across their software development processes, said Chin.

    It’s not clear at what rate organizations are embracing DevSecOps best practices to achieve that goal, but in the wake of an executive order issued by the Biden administration that requires federal agencies to better secure their software supply chains, the expectation is that more enterprise IT organizations will follow suit. The challenge is finding a way to build and deploy more secure applications without negatively impacting the rate at which they are built and deployed. Earlier this year, JFrog launched the open source Pyrsia project that employs blockchain technology to ensure open source software packages are not compromised by vulnerabilities and malicious code.

    In the longer term, it’s only a matter of time before the historic divide that has existed between DevOps teams and cybersecurity professionals starts to narrow as applications become more secure. The biggest immediate challenge is that many of the applications deployed in production environments already have known vulnerabilities. In the months ahead, many DevOps teams will be asked to assess the severity of those vulnerabilities as part of ongoing efforts to improve application security. Those patching efforts will undoubtedly have a significant impact on the amount of time available to write the code needed to build new applications or update existing ones with new capabilities.

    One way or another, however, the amount of technical security debt that has been allowed to build over the last several decades needs to be addressed.

  • Keeping the DevOps Pipeline Flowing as Attack Surfaces Grow

    Keeping the DevOps Pipeline Flowing as Attack Surfaces Grow

    The attack surfaces that today’s businesses and public entities must manage have never been more complex and difficult to protect. The introduction of cloud and SaaS offerings over the past decade has caused seemingly insurmountable IT sprawl, with a third of successful attacks now occurring via unmanaged or virtually unknown assets. And while shadow IT is an urgent threat confronting many organizations, the migration of the software development life cycle to the cloud—and the rise of machine identities—is compounding the attack surface in profound ways. Emerging out of the Agile and lean strategies for software development in the 2010s, DevOps predates the mainstreaming of cloud computing.

    But once the value of cloud-based IT resources came to be widely appreciated, development teams began moving their developer environments to the cloud in a big way—to the point where many people today think of DevOps and cloud as inseparable. Recent Gartner research showed that much of enterprise IT spending is now directed toward cloud projects. In fact, for IT categories like application software, infrastructure software, business process services and system infrastructure, more than 51% of IT spending will shift away from traditional solutions to the public cloud by 2025. Nearly two-thirds of spending on application software alone will be directed toward the cloud by 2025.

    The explosion in cloud DevOps has brought with it a huge uptick in the number of privileges, credentials, identities and access rights under management. We’re all familiar with how digital identity and access management (IAM) works in conventional settings through combinations of usernames and passwords. In the DevOps world, where cloud containers can run as distinct microservice processes interacting with other processes (machine-to-machine communication), identities and privileges are usually governed through SSL/TLS certificates. These machine identities have emerged as a critical security priority across multiple industries, not just because they’ve proliferated exponentially, but because of the lack of effective tools and strategies available to IT security teams.

    Machine Identities—Scope of the Issue

    A recent global survey of 1,000 CIOs found that the average organization has 250,000 machine identities under management coming into 2022, with that number expected to double by 2024. The study also found that 83% of organizations suffered a machine identity-related outage in the past year, with 57% of organizations experiencing at least one data breach or security incident related to compromised machine identities over the same time period.

    Because the perimeter defense solutions and strategies that formed the basis of IT security best practices for many years are of little use in cloud environments, security admins have had to rely on existing IAM and privileged access management (PAM) tools to secure data, accounts, and development pipelines. Not surprisingly, identities and access privileges have become the leading attack vector in the cloud, providing attackers with an exponentially higher number of vulnerabilities to target within the enterprise than were available even a few years ago. We’ve seen these points of weakness come into play in recent high-profile breaches and attacks such as SolarWinds, Codecov and PHP.

    Embracing Zero-Trust Principles to Secure the DevOps Pipeline

    Fortunately for today’s DevOps teams, cloud-oriented privilege management technologies have evolved rapidly in recent years, especially in the categories of cloud infrastructure entitlement management (CIEM) and time-based role access management or just-in-time privileging. CIEM solutions target the security weak points and process vulnerabilities that are unique to the cloud environments. These include the lack of deep visibility into user, group, and role privileges, which together make it difficult to conduct oversight and control over the activities of users within cloud infrastructure and applications. Where conventional approaches such as IAM and PAM are designed primarily to secure data, apps, and users within the organization, CIEM is designed to manage permissions risks to prevent cloud breaches and data theft.

    Emerging CIEM solutions are bringing to bear advanced capabilities in discovery, monitoring and auditing to significantly broaden visibility into DevOps processes for IT security professionals. With a CIEM solution, security teams can scan cloud applications for existing users, groups, roles and granted privileges down to the resource level. Each user, group, role, and privilege can be categorized to identify and verify that privileged users have the appropriate levels of access. Access reports can be generated to concisely outline users and groups with the most privilege and the highest risk.

    On the audit front, CIEM solutions provide the capability to track and report on levels of privilege as well as all activity associated with those privileges. In this way, it becomes possible to generate privileged access reports for consumption by internal and external auditors to ensure compliance with regulatory and other requirements such as SOX, GDPR, HIPAA, GLBA, PCI-DSS, ISO 27001, SOC 2, FedRAMP and more. Similarly, with the ability to monitor privileged access across all cloud applications, security teams can apply behavior analysis to identify privilege abuse and abnormal activities from users. As with conventional SIEM tools, CIEM solutions will generate alerts to administrators to suspect activities or behaviors—for instance, if a user attempts to circumvent policies preventing direct privileged access to cloud services.

    Just-In-Time Privileging and Zero Standing Privileges

    Importantly, CIEM solutions are also effectively eliminating standing privileges to cloud resources based on least-privilege access and zero-trust principles. The automated granting and expiring of permissions—just-in-time (JIT) privilege grants—is highly effective at minimizing attack surfaces. Because these JIT/zero standing privilege (ZSP) solutions work on a zero-trust model, no one and nothing is trusted with standing access to cloud accounts and data.

    With JIT permissioning, elevated privileges can extend either for the duration of a session or task, or for a set amount of time. Once the task is complete, those elevated privileges are automatically revoked–all without sys-admin involvement. Where a user previously had standing access privileges potentially extending around the clock for months at a time, converting to JIT granting would compress that attack surface to several hours per month. Further, JIT permissioning largely frees organizations from having to maintain and pay for both privileged and non-privileged accounts. Dynamic secrets generation – where a dynamic secret is generated on demand and is unique to a client, instead of a static secret being defined and shared ahead of time – also provides a better model for securing temporarily deployed services and features.

    DevOps and DevSecOps are still new and fast-evolving concepts within the wider computer science and cybersecurity universe. No doubt, DevOps has been wildly successful in accelerating automation and speeding time to market for innovative applications and business services. To date, however, security solutions providers have struggled with cloud access management. In other words, they have struggled to accelerate privileged access solutions that could secure the devices, data, and resources used by DevOps teams, especially in cross-cloud environments. Dynamic permissioning platforms using JIT privilege grants and employing ZSP principles show great promise in solving these problems.

  • Kill the Password: Google on Board | 4-Day Week Proves Worthy

    Kill the Password: Google on Board | 4-Day Week Proves Worthy

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Passkeys is getting another big-tech supporter, and the four-day workweek train picks up speed. (more…)

  • Implementing Data-Driven DevSecOps

    Implementing Data-Driven DevSecOps

    Right now, the way DevSecOps is typically implemented doesn’t fit with the rapid and agile DevOps CI/CD pipeline at all. It’s like applying 19th-century firefighting methods to a modern forest fire.

    Back then, firefighters employed a “bucket brigade,” where they would form a queue and pass buckets from one hand to another to put out a blaze. No doubt, it’s a lot of work, but much of the effort is wasted. Water inevitably spills out of the buckets as they are handed from one person to the next and by the time the bucket is emptied into the fire, half the water is gone. And not only is so much effort wasted, it’s far too slow and ineffectual to combat the kind of wildfires we face today.

    Likewise, the largely manual methods of current DevSecOps initiatives are ineffectual in fighting the fires of digital threats and cyberattacks modern mobile apps face. Like a modern megafire, these threats and attacks are growing and changing every second, seeking new vectors to spark new attacks elsewhere. Traditional DevSecOps tools like code scanning and penetration testing identify vulnerabilities and then the security teams start the manual “bucket brigade” to add as much protection as they can against them before the app has to be released. But neither the threats nor the CI/CD process has paused. New features have been added that have created new vulnerabilities, and the threat landscape has evolved. It’s inefficient because pentesting can’t provide the kind of real-time data about attacks and threats that developers need to provide protection against current threats. The result is the release of vulnerable mobile apps. 

    A Data-Driven Process

    In business, the C-suite is working hard to transform their companies into data-driven organizations where decisions are made not on the basis of gut feelings or expert opinion, but rather on analysis of hard data. That same approach needs to be brought to security implementation for mobile applications.

    Additionally, the implementation of security into a mobile app needs to be automated. With so much of the CI/CD process already automated, security cannot lag behind.

    These two elements combined create data-driven DevSecOps. In this method, the development and security teams have a system of record that provides real-time cyberattack and cyberthreat information about apps in the field, which drives decisions the team makes about the protections that are most urgent and that must be included in the next build. 

    It’s now well within the realm of possibility for mobile app developers to gather near-real-time information on exactly the kinds of threats and vulnerabilities that their mobile apps are facing in the field. When combined with location, network and other kinds of data, developers can gain a granular understanding not only of the most common threats their apps experience, but also which threats are most prevalent in specific geographic regions. They can also proactively identify the rapidly mounting threats that will become an enormous problem in the near future.

    With this data in hand, development teams can make informed decisions about which protections they should prioritize in the next build to make the best use of their time and resources to provide maximum protection to their end users. 

    The Need for Automation

    But having data isn’t enough. Development teams need the ability to act quickly on the insights provided, and manual implementation methods are too slow and cumbersome to keep up with the rapidly changing threat landscape for mobile apps. Once the decision is made, a system must also exist to automate the incorporation of mobile app security, anti-fraud and anti-cheat protections from within the CI/CD pipeline, so that security implementation runs as smoothly as feature creation.

    The advantages of data-driven DevSecOps are many. First, publishers, developers and security teams can understand exactly what cybercriminals are doing against their own apps with real users. With this data and the wealth of other data that organizations can collect from their apps, they can determine which threats are the most pressing, which are rapidly emerging and even how they are distributed among different geographic regions—all of which is extremely valuable information when deciding which protections to build into the next release.

    Finally, once the app is released, teams can use the data they receive about threats and attacks to prove the effectiveness and value of each protection that was included. It’s important information not only for continuous improvement, but also to justify the value of data-driven DevSecOps to management and the C-suite.

    With data-driven DevSecOps, organizations can provide data, transparency and visibility around security to all stakeholders in the CI/CD process—all while making the incorporation of security far more efficient and effective, along with the real-time data to prove it.

  • GraphQL: Security by Obscurity Just Isn’t Enough

    GraphQL: Security by Obscurity Just Isn’t Enough

    The debate about how to secure GraphQL rages on. Many organizations are hesitant to adopt GraphQL for public-facing APIs as there is no precise method to handle authorization concerns as of yet. Without a role-based access layer to enable fine-grained permissions for each field (and underlying services that GraphQL might wrap), the query language can be prone to access control issues.

    One method to hamper GraphQL access is to turn off introspection features altogether. Some may assume that hackers can’t perform introspection on an endpoint if it’s not visible. But this isn’t entirely true—even though an “internal” endpoint may not be fully documented, any endpoint on the public web can be searched and introspected quite easily by brute-forcing queries or through field suggestions.

    I recently met with Karim Rustom, cybersecurity engineer, Escape Technologies. According to Rustom, “making something obscure isn’t enough to make it secure.” Leveraging a single slip-up in access control could be used as a doorway into these systems. Although it’s a best practice to limit introspection, it’s likely not enough to protect production GraphQL endpoints, he said. Below, we’ll consider some common GraphQL threats and alternative solutions to retain a safe GraphQL implementation.

    Common GraphQL Threats

    Having run thousands of scans on production GraphQL endpoints, Rustom found common trends emerge around GraphQL threats. First, he noticed the potential for denial-of-service attacks to be quite high. GraphQL, by design, is intended to query whatever data the client requires. But sometimes, these queries can return a mountain of data.

    If GraphQL owners don’t place a limit on what can be requested in a single query, the platform can easily fall victim to denial-of-service attacks and service interruptions. And limits aren’t only necessary for bulk read-only operations—although GraphQL is primarily used for fetching data, some methods might initiate intense server-side processing, which should be limited in some form. According to Rustom, such data manipulation methods are especially pervasive in FinTech, food delivery or health care-related applications.

    Another rampant GraphQL threat lies in access control issues. This comes as little surprise, as access control is ranked as OWASP’s most common web application security risk. Often, when creating new objects or new queries, a developer might forget to add the correct authorization, said Rustom. This can become a huge problem and lead to privilege escalation.

    Finally, too many production GraphQL implementations rely on security by obscurity. “Folks think that disabling introspection is a layer of security—we highly stress that it’s not,” said Rustom. He recalled performing penetration testing for a certain client; even though they had disabled introspection, the team was able to fetch ample open paths and discover an unsecured doorway to divulge secret internal knowledge.

    Solutions and Best Practices

    To avoid some of the vulnerabilities presented above, it’s recommended to first place request limits on overly complex GraphQL calls to prevent denial-of-service attacks. One nuance here that Rustom recommended is to consider request complexity—not in terms of the query size, but the overall processing required. He suggested assigning scores for every query and permutation to help inform the decision about whether or not to process the incoming call.

    Furthermore, it’s recommended to disable field suggestions as the feature could be prone to abuse. Using field suggestions, hackers can leak a lot of information to build a partial or complete introspection. These tactics are similar to the word-busting techniques black hats use to perform introspection on REST endpoints.

    Although it’s a good practice to limit the possibility of surveillance, hackers can still discover systems and leverage other common vulnerabilities like broken access control. Rustom suggested other tactics, such as luring attackers into a honeypot or logging them into a fake account. Or, creating error messages that are intentionally misleading for a attacker.

    To retain a safe GraphQL implementation, organizations can use pre-made free and open source tools to audit their surface area and scan for known vulnerabilities. For example, GraphQL.security quickly scans your GraphQL URL endpoint to discover vulnerabilities. Or, Graphinder can examine a subdomain to discover GraphQL endpoints, which could aid auditing purposes. There’s also middleware that wraps a GraphQL service with additional security, such as GraphQL Armor and GraphQL Shield.

    Future of GraphQL in the Market

    There appear to be many flavors of GraphQL throughout the market. We’ve seen it emerge as a mechanism to power backend microservices. We’ve also witnessed the technology adopted as a layer to centralize multiple REST services with a more usable data retrieval mechanism for frontend engineers. Some GraphQL endpoints also run on the public web to serve end clients and external partners.

    “GraphQL is a beautiful technology, just lacking in security,” said Rustom. And unfortunately, his outlook for API security looks grim—amid political turmoil and economic instabilities in regions like Europe, some companies may seek to cut cybersecurity investments to stay lean. For companies that haven’t yet suffered a breach, security is easy to ditch to slim budgets, but it should remain a priority, said Rustom. “Keep securing GraphQL.”

  • Endor Labs Applies Graph Analysis to Secure Software Supply Chains

    Endor Labs Applies Graph Analysis to Secure Software Supply Chains

    Endor Labs exited stealth mode today to launch a platform that applies graph analysis to identify the depth of dependencies that exist within an application.

    Fresh from raising $25 million in funding, Endor Labs CEO Varun Badhwar said the Dependency Lifecycle Management Platform makes it simpler for organizations to manage dependencies within applications that can involve tens of thousands of components.

    Existing software composition analysis (SCA) tools are not able to as accurately identify the dependencies between software components because they generate too many false positives, he added.

    To address that problem, the Dependency Lifecycle Management Platform provides the equivalent of a credit risk score for each component employed with an application environment, said Badhwar.

    Designed to run out-of-band within a continuous integration process, the Dependency Lifecycle Management Platform is currently being made available via a private beta program.

    On average, bout 80% of an application is made up of open source components that developers downloaded from a repository. The challenge is that many of those components have known vulnerabilities. The Dependency Lifecycle Management Platform makes it simpler to both identify those vulnerabilities during the build process and find components that might be impacted by a newly discovered zero-day vulnerability, said Badhwar.

    Overall, the Dependency Lifecycle Management Platform improves the productivity of both application development and cybersecurity teams by streamlining vulnerability management, he added.

    Having a full understanding of their dependency graph also lets customers generate and analyze accurate software bills of materials (SBOMs) as applications are dynamically updated, noted Badhwar.

    In the wake of a series of high-profile breaches, there has been increased focus on software supply chains. The challenge is most developers don’t have a lot of cybersecurity expertise and, even when provided with tools to identify vulnerabilities, there are simply too many alerts generated. In the absence of any context, most of those alerts simply wind up being ignored. In the meantime, the rate at which new applications are being deployed and updated isn’t slowing down, especially as more organizations embrace cloud-native applications that make it easier to rip and replace software components.

    It’s going to take years for organizations to implement DevSecOps best practices to teach developers how to build more secure applications, but that journey needs to begin with tools that make it simpler for developers to address issues before applications are deployed. The best way to combat application vulnerabilities is to make sure they don’t manifest themselves in code in the first place. Based on the number of vulnerabilities that continue to find their way into application environments, it’s apparent that the current processes being used to build applications are fundamentally broken.

    At this juncture, it will take years to fix the application development and deployment process, but as more secure applications eventually replace millions of insecure applications the overall security posture of organizations will steadily improve. The issue now is finding ways to achieve that goal before another known vulnerability leads to another major security crisis.

  • Protecting CI/CD Pipelines to Secure the Software Supply Chain

    Protecting CI/CD Pipelines to Secure the Software Supply Chain

    Cloud-native applications have become increasingly complex. Composed of an infrastructure stack of mash-up code, hundreds of open source and commercial components, services and APIs, today’s software supply chain is fraught with security risks. At the same time, the ability to manage existing application requirements while building innovation and taking on new areas of responsibility like security is stretching DevOps teams beyond their capacity. The concern for the number of connected and exploitable weaknesses across the software supply chain are now requiring organizations to take a proactive approach to protect their continuous integration and continuous delivery (CI/CD) systems, as they have become a target through which attackers infiltrate multiple production systems. But meeting this requirement is no easy feat as it often requires organizations to choose between how much security to enforce in the build cycle while quantifying their risk to build and deploy quickly. 

    A recent survey revealed that nearly all (97%) of an organization’s assets—and security issues—are now in the cloud. And yet, only 29% of an organization’s assets are associated with cloud-specific policies. Addressing this gap requires organizations to have a proactive approach to protect their technology infrastructure, especially securing the artery of their software supply chain: The CI/CD pipeline. By embracing new security and compliance strategies within their governance approach, organizations can better protect their CI/CD pipeline and cloud applications. 

    There is a growing shift toward programmable infrastructure and infrastructure-as-code (IaC) practices to build out infrastructure and manage configuration files. But with this practice comes the increased risk of exposing application credentials, API keys, encryption keys and digital certificates. Tools supporting the CI/CD pipeline have now become the artery of cloud applications, and it’s becoming more challenging to store, transmit and audit data securely. In addition, when CI/CD tools interact with other systems in the DevOps environment, it increases the risk of exposure for configuration files. A compromise in the build system, for example, can be used to get access to production systems that can result in the exposure of sensitive information or the injection of malicious code, corrupting the entire CI/CD pipeline and thus the software supply chain. 

    A recent Unit 42 Cloud Threat report revealed that overly permissive credentials created added risk opportunities in the CI/CD pipeline. The study found that 63% of IaC templates contained misconfigurations and 91% of container images contained high or critical security vulnerabilities. To address this, organizations need to catch security problems in the build process before cloud workloads are provisioned. During the SolarWinds compromise, for instance, attackers used custom malware designed for the company’s build cycle to observe each step. The sophistication of the intrusion in this attack during the software build process is a key lesson for DevOps and DevSecOps professionals and revealed how security needs to be approached. 

    One way to address this is by being able to identify CI pipeline misconfigurations, which can improve security in the software supply chain. Because CI pipelines are configured in the code, the same policy-as-code approach can be used to identify IaC misconfigurations or open source vulnerabilities to surface CI/CD weaknesses. DevOps and developer teams can keep configurations accessible to just a small group who are building for production release. By establishing enforceable permissions to the pipeline, there is better control over who can commit code changes to the repositories, create containers and deploy code to different environments. 

    Strong chains of trust throughout the build cycle are not commonplace today. But as organizations look at ways to strengthen their risk posture by shifting left, security and DevOps teams can work toward proactively hardening pipelines with the right CI/CD tools and frameworks and push the industry toward a more secure software supply chain. 

  • Linux 6.0 is Faster, Cooler | Debian Goes Proprietary | Google Africa Region

    Linux 6.0 is Faster, Cooler | Debian Goes Proprietary | Google Africa Region

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Linux 6.0 promoted to Stable, Debian 12 will include closed-source binaries, and Google Cloud opens its first Africa region. (more…)

  • Driving Organizational Success With the 2022 State of DevOps Report

    Driving Organizational Success With the 2022 State of DevOps Report

    “Ship it!”

    We have just completed the research and analysis for the 2022 Accelerate State of DevOps Report (SODR) and oh boy, do we have some interesting things to share! First, let me take a short step back for those of you that may be new to the DevOps Research and Assessment (DORA) and our annual report, the SODR. DORA is an academically rigorous research program that seeks to answer the questions, “What practices enable teams to be high-performing software teams? And how do these practices impact organizational performance?”

    Since 2014, we have surveyed 33,000 practitioners around the world spanning all major industries. The project is intentionally–and staunchly–tool- and platform-agnostic. The research builds on itself, and each year we seek to both evaluate previous findings and expand in other directions or areas of research. This is important to note because many of the core findings, such as “software delivery performance drives organizational performance,” have been validated year after year. I would encourage you to go back and read all the reports from years past.

    Security

    This year, we doubled down on our 2021 research into software supply chain security by looking at the technical practices that improve software supply chain security and the non-technical practices that impact an organization’s ability to excel at securing their software supply chains. We leveraged two frameworks to focus our research: Supply Chain Levels for Software Artifacts (SLSA) and NIST’s Secure Software Development Framework (SSDF). Below is a summary of some of our key findings in this area:

    • Shifting left on security is a broadly adopted practice. Our research shows that two-thirds of respondents are actively pursuing software supply chain security by seamlessly baking security into the development process.
    • Culture is the primary driver of security practice adoption. One might expect that technology is the primary driver, but our research showed that a generative organizational culture (eg. performance-oriented, highly cooperative, shared risk) leads to healthier software practices.
    • Technical practices around CI/CD predict success in security. Companies that use source control, continuous integration (CI) and continuous delivery (CD) have more established SLSA practices. These practices shift security left to the developers and ensure consistent security scanning.
    • Cloud enables secure software practices. The five characteristics of cloud computing as defined by NIST enable the successful adoption of software supply chain security which, in turn, predicts greater organizational performance.

    Our research showed that companies who prioritize and excel at securing the software supply chain experienced fewer service outages, anticipate fewer security breaches and demonstrate high levels of both software delivery performance and organizational performance. The data showed that through the use of modern practices like continuous integration, teams can improve their security posture and even amplify the positive impact these security practices have on software delivery metrics (MTTR, deployment frequency, lead time-to-restore service) and overall organizational performance.

    Culture

    We saw above that the biggest predictor of an organization’s application development security practices was the presence of a generative organizational culture based on shared risk and information sharing. We also see that elements of these types of cultures lead to higher overall organizational performance, as well. Our research showed that high organizational performance can be achieved by fostering environments that are:

    • Supportive: Teams that felt supported and that they had buy-in from leadership (e.g., more financial support, more allocation of resources, sponsorships, etc.) were associated with high-performing organizations.
    • Stable: Teams that did not experience much change in membership over the last 12 months were more likely to be a part of high-performing organizations.
    • Flexible: Organizations with higher levels of working flexibility around where work was performed–remote, in-person or hybrid–were higher-performing overall.

    We also looked at burnout again this year and expanded the scope to understand which elements of culture contributed to lower levels of burnout. We found that generative culture, team stability and work flexibility all contributed to a reduction in burnout among employees.

    Reliability

    In previous years, our research told us that those that excelled in technical practices also excelled at organizational performance. This year, we have more nuanced data about this topic. This year, we saw that software delivery does not predict strong business outcomes unless these practices were paired with reliability. Think about it–will a customer be satisfied with new features if the service isn’t stable? What is the benefit of pushing code quickly into a fragile environment? Reliability is an essential component of driving organizational performance through software delivery performance.

    We also see that site reliability engineering’s (SRE’s) impact on organizational performance is non-linear; reliability engineering practices often do not result in additional reliability or organizational performance until a certain maturity is reached. It’s important for teams to know this and approach their SRE practice as an investment. It will likely not be sparkly ponies and unicorns initially as you build the reliability muscle but, as you advance, high performance and success are likely.

    Cloud

    Public cloud use is up a whopping 36% over 2021, whereas companies reporting no cloud usage at all are down by 50%. The use of hybrid cloud is up by 25%. Unsurprisingly, the use of cloud computing was associated with higher organizational performance. In past years we saw that it wasn’t “using cloud” per se that led to organizational performance, rather it was the achievement of the five essential characteristics of cloud computing–on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service. This year, we saw that cloud computing enabled things like reliability, continuous delivery and improved supply chain security which drive organizational success.

    Over 50% of folks who took our survey reported that they leveraged multiple cloud providers. We asked respondents what benefits they realized from using multiple cloud providers. Here are the top three:

    1. Availability
    2. Leveraging the unique benefits of each provider
    3. Trust is spread across multiple providers

    Given the fact that reliability seemed to be the key to being a successful software delivery shop as well as a successful organization, it is not surprising to see availability listed by almost 63% of respondents as a benefit derived from using multiple clouds.

    Summary

    Our research continues to progress and we continue to dig deeper and deeper into the capabilities and practices that impact your business. We see broad adoption on the topics of security and reliability which are both heavily anchored in culture. Good culture leads to success. Period. There is no magic culture button and it takes work, but it is very achievable. Start by defining organization-wide or line-of-business-wide goals and then just start continuously improving. Don’t worry about creating a three-year plan for improvement; create a one-month plan and just get to work instead. At the end of that month, evaluate your learnings, refine your focus and get back to work. With a commitment to hard work and continuous improvement, your investments should start paying off and, as an added bonus, you’ll have a better culture.

    We hope you have enjoyed these juicy findings as much as we have and we encourage you to read the entire 2022 State of DevOps Report.

    Also, please join us at dora.community to continue the discussion about these findings and to share and discuss your experiences on your journey toward excellent software delivery and operations.