devopsy.kinsta.cloud

Tag: Software Supply Chain Security

Harness Merges with Traceable to Provide Integrated DevSecOps Platform

Harness today announced that Traceable will be merged into the company to create a combined company that will further advance the adoption of best DevSecOps practices across the software development lifecycle (SDLC).

Created to focus specifically on application programming interface (AP) security, Traceable and Harness both trace their lineage back to BIG Labs, a startup foundry created by Jyoti Bansal, who also serves as CEO of both companies. Previously, Bansal founded AppDynamics, which was sold to Cisco in 2017. Going forward, the two companies will operate as Harness, with Traceable CTO Sanjay Nagaraj now heading up application security.

Nick Durkin, field CTO for Harness, said that as DevSecOps has evolved it’s become clear that software engineering teams are assuming responsibility for securing APIs alongside all the other artifacts that make up the software supply chain. As such, the need for a separate platform to solely focus on securing APIs becomes less of a requirement, he added.

The overall goal is to make it simpler to share critical security information with application developers that enables them to create more secure applications without adding responsibility for ensuring those tasks are completed, said Durkin. Instead, a DevSecOps platform should surface the right information a developer needs to know at the time when they are writing code, he added. Otherwise, developers are not going to have enough context to address an issue found in code they may have written weeks earlier, noted Durkin.

At the same time, cybersecurity teams should be able to define policies that are enforced by a DevSecOps platform that informs them whenever an issue arises, and then how it was addressed, said Durkin.

With the rise of artificial intelligence (AI), the volume code that might have a security issue that needs to be addressed before being added to a software build is starting to exponentially increase. Application developers are already struggling to keep pace with the volume of requests to create patches for existing applications. The addition of code written by machines is only going to further exacerbate that issue. There needs to be a platform that facilitates workflows in a way that in real time keeps all parties informed of security issues and how they are being addressed, said Durkin.

In fact, the melding of Harness and Traceable is in many ways the latest substantiation of a need to embrace platform engineering as a methodology for meaning DevSecOps workflows at scale, he added.

It’s not clear to what degree the need to better secure software supply chains is driving organizations to replace their existing DevOps platforms, but Harness has been making a case for an integrated platform that makes it simpler to apply artificial intelligence (AI) across the SDLC, including the ability to enforce best application security practices.

The challenge, as always, is empowering all the stakeholders involved in a way that doesn’t make it too hard for application developers and software engineers to do the right thing by, for example, reusing a script that has already been vetted for cybersecurity issues, said Durkin. Otherwise, individual members of a team will conclude it’s simply faster to write yet another script that they perceive as being the fastest way to achieve their end application deployment goal, he added.

Regardless of DevSecOps maturity, the one thing that is certain with more attention than ever being paid to software supply chain security, it’s now more a question of when, rather than if, existing software engineering workflows will be finally revamped to fully address these longstanding concerns.

February 10, 2025
GitLab Fixes Security Flaw That Lets Attackers Run Pipeline Jobs

GitLab is urging users to upgrade the latest versions of GitLab Community and Enterprise editions, which fix a critical security flaw that allow bad actors to run pipeline a pipeline as another user.

The widely used web-based Git repository this week released versions 17.1.2, 17.0.4 and 16.11.6, which included a range bug and security fixes in both editions. The other vulnerabilities were rated as medium or low in their severity levels, but one – tracked as CVE-2024-6385 – was given a CVSS rating of 9.6 out of 10.

The security flaw affects Community and Enterprise edition versions 15.8 prior to 16.11.6, 17.0 prior to 17.0.4, and 17.1 prior to 17.1.2. The vulnerability “allows an attacker to trigger a pipeline as another user under certain circumstances,” Rohit Shambhuni, staff application security engineer at GitLab, wrote in a security notice.

Shambhuni didn’t detail the circumstances needed to allow such an attack.

However, Contrast Security CISO David Lindner said this vulnerability is something administrators need to take notice of, and heed GitLab’s advice to upgrade immediately.

“This is REALLY bad, as it effectively turns off access controls for running pipelines, which is the lifeblood of moving software from development to production,” Lindner wrote in an email. “This vulnerability could allow unauthorized users to execute pipeline jobs as any other user, which in turn could enable attackers to run malicious code, access sensitive data and compromise software integrity.”

Ray Kelly, a Fellow at Synopsys Software Integrity Group, said that given the fast pace in the SecDevOps world, “any mention of a vulnerability in pipeline functionality can certainly make the hairs on your neck stand up. Once a pipeline is compromised, software can be altered with malware, backdoors, or used to steal private information from organizations.”

All of that is difficult to detect, with security scans typically run earlier in the software development lifecycle, Kelly said. With recent supply-chain breaches, organizations need to patch flaws immediately to keep bad actors from compromising their software. In addition, running security scans in the pipeline can help detect issues before they’re deployed.

A Foundation for CI/CD

GitLab pipelines are key parts of a developer’s autonomous continuous integration and continuous deployment (CI/CD) process, driving software development through its multiple steps, from building to test to deploying code. According to GitLab, “Pipelines are comprised of jobs, which define what will be done, such as compiling or testing code, as well as stages that spell out when to run the jobs. An example would be running tests after stages that compile the code.”

They’re important for many of GitLab’s 30 million users, which include such high-profile companies as Lockheed Martin and Boeing in the aerospace field, French multinational Thales, global management consulting service Booz Allen Hamilton and General Dynamics Information Technology, an IT service management firm.

The fix for the security flaw comes less than a month after GitLab released new versions for Community and Enterprise editions to fix a similar vulnerability, which is tracked as CVE-2024-5655 and also allows an attacker to run a pipeline as another user. That flaw also came with a CVSS severity rating of 9.6.

Code Repositories as Targets

The push to rapidly such security flaws is driven in part by the increasing focus cybercriminals are taking to target code repositories like GitLab, GitHub, Python Package Index (PyPI), npm and – in the ever-expanding generative AI space – Hugging Face.

Such repositories not only hold a lot of information, but they’re an avenue for bad actors in software supply-chain attacks, who try to insert unauthorized code into the CI/CD pipeline to have it included in applications that then reach downstream IT environments. This can have wide-ranging effects, as shown by the SolarWinds incident from 2020, when hackers deployed malicious code into the company’s Orion IT monitoring and management software.

“Code repositories have become a prime target for threat actors targeting credential theft and supply chain attacks,” according to a blog post in March by cybersecurity firm SentinelOne. “Git repositories can contain not only proprietary code but also sensitive credentials like API keys, passwords, and cryptographic keys.”

Researchers with cybersecurity firm Apiiro in a report in February wrote about a repo confusion attack that affected more than 100,000 GitHub repositories that played out “when unsuspecting developers use repositories that resemble known and trusted ones but are infected with malicious code.”

Pushing for Secure Development

CISA and other federal security and law enforcement agencies have been urging organizations and programmers to focus on security in every step of the development, a cornerstone of the White House’s Secure By Design initiative. It’s an effort aimed at not only creating more secure code but also shifting the responsibility for software security from customers to developers.

As part of the program, CISA and the FBI this week issued an alert pushing tech companies to eliminate command injection flaws in operating systems, which enable threat actors to remotely execute code on devices at the network edge.

“OS command injection vulnerabilities have long been preventable by clearly separating user input from the contents of a command,” the agencies wrote. “Despite this finding, OS command injection vulnerabilities … are still a prevalent class of vulnerability.”

July 12, 2024
Why DevOps is Key to Software Supply Chain Security

DevOps is designed to increase business value and responsiveness through rapid, high-quality service delivery, made possible through fast-paced, iterative IT service delivery. The term DevSecOps emerged as a way to remind us that security was always intended to be part of the software delivery process. However, with the barrage of attacks on the software supply chain showing no signs of abating, the question now is whether even the tightest DevOps pipelines can maintain the pace-protection balance.

It’s important to note that the term “software supply chain” has been used primarily to refer to the tools and processes used to build and deploy software applications. Now, however, with the rise of infrastructure-as-code (IaC), the term also refers to tools and processes used to build and deploy IT infrastructure.

Meet Mandates, Reduce Risk

Software supply chain attacks such as SUNBURST, HTTP/2 Rapid Reset or the breaches associated with the MOVEit vulnerability have fed news cycles for the last few years, and that likely won’t change anytime soon. Attacks on the software supply chain increased 742% since 2020. What we are seeing as a result are new concerns about supply chain security and DevOps requirements. This has led to confusion and often a mish-mash of tooling that may exacerbate rather than mitigate supply chain security issues.

Arguably the most impactful salvo over the software supply chain security line was president Biden’s 2021 Executive Order 14028, which established new mandates, including the delivery of SBOMs, as minimum requirements for software used by the U.S. federal government. More recently, the FDA has mandated that “all medical devices running software must create and maintain” SBOMs. I expect to see government and industry requirements increase over time.

These are just a couple of relatively recent examples. There are many more—and no doubt many more to come.

So, if you’re moving at the speed of DevOps, how do you address these current and future mandates for software supply chain security, as well as the continually emerging vulnerabilities and attacks? I would venture to say that only by moving at the speed of DevOps—and within the parameters of the DevOps model—can organizations not only meet mandates but also effectively reduce risk for the business.

Indeed, it’s not a matter of whether DevOps is viable within the current security climate but what requirements need to be added to support and enhance current DevOps work.

New Guidelines, New Processes, New Tools and New Data

In response to these new threats and regulations, the software community has been working on developing and updating guidelines to help organizations improve the security of their supply chains—for example, SLSA, NCSC UK supply chain security guidance and NIST cybersecurity supply chain risk management. However, these new guidelines can be complicated and require new processes and new tools.

Before adding any new tooling or changing any processes, it’s critical to examine existing tools to determine whether and to what extent they satisfy supply chain security mandates. Organizations must also evaluate how well existing processes work to protect the business, then strategically add/subtract from there as needed.

No matter what solutions are leveraged, more and different tools generate reams of more and different data. What’s important — and to whom? How do I manage the data? When can I trust it? Where do I store it? What problems does the new data help me solve? Organizations will need a way to effectively sift this information and deliver the right data to the right teams at the right time. To preserve the ability to quickly and continuously innovate, it will be important to focus on shifting security left as well as integrating automation whenever and wherever possible.

As new security metadata becomes available, such as from SBOMs, new solutions for managing that metadata will be key. An open source initiative sponsored by Google, GUAC is designed to integrate software security information, including SBOMs, attestations and vulnerability data. Users can query the resulting GUAC graph to help answer key security concerns, including proactive, preventive and reactive concerns.

Continue to Shift Security Left

Another way that organizations can maintain their DevOps momentum while protecting the software supply chain is to continue to invest in shifting security left in the software supply chain. The software community has been talking about this for some time, but many organizations struggle to implement impactful changes. Security teams must work to understand how software is built and collaborate with development and DevOps teams to ensure that improvements are made to both application and infrastructure-as-code supply chains.

We need to go beyond shift-left vulnerability analysis to ensure that we are evaluating the quality of configuration-as-code (for both applications and infrastructure), that we are taking advantage of new recommendations, and that we are leveraging new attestation solutions designed to work with pipelines such as sigstore and Tekton Chains. After all, the sooner an issue is discovered, the more quickly it can be addressed and the less disruptive and costly it is to fix it.

No One-Size-Fits-All Solution

While all of these solutions mentioned here are helpful, there is no single product or framework that will enable organizations to maintain the security-innovation balance. Success, rather, will require a focus on agility, flexibility and effective communication and collaboration—the very hallmarks of the DevOps model.

Rather than being obviated by the demands of software supply chain security, DevOps provides the opportunity to integrate the kinds of protections that can prevent or mitigate supply chain vulnerabilities and breaches. DevOps also enables organizations to quickly and effectively respond to new threats and new requirements.

With that said, there’s no doubt that organizations will need to think differently about how they manage software security, including the strategic evaluation and integration of tools that will meet current and future regulatory mandates and, more importantly, protect the business over time. Organizations may also need to think about weighing security over speed, at least until all of the pieces of the software supply chain protection puzzle are in place—but also when future circumstances warrant a fresh look at how to best balance the DevOps pace-protection balance.

February 21, 2024
Techstrong Research: Combatting CI/CD Security Anti-Patterns

Recently, Techstrong Research released our PulseMeter report, “Tackling CI/CD Security Anti-Patterns.” Our research shed light on the evolving landscape of continuous integration/continuous deployment (CI/CD) pipeline security, drawing from the collective wisdom and experiences of professionals across DevOps, DevSecOps, software development and security disciplines.

Our aim is not merely to explore the challenges but to provide guidance toward actionable steps and best practices that can fortify the backbone of modern software delivery processes. The findings of our research underscore a pressing reality: The security of the software supply chain and CI/CD pipelines is now as critical as the integrity of physical supply chains in traditional sectors.

This comparison is not made lightly; as digital infrastructure becomes the backbone of global commerce and communication, the repercussions of security lapses in software delivery can have widespread and consequential impacts. Our study reveals that the risks to software supply chains demand rigorous attention and strategic action, echoing the necessity for a shift in how organizations approach DevOps and CI/CD security.

A significant portion of our research was dedicated to identifying CI/CD security anti-patterns—practices that, albeit unintentionally, may provide a false sense of comprehensive security and even introduce vulnerabilities into development pipelines. These anti-patterns, such as the incorporation of external code from unverified sources or the adherence to rudimentary security measures without advancing to more robust strategies, significantly broaden the attack surface accessible to cyber adversaries. The research emphasizes the criticality of recognizing and rectifying anti-patterns to help safeguard the software supply chain from potential breaches and attacks.

Insightful Survey Findings: A Wake-Up Call

The survey provides a stark illustration of the current state of CI/CD pipeline security. With 84% of respondents acknowledging the importance of CI/CD pipeline security—motivated by the sobering statistic that over 20% of organizations reported a security incident within their pipelines in the previous year—the message is clear: Pipeline security must be improved beyond current measures.

These incidents showed tangible impacts, including but not limited to delayed deployments and significant data breaches. Moreover, the survey identified source/image repositories and code as particularly vulnerable points within the software development life cycle (SDLC), with a significant emphasis on the risks posed by open source vulnerabilities, secrets exposures and insecure code. This feedback from the field underscores the necessity for a proactive stance on security measures, one that anticipates potential vulnerabilities and implements robust defenses against them.

Drawing from the insights garnered through our research, Techstrong Research advocates for a holistic, strategic approach to enhancing CI/CD pipeline security. This includes the deployment of defensive measures designed to prevent unauthorized tampering with software production processes and the rigorous vetting of CI/CD pipeline artifacts to ensure their integrity through comprehensive role definitions and strict authorization protocols. Our endorsement of the NIST SP 800-204D draft underscores our belief in the necessity of a foundational framework that organizations can adopt to bolster their CI/CD pipeline security. Furthermore, our research shines a spotlight on the pivotal role of software composition analysis (SCA) in fortifying CI/CD pipelines against security threats. SCA facilitates the meticulous tracking of software elements and their dependencies, enabling organizations to identify and mitigate vulnerabilities swiftly. This tool is invaluable not only for preventive security measures but also as a critical component of incident response strategies.

Beyond traditional security measures, the research highlights the use of cloud-native continuous integration approaches, including containerization, to enhance security. These methodologies provide a framework for executing builds and tests within isolated environments, thereby minimizing the risk of conflicts and vulnerabilities. The adoption of containerization offers a myriad of benefits, including standardized packaging, enhanced transportability and improved traceability of software from development through to production.

The PulseMeter report on CI/CD security anti-patterns serves as a clarion call to the industry. The imperative to secure the software supply chain and CI/CD pipelines is undeniable and urgent. In this era of unprecedented digital reliance, our collective commitment to advancing CI/CD pipeline security is not just beneficial—it’s essential. The journey ahead is challenging, but with informed strategies and collaborative efforts, we can significantly reduce the vulnerability of the software we deliver as part of our digital infrastructures.

February 21, 2024
Checkmarx Report Details Havoc Caused by ‘Everything’ Package on NPM Registry

Checkmarx this week released a report detailing a package distributed via the NPM registry that cripples any machine that is used to download it.

Dubbed the “Everything” package, a user known as PatrickJS, also known as gdi2290, uploaded a package that has more than 3,000 sub-packages. Each sub-package is designed to split dependencies into chunks and to depend on all publicly available NPM registry packages, resulting in transitive dependencies that disrupt pipelines and exhaust storage space.

Following significant effort, The Everything package has since been removed from the registry, but the perpetrators did create a domain where they showcase their handiwork. Whether the Everything package was created as a joke to be played on developers or has some other malicious vandalistic intent is unclear.

Jossef Harush Kadouri, head of software supply chain security for Checkmarx, said these types of incidents are likely to happen from time to time because there is no process for reviewing packages prior to them being loaded into a registry. In fact, a similar incident previously occurred when a “no-one-left-behind” package by Zalastax that was dependent on every publicly available npm package was uploaded in the registry.

It’s only when an issue is identified that the maintainers of the NPM registry will remove a package. Those maintainers simply lack the resources and tools that would be required to scan every package, noted Harush Kadouri.

Heading into 2024, it will be incidents like these, along with vulnerabilities found in NPM packages, that will serve as an impetus for reviewing the workflows that make up an organization’s software supply chain. For well over a decade, developers have been downloading software components from various repositories with little to no regard for vulnerabilities. As more of those vulnerabilities are exploited, organizations that rely on that software are now requiring providers of applications to provide visibility into the provenance of the components used to construct it. The overall goal is to make it simpler to remediate applications, especially when a zero-day vulnerability is discovered.

In the meantime, organizations would be well-advised to curate a repository of software components that have been vetted. Developers can still readily access those components to build applications, but there should be fewer downstream application security issues if the components residing in that registry have been scanned. If developers are allowed to directly download packages and components from a public registry, it’s only a matter of time before an application security issue is going to arise.

Hopefully, as organizations adopt DevSecOps best practices, the number of application security issues encountered will steadily decline. Each organization will need to determine how far left to shift responsibility for application security toward developers, but one way or another, software supply chains need to be better protected.

Of course, the issue will ultimately be forced by regulations that are becoming increasingly more stringent. Once the first fines are levied, it won’t be long before just about every organization that builds and deploys software will quickly move to limit their liability as much as possible. The challenge and the opportunity now is to proactively secure software supply chains today before any remedy that might be imposed on an organization disrupts application development workflows tomorrow.

January 5, 2024
GitGuardian Adds Tool for Discovering Secrets in Public Repositories

GitGuardian today added a tool that makes it possible for DevOps teams to search GitHub repositories to determine if application secrets have inadvertently found their way into other applications.

Vikram Ghosh, chief growth officer for GitGuardian, said a HasMySecretLeaked toolset makes it possible to determine if developers have copied and pasted application secrets from one repository into another without permission.

The toolset is based on a private database that houses more than 20 million records of hashed secrets that have been found in public sources, including GitHub.com. Users can query the database by submitting a hashed version of their secret in the search console. The GitGuardian toolset will then search for perfect matches without revealing any other secrets or their locations, said Ghosh.

Alternatively, DevOps teams that have adopted the GitGuardian platform to protect secrets can use a command-line interface (CLI) exposed via a ggshield tool to launch queries. That tool also includes plug-ins for pulling secrets from tools such as HashiCorp Vault and AWS Secrets Manager and staging them in local environments to surface potential leaks. Eventually, HsMySecretLeaked will be extended to add support for additional repositories, noted Ghosh.

Collectively, those capabilities make it possible for organizations to audit how application secrets are being secured, he added.

The GitGuardian platform scans every public commit on GitHub for leaks, spanning API keys, database assignments and developer secrets. In 2022, GitGuardian found 10 million secrets that had been exposed.

Secrets management is, of course, getting more attention as more organizations review their software supply chain processes in the wake of a series of high-profile security breaches. It’s still early days as far as adopting DevSecOps best practices to secure those software supply chains, but it’s clear more responsibility for application security is being shifted left toward developers and the DevOps teams that support them.

The challenge is that many developers still hard-code secrets in plain text into applications to create shortcuts as software is developed. Unfortunately, many of them forget to remove those secrets before an application is deployed in a production environment. Once an application is deployed in a production environment, it may be months before anyone discovers there is an issue—if it’s discovered at all. In the meantime, cybercriminals have become more adept at using scanning tools to discover those secrets.

DevOps teams can reduce the number of applications that might have this issue by making it easier to discover secrets within applications before they are deployed. In the meantime, it’s only a matter of time before more stringent regulations require organizations to revisit the security of software supply chains. Countries around the world are debating legislation that would hold organizations much more accountable for the security of the applications they build and deploy.

Hopefully, DevOps teams will implement DevSecOps best practices to get in front of those issues before any fines are levied. But the challenge, as always, is ensuring application security without adversely impacting the rate at which software is currently being built.

October 17, 2023
Summit Highlights Open Source Software Security Progress

More than a year after the Open Source Security Foundation (OpenSSF) summit kicked off an initiative to better secure open source software-based software supply chains, a lot of progress has been made—but much work remains to be done.

The OpenSSF this week hosted a Secure Open Source Software (SOSS) Summit 2023 event in Washington, D.C., during which it made available a Secure Open Source Software Vision Brief 2023. The brief describes the various efforts being made to improve open source software security, including, for example, providing maintainers of open source software projects with free DevSecOps tools.

This week, the Cybersecurity and Infrastructure Security Agency (CISA) also published an Open Source Software Security Roadmap that defines its role in providing more visibility into how open source is being used and the associated risks.

However, it’s not clear how much maintainers of these projects are embracing DevSecOps workflows. Many projects are led by a handful of maintainers that often lack the time and skills required to ensure the components they employ have not, for example, been compromised by malware that might find its way into downstream applications.

Sonatype CTO Brian Fox said the focus of the summit’s participants will be on education and securing public code repositories. While there is a lot of focus on maintainers, he added that there is not enough attention to all the organizations that have not updated open source components within a software build with known vulnerabilities. Many IT teams are reluctant to upgrade components for fear of breaking applications with complex dependencies, but the amount of code that is potentially insecure is massive. “There’s a huge mountain to climb,” he said.

The challenge is determining which code to upgrade first in a way that will have the most impact in terms of improving application security, added Fox.

Shawn Ahmed, chief product officer for CloudBees, also noted that organizations should evaluate the DevSecOps processes being applied by maintainers of open source software projects. CloudBees, for example, reviews the plugins provided by the open source community for the Jenkins continuous integration/continuous delivery (CI/CD) platform for vulnerabilities every month, he noted.

It’s also critical for organizations that consume open source software to contribute code to improve the security of open source software, added Ahmed. Too many organizations are focused on open source software security without devoting resources that would help resolve the issue. he added. “We need to make sure open source software is not a vulnerability threat,” said Ahmed.

Moran Ashkenazi, chief strategy officer (CSO) and vice president of security engineering for JFrog, said on the plus side, there is a lot more awareness of open source software security issues since the disclosure of a zero-day vulnerability in the Log4j shell tool widely used to manage logs in Java applications. But added that this issue is not going to be resolved overnight. “That was a wakeup call,” she said.

Tapabrata Pal, vice president of architecture for Fidelity Investments, noted that the fundamental problem is that cybercriminals have become more adept at injecting malware into software supply chains, so no one knows whether their application environments might be compromised. “It’s a serious issue,” he said.

It’s not known whether the open source software that many developers routinely incorporate into their applications might contain vulnerabilities that could be exploited, but the level of technical debt that has accrued over the last several decades is considerable. The issue is finding a way to address it without becoming overwhelmed by the enormity of the task.

September 14, 2023
Stop Leaking Secrets!

Merriam-Webster defines a secret as something kept from others or shared only confidentially with a few. Modern software development teams use software secrets such as passwords, tokens and API keys to properly set up, protect and maintain development environments and delivery pipelines and to enable programmatic access to services such as AWS Buckets or Azure Blob Storage for cloud-native apps. Yet, while secrets are critical for modern software to function, they are difficult to manage across delivery stages. All too often, software teams trip over the complexities and inadvertently leave these secrets exposed in both public and private software repositories.

Potential exposures can stem from many factors: Plain text passwords, weak cryptography, flawed CI/CD or packaging automation and compromised developer accounts or malicious insiders. Whatever the cause, bad actors are regularly scouring repositories and application binaries for hard-coded secrets they can exploit. Look no further than the CircleCI and CodeCov incidents, where exposed secrets left these organizations vulnerable to adversaries intent on attacking software supply chains.

Rethink Secrets Prioritization

According to GitGuardian analysis, 10 million new secrets occurrences were exposed on GitHub in 2022, a 67% increase compared to the company’s 2021 findings. As software becomes larger and more complex–with open source, third-party and commercial components–the number of detected secrets will only increase, creating a different challenge: Prioritization for remediation.

Regrettably, teams lack a scalable means to understand which detected secrets developers can’t or shouldn’t remediate. Today, teams must manually sift through potentially thousands of detections and understand their context to triage these threats. Gaining context means discovering where secrets reside (in your code, in a software dependency, in the documentation included with your software) and understanding their purpose (to access resources, to enable user testing, to enable intrusion detection).

As teams consider how to simplify and automate the prioritization of secrets, important questions to consider include:

● What secrets are your developers responsible for protecting? For example, secrets used to access internal CI/CD pipeline technologies. Depending on permissions, leaking these secrets enables malicious actors to change automated build processes with no one being the wiser. Similarly, API credentials to access data or services needed for their code to function. Leaking these secrets gives malicious actors programmatic access to those APIs, which can be a gold mine of customer data or further compromise the environment.
● What secrets are worth protecting? There are some you may be responsible for but which are not worth protecting. For example, to facilitate automated software testing, developers typically publish API keys and certificates in public repositories alongside their applications or in documentation bundled into release packages or software containers. Since these testing “secrets” are designed to be commonly shared, they should not be encrypted or protected.
● Which detected secrets are not actionable? For example, developers typically cannot manage secrets in third-party or commercial components and dependencies within software. Instead, these exposures represent a third-party risk management problem. Another example is canary secrets added to software by security operations; remediating these secrets hampers intrusion detection.

Once teams have established their secrets definitions, building a process to identify and prioritize them comes next.

Managing Secrets

Any organization involved in software development should have clear rules for handling and incorporating secrets into code and for communicating that process and those rules to developers. Here are secrets best practices every organization should take into consideration:

● Promote developer best practices — and monitor compliance: Secrets leaks are more likely to occur in development organizations that lack clear guidelines for properly managing secrets in their code and easy-to-use tools to implement those guidelines. DevOps teams should formulate policies based on an understanding of how secrets are generated within their development environment, how those secrets are protected once deployed and how (hopefully) they are eventually cycled out of use.

● Revamp how applications use secrets: Design applications in a way that eliminates the reliance on access tokens that reside in code. Use technologies like OAuth OpenID Connect (OIDC) tokens that allow developers to authenticate users without directly exposing user credentials in code. Reduce the risk of abuse by using multi-factor authentication, leveraging identity and access management platforms and setting IP ranges that limit inbound connections for APIs to known addresses or address ranges.

● Establish end-to-end security: Misconfigurations in CI/CD tooling or release oversight can result in the inclusion of secrets during final builds that might otherwise escape notice during a code audit. Consider implementing binary analysis to extend visibility into leaked secrets beyond raw code and into software packages and containers ready for release. This capability can detect exposed secrets resulting from build or packaging automation.

● Streamline secrets prioritization: Noisy scanning results create a ton of triage work just to weed out false positives (e.g., widely distributed testing keys or network credential placeholders) before remediation can start. Automating some of this triage with relevant file intelligence saves countless hours of effort and ensures developers focus on addressing actual risks.

No Silver Bullet

Unfortunately, there is no “silver bullet” fix for software supply chain risks and attacks, nor for leaks and exposures. Instead, organizations need a holistic approach that includes scoping the problem, addressing short-term risks posed by leaked credentials and then extending changes in all processes with an eye toward taking the toil out of identifying, prioritizing and remediating secrets leaks—and therefore preventing potential supply chain attacks before they can happen.

September 6, 2023
I Guess This is Growing Up: Devs and CISA’s Secure-by-Design Guidelines

With the downward pressure of a global recession, inflation and general post-pandemic turbulence underpinning disruption to multiple facets of life, it seems only fair that we in the IT, software and security industry would eventually feel the winds of change, too. We have rolled with the punches of escalating cybercrime and data breaches; the collective, fragmented sheriffs of a digital Wild West, and it’s high time we updated our strategy for safeguarding software against bad actors who seek to do widespread harm.

The recently released National Cybersecurity Strategy signals the need for a seismic cultural shift for most companies and their developers and DevOps teams. The most glaring recommendation comes in the form of security accountability that falls primarily on software vendors. This is a positive step, though it is sure to cause teething problems, especially as many organizations struggle to accurately assess their security maturity across the board, particularly among the development cohort.

Since CISA Director Jen Easterly’s trailblazing speech on secure design principles, we have eagerly awaited formal guidelines on how, exactly, organizations can approach this significant restructuring of software development as we know it. Thanks to a collaborative effort between the cybersecurity authorities of seven countries, we have a rather comprehensive overview of how to drive down vulnerabilities in software.

This should be cause for celebration, but understandably, there is hesitation about how this will impact innovation, feature delivery and, ultimately, the bottom line as the pressure of security stewardship mounts for vendors. We’ve grown accustomed to dealing with cumbersome tooling that is ill-fitted with developer workflows, overwhelming them with a list of fixes in an information avalanche. Add SCA and SAST scanning, and you’ve got thousands of identified issues in a codebase that are just not practical to solve.

Despite this, in terms of our collective attitude towards code-level security, is it time we grew up and faced the music?

Security Commitment and Transparency Boost Our Potential

Back in March, the White House announced that one of its top priorities in the National Cybersecurity Strategy was to shake things up and start holding entities liable for sloppy software security measures:

“Companies that make software must have the freedom to innovate, but they must also be held liable when they fail to live up to the duty of care they owe consumers, businesses or critical infrastructure providers,” the White House noted in the strategy.

In the era of “speed at all costs” software development, my opinion may be unpopular, but I truly stand behind the idea that coding with security front-of-mind from the beginning is the only way we safely create our digital future. We were already facing insurmountable challenges with data breaches, nation-state attacks and infrastructure disruption, and that was before code-capable AI tools hit the mainstream. Until every person touching code cares more about the security implications of their actions, we will continue to see cybersecurity issues surge at a rapid rate.

I know this is easier said than done. I have a lot of empathy for those standing at the bottom of this steep mountain trying to change the conversation around security accountability and enable developers in their organizations to share responsibility. To adequately fulfill key secure-by-design principles in software development, it is going to require precision training that establishes a basic skillset before deep-diving into more advanced security architecture methods. Developer-led threat modeling is a golden opportunity for the AppSec and development teams to work together toward a common goal and explore access control and insecure design pitfalls that plague less mature organizations.

Still, the core issue has always revolved around a lack of ownership for security (until something goes wrong—then it’s the AppSec team left holding the bag) coupled with a low appetite for best practices to be established and executed across every part of the SDLC. The truth is, it was never acceptable for developers to start coding with so little security expertise, nor for us to focus on speed and functionality at the expense of safety and quality. Yet, here we are, and this is our crucible moment to accept shorter-term pain for longer-term ongoing success.

We should want to create the most secure, robust software possible, and the “radical transparency” called for by Jen Easterly is a boon to vendors and clients alike. It is definitely a mark of value to showcase the company’s security prowess; brand trust and reputation are everything in this hyper-competitive landscape.

Developers Need Guidance and Support to Thrive in a Secure-by-Default Environment

The CISA guidelines pay particular attention to open-source and third-party components, both of which are widely used in most software development projects. It is imperative that developers are sufficiently security-aware to navigate potential inherent bugs with any preexisting code they use. However, in addition, they must be able to apply working knowledge of security fundamentals in the languages and frameworks they actively write in every day. Annual compliance videos played at 2x speed won’t cut it; they need training and a tech stack that is hyper-relevant, easy to access without context-switching from their workspace and that ultimately drives them to apply secure coding best practices as second nature.

They cannot do this alone, and they can’t be expected to upskill in their own time. The cultural shift required to make the development team a fundamental cog in a thriving security program takes courage, time and commitment. But the reward is cohesion between AppSec and developers that would have been virtually impossible a decade ago and software that stands above those who are, in one way or another, happy to let security slide despite the risks.

I guess this is what growing up looks like for all of us in the software industry, and it’s crucial if we want to stop the same old vulnerabilities that have plagued us for so long that they are old enough to drink.

May 31, 2023
The Role of SBOMs in Software Supply Chain Security

The software supply chain has become increasingly complex and dynamic with the rise of cloud computing, open source software and third-party software components and APIs. Widespread damage can occur if third-party APIs, cloud services, SDKs and open source software have security flaws. As a result, software supply chain security has emerged as a critical concern for organizations across industries. Supply chain attacks have become more frequent, sophisticated and devastating, as evidenced by high-profile incidents such as SUNBURST, Kaseya and Apache Log4j.

To address the growing challenge of software supply chain security, organizations have turned to leveraging software bills of materials (SBOMs). SBOMs are a standardized inventory of software components used in a particular product or system, including their versions, dependencies and sources. SBOMs provide transparency and visibility into the software supply chain. They can be an effective approach for identifying and mitigating security risks, compliance issues and operational challenges – assuming organizations have the right tools to fully benefit from SBOMs, including runtime discovery, in place.

SBOMs are not a new concept, but their adoption has accelerated in recent years due to a number of important factors, including the following.

First, regulatory agencies, such as the U.S. National Institute of Standards and Technology (NIST) and the Cybersecurity and Infrastructure Security Agency (CISA), have been advocating for SBOMs as a best practice for software supply chain security. NIST has released a draft guidance document on SBOMs, and CISA has mandated SBOMs for federal contractors and subcontractors.

Second, industry initiatives, such as the Open Source Security Foundation (OpenSSF) and the Software Assurance Forum for Excellence in Code (SAFECode), have promoted SBOMs as a key element of software security. OpenSSF has launched a working group on SBOMs and published the Open Source Software Security Mobilization Plan to improve the resiliency and security of open source software (OSS), including guidelines for SBOM management; SAFECode published a whitepaper on SBOMs.

Third, supply chain attacks have highlighted the need for SBOMs as a proactive and preventive measure. SBOMs can help organizations detect and respond to vulnerabilities, malicious code and unauthorized changes in the software supply chain. SBOMs can also support incident response and recovery efforts by providing accurate and comprehensive information about the affected software components.

Fourth, customer demands and market pressures have influenced the adoption of SBOMs. Customers are increasingly aware of software supply chain risks and are asking for SBOMs as part of their procurement and evaluation processes. Some industries, such as health care and automotive, have mandated SBOMs for safety and liability reasons. SBOMs can also enhance brand reputation, customer trust and competitive advantage for software vendors.

Fifth, technological advancements have facilitated the creation and consumption of SBOMs. The emergence of open standards, such as CycloneDX and SPDX, has enabled interoperability and integration of SBOMs across different tools and platforms. The development of automated tools, such as software composition analysis (SCA) and vulnerability scanners, has enabled the generation and validation of SBOMs in a timely and accurate manner. The integration of SBOMs with other security measures, such as identity and access management (IAM) and security information and event management (SIEM), has enabled an effective view of software supply chain security.

However, SBOMs are not a silver bullet for software supply chain security. They are only as good as the data they contain, and the quality of data can vary depending on the source and the method of collection, particularly around the application software stack of APIs, cloud services and SDKs. SBOM inventory is constantly changing, and being able to leverage their up-to-date data requires more than a snapshot from traditional source code static analysis approaches, but rather continuous runtime analysis and dynamic inventory.

For example, common vendor management and software composition analysis (SCA) approaches often lack source code access for mobile, web, cloud and commercial-off-the-shelf (COTS) software, as well as third-party API services.

For a more effective approach, organizations can benefit from a full-stack attack surface management (ASM) software supply chain solution that delivers continuous third-party application asset discovery and dynamic tracking of third-party vendors. These newer approaches automatically categorize assets under known vendors, allow customers to add additional new vendors, curate individual assets under any vendor, and alert on increases in policy violations and high embed rates of third-party vendors within key applications. These automated capabilities allow vendor management teams to remedy supply chain security problems more quickly and easily and much more accurately than older approaches.

It is important to note that SBOMs also require collaboration and communication among multiple stakeholders, including software vendors, suppliers, customers and regulators. They may also pose some privacy and intellectual property concerns since they can reveal information about the software components and their relationships.

Overall, organizations need to adopt a comprehensive and risk-based approach to software supply chain security, of which SBOMs and tools to best leverage their information are an important component. Organizations should establish policies and procedures for SBOM creation, management and sharing, as well as for validation and verification. Organizations should also integrate SBOMs with other security measures and practices, such as threat modeling, penetration testing and code review to best ensure the security of their software supply chains.

May 23, 2023