devopsy.kinsta.cloud

Tag: open source security

Establishing Visibility and Governance for Your Software Supply Chain

Among the top cybersecurity projects for CSOs in 2025 is “establishing asset visibility and strong cloud governance.” But what does this mean in practice? Many organizations have a good handle on their data storage and compute infrastructure. However, while they may know which virtual servers or serverless platforms are in use, they often don’t know what’s in the applications that are running. The Log4Shell vulnerability was a wakeup call: companies scrambled — and often failed — to find all affected systems.

Why it Matters

The pace and sophistication of attacks continue to increase. The model of a lone hoodie-wearing hacker sitting in a dark room defacing websites doesn’t represent the modern threat landscape. Attackers are large, organized operations, often crime rings or even nation-states. They’re not just going after the easy targets; they’re going after impactful targets: companies that can afford to pay large ransoms, critical infrastructure controls, and government databases. Sometimes the motivation is purely financial, sometimes it’s geopolitical. Either way, attackers are well-resourced and determined.

Even if your organization isn’t the direct target, you can fall victim to attackers. A supply chain attack designed to gain access to a bank, for example, could also poison your supply chain. The attackers will gladly take your customer information or hold your servers hostage to ransomware.

Modern software supply chains are incredibly complex webs of third-party code. To properly secure the supply chain, organizations must first gain visibility into all of the components that go into their applications. This is necessary not just on a per-application basis, but across the entire portfolio. From there, organizations can make prioritization decisions based on context and set automated policy enforcement to ensure that applications are securely developed and deployed.

Security requires constant vigilance, and vigilance requires visibility. How can you hope to secure what you can’t see? That means you need to know what goes into your software and you need to know where that software is running. The faster you can start fixing vulnerabilities instead of looking for them, the quicker you’re protected against the latest attack.

Challenges in Securing Your Supply Chain

Most of your software isn’t yours. In other words, your employees didn’t write it; they brought in third-party dependencies. Whether from vendors or open source projects, these libraries provide a foundation that allows your developers to focus on the differentiated functionality that your company provides, which speeds your innovation and delivery.

The tradeoff for this increased speed is taking on the risk of your dependencies’ security posture. Do you know what vulnerabilities exist in your dependencies? If you do, you can’t necessarily fix them yourself. Even if you have a good relationship with the providers of your direct dependencies, they have their own dependencies, and these transitive dependencies can number in the hundreds and be several layers deep. How do you track all of those dependencies for vulnerabilities?

Once you know what vulnerabilities exist in your supply chain, do you know where the vulnerable versions are deployed? 10 days after the Log4Shell vulnerability was announced, less than half of the publicly-reachable vulnerable applications had been patched. That’s a big window for attackers.

Knowing where all of the vulnerable applications are running doesn’t give you the full picture. Just because an application contains a vulnerability doesn’t mean the vulnerable part of the code is used. It may be the case that your application doesn’t call the vulnerable code path or that the vulnerability is conditional on a configuration that you’re not using. While you may still want to address the vulnerability, it’s not as high a priority if it doesn’t affect your current application portfolio.

Establishing Visibility and Governance

Thankfully, you can overcome all of the challenges. The first step is to start building software bills of materials (SBOMs) at build time. The SBOM records what goes into your software, so it is the foundational piece of building asset visibility. You can then use that information to build a knowledge graph about your supply chain, including vulnerabilities and software licenses. When you aggregate these SBOMs across all of your application portfolio, you get a holistic view of all dependencies.

This portfolio-wide view helps you prioritize which issues to fix first. The impact of fixing a vulnerability in one or two applications is a lot less than fixing a vulnerability in all applications. Similarly, fixing a vulnerability in a production, external-facing application is more impactful than fixing a vulnerability in an internal development environment.

As mentioned above, knowing if a vulnerability even affects the application that contains it is a key part of the overall visibility picture. Ingesting VEX (Vulnerability Exploitability eXchange) statements, when available from your dependencies, will help you eliminate the vulnerabilities that are irrelevant to your code. Runtime analysis, like with eBPF, adds more filtering by determining which code paths are actually loaded into memory. This gives you more detail on which vulnerabilities are exploitable in your specific application, especially when VEX statements aren’t available.

Context-aware prioritization makes responding to issues easier and faster. But what’s even better is preventing issues in the first place. Creating SBOMs at build time means you can enforce policies to prevent issues. For example, you can use the SBOM that you just produced in the build pipeline to check for known vulnerabilities in the dependencies and block merging the code into the main branch if it contains vulnerabilities or if it has a poor security posture. After build, tools like Kubernetes Gatekeeper can automatically enforce policies to block deployment of applications with critical vulnerabilities, missing or incorrect cryptographic signatures, and other security concerns.

One final piece of the puzzle is tracking software provenance. Tracking and gating on software provenance gives you another avenue to protect yourself from vulnerable code. This is often overlooked, but given the prevalence of attacks against open source library repositories, it’s more important than ever. Provenance is a provable record of how software was built, where it was built, and by whom it was built. These cryptographically-signed attestations mean that you can know if the package you downloaded from npm came from your dependency’s official build system or from hackerman9000’s laptop after he compromised the project’s npm account. But provenance doesn’t just help with deliberately altered, it also gives you a sense of the security practices used in the build process. Standards like SLSA give you assurance that builds took place in environments with sound security.

Start Somewhere

Implementing all these practices takes work and cultural change. You don’t have to do everything at once; each step you take reduces risk. For example, we know that SBOMs alone won’t solve supply chain issues, but having them is better than not having them. Once you have SBOMs, you can build on them to collect more data. Don’t let the perfect be the enemy of the good. After all, security is a process, not a destination.

October 14, 2025
How GitHub Plans to Secure npm After Recent Supply Chain Attacks
The npm registry just weathered one of its most sophisticated attacks yet. On September 14, 2025, the Shai-Hulud worm infiltrated the JavaScript ecosystem through compromised maintainer accounts, spreading malicious code across popular packages. GitHub’s response signals a significant shift in how we approach security for package registries.

The Shai-Hulud Attack Changed Everything

This wasn’t your typical malware injection. The Shai-Hulud worm was self-replicating, spreading through npm packages while stealing multiple types of secrets—not just npm tokens. It used post-install scripts to propagate, turning trusted packages into attack vectors.

GitHub moved quickly. They removed over 500 compromised packages and blocked uploads containing the malware’s indicators of compromise. However, the damage highlighted a harsh reality: Current authentication methods are insufficient to protect the open-source supply chain.

GitHub’s New Security Roadmap

Starting soon, npm will only support three publishing methods:

Local publishing with mandatory two-factor authentication. No more bypassing 2FA for convenience. If you’re publishing from your machine, you’ll need proper authentication every time.

Granular tokens with seven-day expiration limits. The days of long-lived tokens are ending. These shorter-lived credentials reduce the window for abuse if compromised.

Trusted publishing. This removes API tokens from CI/CD pipelines entirely, letting package repositories verify your identity through your build environment.

What’s Getting Deprecated

GitHub isn’t just adding new security features — they’re removing weak ones:
- Legacy classic tokens are going away
- Time-based one-time passwords (TOTP) are being replaced with FIDO-based 2FA
- Publishing tokens will be disabled by default
- The 2FA bypass option for local publishing is disappearing
These changes might disrupt some workflows. But GitHub plans a gradual rollout with migration guides and support.

Why Trusted Publishing Matters

Trusted publishing started with PyPI in April 2023. The idea is simple: Instead of managing API tokens in your build system, the package registry verifies your identity through your CI/CD platform.

RubyGems added it in December 2023, followed by crates.io and npm in July 2025, and NuGet in September 2025. The timing isn’t coincidental—package registries are racing to eliminate token-based authentication vulnerabilities.

Here’s how it works: When your GitHub Actions workflow runs, it generates a signed token that proves the code came from your repository. The package registry verifies this signature and publishes your package without needing stored credentials.

No tokens to steal. No secrets to manage. No attack surface for malware to exploit.

The Broader Context

The Shai-Hulud attack wasn’t an isolated incident. Package registries have faced a surge in account takeovers recently. Attackers are getting more sophisticated, combining traditional credential theft with self-replicating malware designed specifically for package ecosystems.

The open source community’s response has been swift. The OpenSSF Securing Software Repositories Working Group now recommends trusted publishing as a best practice for security. Package registries are implementing it faster than originally planned.

What Maintainers Should Do Now

Don’t wait for the forced migration. Here’s what you can implement today:

Switch to trusted publishing for all your packages. It’s already available on npm and eliminates the biggest attack vector.

Enable 2FA requirements for all publishing actions on your accounts, organizations and packages.

Use WebAuthn instead of TOTP when setting up two-factor authentication. Hardware keys and biometric authentication are more secure than authenticator apps.

Audit your existing tokens. Review what’s still active and remove anything you don’t actively use.

The Road Ahead

These security improvements come with trade-offs. Some automation workflows will need updates. Some convenience features are disappearing. However, the alternative, watching malware spread through the packages we all depend on, is unacceptable.

The software industry relies heavily on open-source software. When package registries get compromised, it affects everyone downstream. That’s why GitHub’s aggressive timeline makes sense. They’re not waiting for the next attack to implement these changes.

True supply chain security requires the participation of everyone. Package registries can implement better authentication, but maintainers need to adopt these tools to benefit from them. The ecosystem’s security is only as strong as its weakest authentication method.

The Shai-Hulud attack was a wake-up call. Now it’s time to build the defenses that should have been in place years ago.
September 26, 2025
Lineaje Adds Module to Manage Open Source Software Security Lifecycle

Lineaje has added an Open Source Manager (OSM) that leverages artificial intelligence (AI) to help DevOps teams prioritize their remediation efforts better.

OSM uses an existing Lineaje AI with a BOMbots tool to analyze software bills of materials (SBOMs). It identifies dependencies and surfaces recommendations for remediation in open source software issues across the software supply chain. OSM adds a module to that tool to centralize workflow management, enabling DevOps teams to apply rules that restrict which open source components can be used based on their IT organization’s policies.

Nick Mistry, senior vice president and CISO for Lineaje, explained that, in many instances, an organization may not have any means of remediating a vulnerability because no patch exists. The best that can be done is to alert the open source project’s maintainers in the hope that they will provide a patch or pass along the guidance to a third party to address the issue.

However, DevSecOps teams still need to identify when patches are available, especially if an issue stems from a known vulnerability that has already been remediated. In many cases, organizations run older versions of software with known vulnerabilities that were subsequently fixed in a later update. In other instances, DevOps teams might just as easily discover that the vulnerability discovered is not manifesting itself in software exposed to the Internet, noted Mistry.

The goal is to reduce the mean time to protect (MTTP) when vulnerabilities in open source software across a spectrum of risk levels, Mistry added. That capability also serves to reduce the total cost of maintaining open source software by surfacing issues involving licenses, code quality, security posture, maintainability, age, supplier and provenance.

DevOps teams can model the potential impact of an upgrade using the Lineaje AI with the BOMbots tool. In addition, fingerprinting capabilities can be applied to make identifying components with suspicious and unknown origins easier. DevSecOps teams could identify open source software projects that are not being well maintained, noted Mistry.

While open source software security issues are a major concern, most organizations are not going to reduce their reliance on these components. Just about every packaged or custom application has at least one open source component. The pace at which modern applications are being built would not be sustainable if it were not for all the reusable open source components that have been made available.

The challenge now is securing the software supply chains that organizations rely on to build applications using those components at a time when governments around to world are starting to hold organizations more accountable for the security of the applications being deployed. OSM, in effect, is designed to enable organizations to achieve continuous compliance with those regulations, said Mistry.

Regardless of motivation, more organizations than ever are clearly focused on securing open source software. The only thing that remains to be determined is how quickly that goal can be achieved before one or more known vulnerabilities lead to a catastrophic breach.

May 30, 2024
Does More Money Improve Open Source Security?

It sounds simple: If you pay developers more money, they’ll improve the quality of their code, right? The evidence isn’t so clear.

There has been little direct evidence that paying open-source developers results in more secure code – until now. The Atlantic Council’s Digital Forensic Research Lab (DFRLab) recently showed a correlation between funding for maintainers and the security of open source code.

Specifically, the researchers found “prima facie evidence of a positive effect of general open source software funding on open source software security.” They looked at the 1,000 most downloaded Python Package Index (PyPl) and npm JavaScript packages. Using Open Source Security Foundation (OpenSSF) tools such as “funder-finder” and the “Security Scorecard” the researchers saw a connection between financial support and security enhancements.

That may sound like an obvious conclusion, but the evidence was not as clear as you might think. The researchers report with only “moderate confidence” that a meaningful connection exists “between more open source project funding and improved security posture. …[M]ore funding generally correlates with more dramatically differentiated security practices.”

This seems to go against common sense. As Dana Wang, OpenSSF’s Chief Architect, said, “Balancing the security, reliability, availability, performance, and cost of open-source software is not trivial. Maintainers sometimes have to choose between new features, bug fixes, and less critical security patches.” And, even after such open source project security fiascos as Log4J, as we all know, security all too often comes too low on developers’ priority list.

Still, other research, such as programming security company Tidelift‘s 2023 State of the Open Source Maintainer Report, showed that “paid maintainers were 25-30% more likely to have completed the [security] work or plan compared to unpaid maintainers. On every development practice surveyed, paid maintainers were significantly more likely to implement it than unpaid maintainers.”

Sometimes security fixes are needed and available, but corporations aren’t adopting them. In another study from Synopsys, the 2024 Open Source Security and Risk Analysis Report, not quite half of codebases contained open source components with no new development in the last two years; 91% contained components ten or more versions behind the latest version. This is a failure by open source users, not underpaid open source developers. Bad companies! Bad!

Returning to the DFRLab study, its researchers also found that diversity in funding sources appeared to be beneficial. That is, projects with multiple supporters apparently do better (with security at least) than those with a single backer. This suggests that investors should look at a variety of support mechanisms rather than solo-source funding.

One reason why the results are so fuzzy is that we don’t have enough data. As the DFRLab crew observed, the “OpenSSF Scorecard tool itself has several quirks… Scorecard does not necessarily capture all the security improvements that might occur during a project. For example, a security audit, paid for voluntarily by maintainers who received funding, might look for and remediate new vulnerabilities while likely not directly improving any Scorecard metric. Some practices may also meet OpenSSF criteria in principle but be missed by the automated scorecard check. For example, a project might include a strong security policy but with a different filename than what the scorecard search is looking for, and thus not pass the subcheck.”

As DFRLab states, we need more research to understand how funding affects security. No one can really doubt that more money helps, but the most effective way to fund projects, maintainers and developers – and how we should measure the results – remain unclear.

Given how critical open source security is to all of our technology, the sooner we figure it out, the better.

Photo credit: Jonathan Kemper on Unsplash

May 13, 2024
How Devs Can Improve Open Source Security in the Enterprise

Modern applications are dynamic. They’re distributed and they’re often born in the cloud. These applications can be developed on the fly, spun up and scaled quickly to meet evolving user and market demands—enabling a level of business agility that allows users to make quick, informed decisions in real-time and take advantage of opportunities as they arise.

Enabling this development velocity are tens of thousands of open source libraries–pieces of code that developers can include in their applications to perform a specific function without having to code it from scratch. Open source libraries prevent developers from having to reinvent the wheel and allow them to focus on their differentiated business logic. While efficient and a major time saver, open source software may be the biggest single threat to software supply chains today. Developers simply have too much faith in the integrity of free, open source code that they casually include and distribute with their applications.

Despite the operational benefits, the pressure to develop quickly shouldn’t expose the organization to increased security risks. Development teams need to rethink how they use open source software and take steps to ensure they meet the same security standards as other enterprise applications.

The Open Source Security Issue

The problem is that most open source projects are maintained by a small number of developers that contribute their time and expertise to build software components that can be reused by the community at large. However, these developers are volunteers who aren’t being paid for their time and effort, nor are they under the same pressure as enterprise developers to ensure there are no vulnerabilities in their code. They may also lack the level of enterprise security expertise necessary to protect applications from increasingly sophisticated attacks. Some may even be hostile to efforts to better secure their code if it means slowing performance or changing the user experience.

The result is that insecure code is increasingly making its way into enterprise applications that use these open source libraries. In fact, the Log4j vulnerability last year was due to a vulnerability in an open source Java logging library. Recognizing the national security importance of open source software, the Biden administration recently issued an executive order and met with cybersecurity professionals to better safeguard the federal government’s computer systems. The resulting recommendations included policies that would help prevent security vulnerabilities in open source packages, improve the process for identifying and fixing defects and reduce time to remediation for distributing and implementing fixes.

It’s imperative that the private sector take notice and make more of an industry-wide effort to secure open source software that makes up businesses’ most critical applications today.

Three Steps for Better Open Source Security in the Enterprise

The bottom line is that enterprises themselves need to take more responsibility for securing enterprise applications—especially as these applications rely more on open source libraries. They need to take a more proactive approach that spans the entire software life cycle, embeds security tools directly into development platforms and provides complete transparency into what makes up their most critical applications.

Here are three things organizations can do to make sure their open source software is secure:

1. Continuously conduct end-to-end application testing

Modern applications are constantly evolving—making it more imperative than ever to continually test for vulnerabilities throughout the entire software life cycle. Starting with the planning stage, security should be embedded throughout the process—providing developers with a solid framework for building secure code before it goes through testing. Once pushed, application security testing should continue as an automatic, built-in step when pushing code to production. DevOps and security teams should work together to ensure security is no longer an afterthought to feature development.

2. Embed enterprise security capabilities directly in development platforms

Forcing developers to take an extra step to ensure secure code when they’re already under pressure to push code quickly is a recipe for failure. Security teams must make it as easy as possible by integrating security controls directly into existing development processes. Standardizing security testing and controls during development makes it more likely policies are recognized, understood and followed.

3. Ensure transparency in every snippet of code you use

Complexity is the enemy of good application security. DevOps teams need to know exactly what makes up their applications—including open source libraries and their dependencies. The federal government is pushing the open source community to standardize a software bill of materials (SBOM). Similar to regulations that require hardware manufacturers to know where every part is sourced, this would force both open source developers and enterprise developers to take more responsibility for the code they include from other sources. That way, if a vulnerability is discovered in a library, enterprises could easily determine if they are at risk and take appropriate action to fully mitigate the issue as quickly as possible.

It’s Time to Take Greater Responsibility

Open source software enables the kind of development agility that makes businesses thrive, but open source also greatly expands enterprise threat surfaces—leaving many organizations at risk of a breach. Organizations need to take greater responsibility for the security posture of the open source libraries they use to speed application development. This requires end-to-end application testing throughout the entire software development life cycle; embedded security tools and controls in development platforms and processes and complete transparency into the open source software that makes up critical enterprise applications. This proactive approach will ensure that vulnerabilities such as Log4j are identified and patched quickly before threat actors are able to exploit them and compromise applications.

December 8, 2022
Securing Open Source Components in a World of Mixed Committer Motivations

Our world runs on software that contains open source components. This places an increased burden on developers, as the primary consumers and deployers of those components, to use code that is fully up-to-date and secure. The vast majority of open source committers with write access to project repositories remain focused on maintaining and making positive improvements to that code, while also helping to benefit the open source community as a whole. However, the sheer complexity of many open source projects has opened the door to different, more insidious and concerted types of attacks which could cause widespread and serious harm.

Developers Have to Move Fast and Reuse Gives Them That Speed

One complicating factor is the prevalence of reuse of third-party open source elements across the majority of open source projects through two main types of dependencies. With direct dependencies, an open source component links straight to a third-party code library, while transitive dependencies occur when a component calls out to a library that is, in turn, communicating with another dependency.

Dependencies enable developers, under ever-increasing corporate pressure to deliver new software rapidly, to move faster by building on top of existing work by their peers. The downside is that those dependency foundations are vulnerable to attack, with some libraries perhaps under the aegis of a single committer for whom the code collection may be a side project and perhaps not attended to on a regular basis.

Sonatype research and that of other organizations tells us that developers working with a Java application may be dealing with a staggering 1,500 dependency changes every year, with the average Java application containing 148 dependencies and the average Java project being updated 10 times annually. While the number of open source projects isn’t ballooning, the amount of versions of those projects is skyrocketing. The majority of vulnerabilities today impacting open source projects result from transitive dependencies, the more complex type of dependency.

In an area where multiple projects may exist effectively solving the same issues, developers need to be aware of which projects are being actively maintained versus those that are either poorly nurtured or no longer in operation. On the other hand, successful open source projects may have tens or hundreds of committers with write repository access, which address a whole different set of issues, including reaching agreement on what a vulnerability consists of and how to fix or patch it.

Amid a Community for Good, Bad Actors and Incidents of Self-Sabotage Emerge

The reasons why individuals choose to commit to open source software are wide-ranging; a strong desire to give back to the community to a passionate and, perhaps very single-minded, hobby to an integral and paid part of a developer’s role and responsibilities at work. A few years ago, the concept of a malicious open source committer was an abstract theory. The thought back then was that perhaps a handful of ‘bad apples’ might exist within a community of millions of developers. However, since 2019, researchers’ tools and those of other organizations have identified over 100,000 malicious, suspicious or proof-of-concept packages, pointing to a much higher degree of behavior by cybercriminals.

Bad actors are deliberately taking advantage of the complexity of open source software to knowingly introduce exploitable vulnerabilities. These attacks are growing in sophistication particularly in relation to malicious code injections into the source code of an open source library. These attacks can often appear as harmless code changes and remain hidden before delivering a malware payload. There is the recent example of a crypto heist that stole over $3 million in cryptocurrency which leveraged compromised access to a private GitHub repository.

Protestware, where a code maintainer chooses to self-sabotage their open source project often to draw attention to a cause, whether geopolitical or specific to that individual, has become an issue over the past year. In this case, although the stated goal is not malicious hacking, real harm, in terms of major disruption, to developers using that code can occur.

To counter both the impact of bad actors and of self-sabotage on open source components, developers need to take advantage of release integrity features to help protect themselves and their organizations from harm. They must be able to have full visibility and control over the software they build and the open source components that they are sourcing as part of that development work. At the same time, governments around the world, including in the U.S., are weighing into issues related to the security of software supply chains, so increased regulations and penalties should also factor into how developers and their organizations behave moving forward.

We’re only one step away from the next potential Log4j attack, and every developer and organization needs to up their game and be ready to move fast on vulnerability identification and remediation. The gathering and sharing of data is what will help to increase everyone’s safety.

December 2, 2022
Sigstore Code Signing Service Becomes Generally Available

A free digital signing service for software created by the Sigstore open source community has become generally available this week via the cloud.

Announced at the SigstoreCon event that occurred during the KubeCon + CloudNativeCon North America conference, the cloud service makes it possible for developers to both cryptographically sign artifacts and verify that the components used to build applications are secure.

The Sigstore community operates under the auspices of the Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation. Sigstore project sponsors, including Google, Red Hat, GitHub and Chainguard, have committed to operating a service with a 99.5% uptime and round-the-clock pager support. More than 70 organizations, including Shopify, Autodesk, Trail of Bits and Rancher Government Solutions are actively involved in maintaining and scaling Sigstore.

More the four million Sigstore signatures have been logged using Sigstore thanks mainly to Sigstore support that is already included by maintainers of open source Kubernetes and Python projects. The npm community also announced it is working to integrate Sigstore in all packages used to build JavaScript applications.

Bob Callaway, co-founder of Sigstore and technical lead and manager for the Open Source Security Team at Google, said this service will make it much simpler for developers to add digital signatures to code in a way that can be more easily verified within a software bill of materials (SBOM).

The goal now is to expand the number of open source projects that will commit to employing Sigstore to better secure software supply chains, he added.

The security of open source software became a major concern in the wake of the open source Log4j zero-day vulnerability. Sigstore was under development prior to that disclosure, but the Biden administration’s executive order requiring Federal agencies to verify the security of open source software has enabled the availability of more resources for the project.

In response to the executive order, the OpenSSF outlined a 10-point plan to secure open source software that would require more than $150 million in funding. One of those goals is to have 50 of the top 200 projects and 1,000 of the top 10,000 projects adopt an interoperable approach to software signing. That project was estimated to cost $13 million for the first year and $4 million per year beyond, with a one-time additional $10 million required after the first year.

It’s not clear how broadly developers will employ digital signatures, but given the current level of concern, it’s probable most organizations will soon require any code they use to be digitally signed so it can be verified as immutable. The next big challenge, of course, is finding a way to operationalize the collection of all the SBOMs that will include those signatures. After all, SBOMs are only the first step in a larger DevSecOps workflow that will enable organizations to accept or reject software components based on all the data being collected.

October 27, 2022
Log4j: Is There Such a Thing as ‘Too Much’ Open Source?

The Log4j vulnerability got me thinking: Is there such a thing as too much open source?

Before anyone immediately fires off a flaming email, rage tweet or scathing blog post, hear me out for a moment. If you know me, you know that I am an open source fanatic. I’ve been asked many times, “Should we use open source software and, if so, how much?” My response was (and still is) that open source software is a massive source of innovation and I can promise you, whether or not you use it, know that your competitors certainly are!

I could go on and on extolling the virtues of open source and how it enabled the cloud, reshaped software architectures, created new product categories and gave us valuable tools like telemetry and increased visibility into cloud-native apps, infrastructure and beyond.

When I first heard about the Log4j vulnerability on December 9, like many others, I quickly informed the tech leaders, engineers and SREs in my network about the serious potential risks such a zero-day vulnerability in the widely-used Log4j tool represented. In one way—probably the only way—the Log4j exploit shared a very important characteristic with the SolarWinds supply chain attack of late 2020: The widespread distribution and ubiquitous use of the software across applications, systems and organizations. Arguably, Log4j is used much more widely.

This similarity raised questions I hadn’t fully considered in the context of such widely used software. Can we use too much open source software? Is Log4j too prevalent in its use and, therefore, a single point of failure? How bad would the Log4j situation be if a fix wasn’t immediately available or if one wasn’t forthcoming?

The answer I kept coming back to is a resounding ‘no’. I still don’t think there’s such a thing as using too much (insert any open source project name) software. Log4j—and so many other open source software projects—are massively beneficial to us, and we virtually couldn’t function without these systems, tools and solutions in today’s age of ubiquitous software.

What the current Log4j vulnerability, recent software supply chain attacks and other cybersecurity incidents with widespread impact highlight is our need to quickly respond to security incidents and take remediation steps anywhere they are found in the software stack. And that includes open source software. High-impact security vulnerabilities can occur anywhere across the numerous software stacks and services we use, though they have even greater impact when they occur in widely-used infrastructure, reusable components and microservices.

In a world of infrastructure-as-code (IaC), it’s no longer a matter of patching a particular set of routers or servers. Unfortunately, much of the software stack is not containerized or built using smaller, easily replaceable containers and microservices—at least not yet. Our mindset and processes going forward from this incident must change, and we have to be prepared to respond to and remediate vulnerabilities both horizontally and vertically across the software stack. In certain circumstances, we have to do this on a large scale.

As is common with active open source projects, Log4j contributors responded quickly with a readily available fix. The real answer to my original question above is that few are as prepared to respond to an important security vulnerability as those incredible, hard-working, active and engaged open source software developers.

December 15, 2021
Snyk Tool Prioritizes Open Source Vulnerabilities

Snyk today announced it has enhanced the ability of its namesake vulnerability scanning tool by adding the ability to identify which open source vulnerabilities should be fixed first using a scoring tool that leverages data science and machine learning algorithms to analyze code.

In addition, DevOps teams can now take advantage of automated pull requests to fix those vulnerabilities.

Finally, DevOps teams also can define policies that require certain classes of vulnerabilities to be fixed whenever they are identified.

Aner Mazur, chief product officer for Snyk, said given the number of open source modules employed and the dependencies that exist between them, staying current on patches and updates has become a major challenge. The only way to achieve that goal is to rely more on tools that help automate that process, he said.

These days it’s easy for development teams to feel overwhelmed by patch requests to fix open source security vulnerabilities. The trouble is, not all those requests are of equal weight. Snyk provides tools that enable developers to understand which vulnerabilities are of the highest severity and how they might impact their code.

Armed with that insight, Mazur said, it then becomes possible to prioritize remediation tasks within the context of a DevSecOps process. Otherwise, developers will simply fix vulnerabilities regardless of relevance as time allows. Developers may remediate dozens of bugs only to discover that a critical flaw that had been left unaddressed led to a major compromise.

It’s also not uncommon for DevOps teams to fail to appreciate dependencies between open source code modules. When patches to code are applied in the wrong order, an application can break, resulting in DevOps teams having to start the whole process over. The prioritization tools provided by Snyk prevent that from occurring, Mazur said.

In the absence of any ability to prioritize vulnerabilities, developers eventually become inured to patch requests because they don’t know whether their patching efforts are making any real difference, he added.

Worse yet, all the time spent on patching low-level vulnerabilities reduces the amount of time developers have available to write new application code. In fact, one of the paradoxes of DevOps is as the number of applications deployed increases so too does the amount of time spent debugging them. Before too long, a DevOps team can find itself spending more time fixing existing code than developing new applications.

As DevOps teams become more dependent on open source code, they benefit from the collective security efforts of all the contributors to that project. The result is more secure code, as patches to remediate specific issues are made available. The challenge now is finding a way to automatically apply those updates in a way that doesn’t conspire to slow down the overall application development process. Achieving that goal requires visibility into what is now a multitude of open source projects that no DevOps team is ever going to be able to consistently achieve on its own.

July 22, 2020
Open Source Vulnerabilities Were Up 50% in 2019 — How Will It Impact Software Development in 2020?

Open source vulnerabilities have been on the rise in recent years, but 2019 was truly one for the record books with a spike of nearly 50% over the previous year.

According to the recently released State of Open Source Security Vulnerabilities report by the WhiteSource research team, the number of vulnerabilities jumped from 4,100 in 2018 to 6,100 in 2019.

Figure 1: Number of reported open source vulnerabilities per year.

In hopes of better understanding what is behind this rise in the number of open source vulnerabilities, what it tells us about the state of open source usage in modern software development, and even where we are likely headed in the years ahead, let’s take a look at the report’s findings for insights on the current state of open source security.

More Code, More Vulnerabilities

According to the research report, the primary driver behind the rise in open source vulnerabilities is a function of the growth of open source. There are now more open source projects, code and members of the community than ever before.

All of these good folks are working hard to not only write more code, often with the backing of large tech giants, but also to seek out vulnerabilities in the code that can put the users of their components at risk. The combination of more code being written and the number of eyeballs reviewing the code for those inevitable human mistakes, comes together to produce a larger number of vulnerabilities being reported.

The increase in the number of vulnerabilities in open source components will have a considerable impact on software development. In recent years, we have seen open source components playing a bigger role in how developers build their applications. Most estimates believe that open source components comprise between 60-80% of the codebase in most modern applications. When a vulnerability is reported in a popular project such as Apache Struts or the Linux kernel, then huge swaths of developers can quickly find their software in need of patching.

The Good, the Bad and the Bazaar

WhiteSource’s research revealed the reassuring news that over 85% of reported open source security vulnerabilities have a fix available.

One of the reasons for this improvement is the fact that more enterprises are dependent on open source components for their own software, so they are investing in making it more secure, via bug bounty programs, backing foundations that support open source security or backing the projects themselves.

However, even as the situation in the field appears to be improving, there is still a way to go.

The fact that many published open source vulnerabilities are not indexed in one central location adds complexity to this challenge for development teams attempting to stay on top of published open source security vulnerabilities. If proprietary software is still considered to be the classical Cathedral, well-organized information and support under one roof, then open source is very much the Bazaar.

The WhiteSource report found that only 84% of known open source vulnerabilities appear in the National Vulnerability Database (NVD). This means that information about another 16% of the known vulnerabilities is out there, spread out among a wide range of resources. These might be security advisories, bug trackers and a laundry list of other locations. While 45% of reported open source vulnerabilities that were not initially published on the NVD eventually end up there, that could take months since their publication on other resources.

The challenge for organizations is that they run the risk of using a vulnerable component unaware of the security updates, while hackers are scouring all resources, looking for vulnerabilities that could be their next meal ticket.

Given the evolving security challenges facing organizations, what does the future hold for open source security in the year ahead?

Our Predictions for 2020

The easiest prediction that we can provide is that 2020 will have even more reported open source vulnerabilities than were published in 2019. The only question is: How big it will be? Chances are that the count will be more significant due to recent advances making it easier than ever to report and an open source vulnerability.

This past year saw the launch of GitHub Security Lab which aims to help researchers, maintainers and users easily report an open source vulnerability and publish a verified fix in one centralized location. By simplifying the reporting and documentation process, GitHub has gone a long way in helping their users to protect their software with better quality reporting.

Key Takeaways

For many, the rise in the number of reported open source vulnerabilities might be concerning, making us confront the fact that we probably have plenty of them in our software right now. However, the situation is actually improving, and even heading in the right direction. When vulnerable open source components are reported, we get to track them and move quickly to remediate. Only once you know you have a problem can you begin to fix it.

Hopefully, the increase in security awareness will keep pushing for a stronger, more organized and better supported open source community that will be resilient in the face of even more vulnerabilities that are likely to show up in next year’s research report.

April 20, 2020