devopsy.kinsta.cloud

Tag: oss

  • Friend or Foe? ChatGPT’s Impact on Open Source Software

    Friend or Foe? ChatGPT’s Impact on Open Source Software

    OpenAI’s ChatGPT took the world by storm, amassing 100 million users in the first two months after its public launch. The continued interest in the tool has created a buzz among developers, especially those in the open source community. But this begs the question: How will ChatGPT impact open source software?

    Amid the excitement and promise that ChatGPT holds, many wonder whether the open source community, just like any other organization, should fear or embrace the technology. Certain concerns for open source contributors and developers are sparking debate as questions surrounding the origin of generated source code surface, and the possible ethical and legal implications are discussed. While valid, developers should not fear or stray away from using ChatGPT but rather shift their focus to understanding how to embrace it for positive outcomes.

    The Top Three Open Source ChatGPT Concerns

    Collaboration between contributors is the cornerstone of open source projects, which some fear will be disrupted with the increased use of AI-based tools. However, the level of synergy within the community cannot be easily taken away or replaced by tools. Instead, ChatGPT or other AI-based tools, including GitHub Copilot, allow developers to produce code more quickly and more efficiently.

    As developers increasingly use AI tooling to assist with new or enhanced code, project collaboration and oversight will help improve the AI-generated code.

    Debunking and navigating the top concerns will be imperative to take full advantage of the technology’s promise and potential.

    Validity. While the creation of code by ChatGPT has generated excitement among developers, critics argue that—without context—the validity of code can be questioned. Some open source developers worry that teams will begin to rely entirely on ChatGPT to generate code, but this concern fails to consider the humans involved in the process—today and far into the future.

    Developers don’t take ChatGPT’s output as the final word; rather, they use it as a baseline and jumping-off point to streamline their code. In fact, code is rarely created from scratch anymore. Developers rely on others’ source code. For example, code from Stack Overflow, GitHub and thousands of open source libraries that are available on public registries like npm, Maven, Nuget and PyPI. Bringing ChatGPT into the fold will not significantly change how developers source their code, but will improve development velocity, saving valuable time and associated cost.

    Training Data. Machine learning (ML) and deep learning (DL) model training must be fair, robust and explainable to avoid biases. If the data is wrong, the results will be as well. Garbage in, garbage out. Coming from ML/DL models, the code ChatGPT returns in response to prompts could raise concerns surrounding its accuracy. Like any other source (Stack Overflow, GitHub, etc.), ChatGPT’s code outputs are not guaranteed to be perfect, and developers must be mindful of this.

    However, there are more benefits of ChatGPT-trained models concerning code outputs. ChatGPT can also explain new or existing code, it can also effectively provide unit tests for code, which will help with writing better software faster.

    Ownership. Questions of ownership have surfaced regarding the use and distribution of the code that the AI tool generates. While the code that ChatGPT generates is the result of ML/DL inference from many sources, it’s the developer’s responsibility to use that code ethically and safely. It should be treated as any other public data or open source software, as not a final product and must be used in the context of your requirements. It’s also important to carefully review any code generated by ChatGPT and ensure that it doesn’t introduce vulnerabilities.

    Similar to GitHub Copilot, ChatGPT is trained by millions of lines of open source software. Code posted in prompts is likely to make it to models, too. Unlike copyrighted pieces of art and writing, the code that ChatGPT outputs should not be considered in a final state and should not be used in a way that is subject to licensing restrictions or legal implications that have dominated the conversations around its use.

    Ownership closely relates to the ethical dilemma of using AI-generated text, code and more. To address that, there are new tools that help detect whether AI generated the content and how much, which will be useful for educators concerned with students overusing the tool. This emerging area of checks and balances exemplifies the constantly evolving and improving technologies that allow humans, and in the case of open source, contributing developers to improve their craft and produce better open source software.

    ChatGPT’s Impact on Open Source Talent

    Discussions about whether ChatGPT will require new skills for existing positions or new jobs with specialized experts are ongoing. While new and exciting, ChatGPT won’t immediately create new or different jobs. Like any new tool introduced to developers, it takes time for developers to become familiar with the technology and understand how to use it best. ChatGPT is no different. For example, consider the previous push to use low-code/no-code technology. While this great technology is used to speed up the creation of apps and improve the usability for non-developers, it has taken time for organizations to properly use low-code/no-code technology. A similar perspective and trajectory will extend to the use of ChatGPT for all software development, including open source projects.

    In the coming weeks and months, it will be essential to encourage the open source community to embrace ChatGPT and explore its possibilities. Technology has already proven to be an effective educational tool. Consider asking ChatGPT for book recommendations about programming languages and coding; it delivers short descriptions for each book. Or, prompt it for the top takeaways from one specific book. By doing this, individuals can make learning so much easier. By learning from other developers and sharing resources, including results from ChatGPT, the community roots of open source will keep thriving.

    Human Involvement is Always Needed

    The public’s reaction to ChatGPT may be novel, but the idea behind the tool’s relationship to open source is not. Analyzing how ChatGPT works within the open source community reveals that it will be useful because it enhances the capabilities of developers, it doesn’t replace them.

    Whether it’s reviewing code, pair programming or learning from fellow developers, humans will not be replaced by generative AI, only enhanced. Leveraging the tool will decrease the time and effort needed to complete tasks, increasing developer quality and efficiency. The use of ChatGPT does not create new positions or overhaul current skill sets; it makes them better and empowers developers to expand their opportunities and enhance their abilities.

    Soon, ChatGPT and other AI tools will be standard practice within the open source community—or until a new technology is created and shakes up software development once again.

  • Report Identifies Top 10 Open Source Software Risks

    Report Identifies Top 10 Open Source Software Risks

    Endor Labs, a provider of a platform for managing open source software, published a report that classifies the top 10 open source software risks of 2023. The company published the list as part of an effort to better educate application development teams about issues that can lead to software supply chain compromises. That list includes:

    Known Vulnerabilities: A version of a software component may contain vulnerable code that is accidentally introduced by its developers. When vulnerability details are publicly disclosed, there may not be a patch readily available.

    Unmaintained Software: A software component may not be actively developed or maintained anymore, resulting in patches for functional and non-functional bugs not being provided in a timely manner—if they’re provided at all.

    Name Confusion Attacks: Attackers may create components whose name resembles the names of legitimate open source or system components. This is also known as typosquatting. Bad actors might also attempt to mimic trustworthy authors (brandjacking) or play with common naming patterns in different languages or ecosystems (combosquatting).

    Compromise of Legitimate Package: Attackers may compromise resources that are part of an existing, legitimate project or associated distribution infrastructure to inject malicious code into software components.

    Outdated Software: A project may use an old, outdated version of a software component even though a newer, more secure version exists.

    Untracked Dependencies: Developers may not be aware of a dependency on a component because it is part of another upstream module they employed.

    License Risks: A software component or project may not have a license at all, may have one that is incompatible with the component’s or project’s intended use or whose requirements are not or cannot be met.

    Immature Software: An open source project may not apply development best practices such as having a standard versioning scheme or lacking a regression test suite, review guidelines or documentation.

    Unapproved Changes: A software component may change without giving developers the chance to notice, review or approve such changes because the download link points to an unversioned resource, a versioned resource has been modified or tampered with or due to an insecure data transfer.

    Unknown Origin: Details about the source code, build process or the distribution process of a software component may be unknown or non-verifiable.

    Endor Labs CEO Varun Badhwar said organizations need more visibility in the potential operational risks that come with increased reliance on open source software. That doesn’t mean organizations should use less open source software, but there are issues that many development teams may not always appreciate. A recent analysis of nearly 2,000 software packages published by Endor Labs, for example, found 95% of all application vulnerabilities can be traced back to a transitive dependency created when a developer employed an open source component.

    Regardless of the root cause, responsibility for application security continues to be pushed left toward application developers. The challenge is it’s not possible to achieve that goal without first knowing what issues developers and the DevSecOps teams that support them need to be focused on. Perfect application security is, of course, unattainable, but any reduction in the number of vulnerabilities littering the software landscape will go a long way toward reducing the current level of stress developers and DevOps teams experience.

  • Managing Risk

    Managing Risk

    We have built some beautiful toolchains that crank out a finished product on the fly without needing anything close to the level of intervention that was historically required. The most advanced organizations on an automation journey could change a line of code and then wait for the new version to hit production without doing a thing. I say “could” because I don’t know of a single org that does this; there are, of course, checks in place and it’s not single-line changes but projects that release. But the infrastructure is there that would allow them to do this. It’s a high probability that one day we all will.

    The problem is that most organizations only look at risk when building these toolchains and don’t bother with contingency plans. But over the long life of any given toolchain—toolchains that are made up of many moving parts—there will inevitably be the need for contingency plans.

    And it’s certainly not only toolchains—remember when popular libraries were purposely corrupted by their author to make a point? And unless you’ve done your best to ignore it, you’re also aware of faked versions of popular libraries that insert code before continuing on to the original…

    Most organizations do not consider a scenario in which critical libraries are suddenly unavailable or a scenario where a tool is crippled by movement in the market—a vendor making harsh licensing changes, the underlying tech making changes that impact the tool, etc. While it is comforting to think that open source protects us from many changes (and it forking does), that is not complete coverage that entirely protects our systems and toolchain. We need to know what is absolutely critical and how the organization will keep delivering in case that absolutely critical element goes away.

    When I have done disaster planning—of which this is an adjunct—I have done so with a vague generality in mind. We have plenty of work to do, so all I ever wanted was a list of critical parts and an idea of how we would adapt without them. I worked at a place that kept a backup data center in case of total regional disaster—the backup data center was intentionally located on the other side of the country—but most of us don’t need that kind of DR. Certainly not if we can go, “This OSS library is used in 60% of our apps; if we have to, we could adapt this other one,” instead of building infrastructure.

    For OSS dependencies, we have two great options that work together. Generate SBoMs, then correlate usage from those across the entire app portfolio. Then set up and maintain a local repository with point-in-time copies of those libraries that are deemed critical.

    A similar, “Our CI tool is critical, but we could easily port processes and YAML to this other one,” is enough, too—it is absolutely not going to be easy if you ever have to do it, but “We looked at this six months ago and we know what to do,” is far better than “Oh crap, now what?”

    You built it, you operate it, you are rocking it. Just take some time to protect it and you are covered.

  • groundcover Launches OSS Cluster Map

    Tel Aviv, ISRAEL, January 9, 2023 – groundcover, a start up with a mission to reinvent the cloud-native application monitoring domain with eBPF, launches Caretta today: an OSS tool which helps teams instantly create a visual network map of the services running in their cluster. Caretta leverages eBPF technology to collect data in an efficient way and is equipped with a Grafana Node Graph dashboard to quickly display the dynamic map of the cluster.

    The main idea behind creating Caretta is gaining a clear understanding of the inter-dependencies between the different workloads running in the cluster which is an otherwise very complicated task.
    Caretta maps all service interactions and their traffic rates, leveraging Kubernetes APIs to create a clean and informative map of any K8s cluster which can be used for on-demand granular observability, cost optimization, and security insights, allowing teams to quickly reach impactful insights such as identifying central points of failure or pinpointing security anomalies.

    Caretta is part of the mission of cloud-environment observability tools – no APM product is complete without network tracing capabilities, and the aggregated data from these traces can help answer many critical questions. Carreta does exactly that and fits those cases where you want a minimal, fast and ad-hoc solution that can help you right now.

    To read more about Caretta, groundcover’s new OSS Kubernetes service dependency map tool, read this blog post.

     About groundcover

    groundcover is a K8s application monitoring solution that reinvents the domain with eBPF. Built for modern production environments, it covers everything yet stores only what matters, allowing teams to scale away without worries. Founded in 2021 by Shahar Azulay (previously ML Manager, Apple) and Yechezkel Rabonivich (previously Chief Architect, CyberMDX). The two served together in an elite cyber unit in the Israeli Prime Minister’s office where they dealt with frustration caused by APM issues. The company has raised $24.5M to date from VCs such as Zeev Ventures, Angular Ventures, Heavybit and Jibe Ventures.

  • TechStrong Con: Open Source Software Community Needs Security Help

    TechStrong Con: Open Source Software Community Needs Security Help

    The only way to make significant improvements in the state of open source security is if more organizations that benefit from open source projects commit to making more resources available to achieve that goal.

    At the virtual TechStong Con event, executives on an Open Source and DevOps panel called for more contributions from enterprise IT organizations to ensure the integrity of open source software.

    Organizations of all sizes now benefit from open source software whether directly or indirectly. Those organizations have a responsibility to participate in the development of that software in any way they can, said Jeffrey Barnes, senior software engineer for Remote Operations, a provider of medical transcription services. “We need to give back to the community,” he said.

    Those efforts don’t necessarily have to involve contributing actual code, added Barnes. Many open source projects are in need of documentation and testing help because the number of actual maintainers of the project is relatively small, he noted.

    The challenge is that many enterprise IT organizations don’t understand how to participate in open source software development processes, noted Tom Sweet, CIO for Industrial Refrigeration Pros. An organization might download a piece of open source software and then extend it. Rather than continuing to maintain that branch on their own, organizations should contribute that code back to the project to reduce maintainers’ overall IT burden, Sweet said.

    “Many organizations are struggling to contribute back,” said Sweet. “It’s an evolution.”

    Organizations clearly need to make contributing code back to open source software projects a natural extension of their DevOps workflows. In the meantime, the Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation, is focusing on 10 streams of investment that, in total, would require more than $150 million in funding to drive greater adoption of DevSecOps best practices among maintainers of open source software projects.

    The core issue is that many of those projects are maintained by a small number of programmers that voluntarily contribute their time and effort to build components that others are free to use. Like any other developer, the amount of security expertise those individuals have is limited. The onus for making sure the projects and software are secure is on the organizations that decide to deploy that software.

    Unfortunately, many IT vendors and large enterprise IT organizations reuse that code without contributing anything meaningful back to the project—whether that be in terms of financing or helping open source maintainers find and remediate vulnerabilities.

    It’s not clear to what degree business and IT leaders realize how dependent their organizations actually are on open source software. Many of them assume that open source projects are supported by IT vendors when, in reality, there are many projects that are dependent on the efforts of a small handful of developers volunteering their time and skills.

    Many organizations, of course, may soon decide not to employ open source software if it doesn’t have a critical mass of developers working on it. However, in an era where DevOps teams and developers routinely download software, enforcing those policies is going to be problematic. The best thing enterprise IT organizations can do is engage with the open source software community more rather than less.

  • Sysdig Open Source is Extended to Secure Cloud Services

    New integration enables any Falco plugin to be used for Sysdig OSS

    VALENCIA, SPAIN, (KubeCon + CloudNativeCon Europe), May 16, 2022 — Sysdig, the unified container and cloud security leader, announced that Sysdig open source, the incident response standard for containers, has been extended to the cloud. Using system calls, Sysdig open source (Sysdig OSS) traditionally offers deep observability into running applications, as well as file system access and network activity, which speeds incident response and troubleshooting. Teams can quickly filter information from Sysdig OSS and take action. With the announcement of this new integration, these capabilities have been extended beyond containers to any cloud environment.

    The complexity of cloud-native applications – with countless components and variables – makes it extremely difficult for security analysts and system administrators to quickly triage alerts and debug problems. Sysdig OSS captures process, file system, and network activity in real time and with a high degree of granularity. The tool, which has nearly two million downloads and 6,850 GitHub stars, surfaces everything from executed commands and file system activity to network activity. Sysdig OSS then offers advanced filtering and troubleshooting capabilities, supporting root cause analysis for security and performance issues.

    Using a new plugin framework – originally developed by the open source community for the CNCF project Falco – Sysdig extends the number of sources Sysdig OSS can be connected with to anything that generates logs or events, including Azure, Google, and AWS CloudTrail logs. Going forward, every plugin developed for Falco can also be leveraged by Sysdig OSS. Using one tool, like Sysdig OSS, to observe events from the entire cloud-native environment streamlines investigations. Using a different tool for each environment adds complexity, which makes it massively harder to troubleshoot.

    Learn more about this framework in the Sysdig OSS 0.29 new release blog

    Sysdig’s Commitment to Open Source

    Sysdig was founded as an open source company and Sysdig Secure and Sysdig Monitor were both built on an open source foundation to address the security challenges of modern cloud applications. Both projects were created by Sysdig to leverage deep visibility as a foundation for security, and they have become standards for container and cloud threat detection and incident response. Falco, which was contributed to the CNCF in 2018, is now an incubation-level hosted project with more than 45 million downloads.

    Sysdig OSS and Falco can be used together as a powerful open source solution to reduce risk at runtime. Sysdig OSS acts as a flight recorder, capturing a detailed record for inspection. Falco acts as a security camera, continuously detecting unexpected behavior, configuration changes, intrusions, and data theft in real time. Teams can use Sysdig OSS and Falco together to detect and respond to threats.

    “If you want to see what is going on inside an application, Sysdig OSS gives you that record,” said Loris Degioanni, Founder and CTO of Sysdig. “Sysdig open source was the inspiration for Falco. While Falco will monitor and alert based on your policies, Sysdig open source will tell you what happened at a particular time, before and after the event. Having the ability to use both open source tools in the cloud is extremely powerful.”

    Resources

    ●  Read the New Release of Sysdig Open Source Leverages Falco Plugins.

    ●  Find Sysdig OSS on GitHub.

    ●  Join the Sysdig OSS Slack community.

    About Sysdig
    Sysdig is driving the standard for cloud and container security. The company pioneered cloud-native runtime threat detection and response by creating Falco and Sysdig as open source standards and key building blocks of the Sysdig platform. With the platform, teams can find and prioritize software vulnerabilities, detect and respond to threats, and manage cloud configurations, permissions and compliance. From containers and Kubernetes to cloud services, teams get a single view of risk from source to run, with no blind spots, no guesswork, no black boxes. The largest and most innovative companies around the world rely on Sysdig.

    Media Contact
    Amanda McKinney Smith
    amanda.smith@sysdig.com
    703-473-4051

  • Agile Development at 20: Can it Thrive in the Hybrid Work Era?

    Agile Development at 20: Can it Thrive in the Hybrid Work Era?

    The Agile Manifesto turns 20 this year, and while many of the core tenets are still applicable, the world has changed considerably in the last two decades. What would the Agile Manifesto look like if it was written today and took into consideration our new reality of hybrid working environments? Changes are most definitely needed, as software leaders still find themselves struggling to navigate the murky waters of 21st-century project management to get the most out of their team.

    Developers have many complaints about Agile, but most of them boil down to the fact that they often spend more time planning for work than doing it. For example, developers sometimes feel that frameworks like Scrum are top-heavy and bureaucratic, which is the opposite of their intended purpose. Developers can also feel disconnected from the bigger goals of their project, being trapped in an endless list of “to-dos” without the time to think holistically about the features they’re creating.

    With these concerns in mind, let’s take a quick look at what the original Agile Manifesto actually included. It started with four primary values:

    • Individuals and interactions over processes and tools
    • Working software over comprehensive documentation
    • Customer collaboration over contract negotiation
    • Responding to change by following a plan

    On their faces, all of these seem reasonable; even approaching plain ol’ common sense. But in their zeal to focus on code development rather than bureaucracy, the authors couldn’t have anticipated how much the industry would change. Consider some of the trends we’re seeing in today’s software development teams.

    First and foremost, the idea that “every company will be a software company” has become a reality. It’s hard to imagine an industry that doesn’t leverage software in some way to address its core competency. This trend has, in turn, fueled another major trend: The war for developer talent. Skilled developers were already at a premium prior to the Great Resignation in the wake of the pandemic. Now, developers hold substantial leverage over their employers in ways never seen before.

    The Pandemic and the New Normal of Hybrid Teams

    The pandemic has also brought us the new normal of remote, distributed teams and asynchronous work styles and skilled workers are spread across the globe. There are also new technologies that enable more automation for software teams, including machine learning and AI and other methodologies (like DevOps and DevSecOps) that reduce workloads for developer teams by using automation and process-driven workflows.

    These trends have given rise to particular needs for almost all hybrid developer teams. They include significantly improved productivity for faster software delivery and developer-friendly tool experiences that speak directly to how developers work today. Teams also require guided, asynchronous processes that map to the reality of highly distributed teams. Finally, as the cloud era is firmly a reality, developers need new technologies that offer the ability to scale predictably without suffering from increased complexity.

    Now, let’s look at the original Agile values through the lens of today’s trends and needs. The first is valuing individuals and interactions over processes and tools—this definitely requires an update for today’s developer teams. Specifically, process is a prerequisite for productivity. It is an enormous challenge to measure productivity or determine if the team is improving without a consistent process to follow. Without agreed-upon rules for working together, software development can quickly devolve into chaos. Second, putting individuals and interactions over tools seems like a restrictive viewpoint today. Good tools are explicitly designed to facilitate productive interactions between individuals. If they don’t accomplish this goal, they are poor tools; but there are many solid reasons that great tools can increase productivity and make developer teams’ lives much more manageable.

    The second value of the Manifesto also seems to present a false choice, as it prioritizes working software over comprehensive documentation. Its heart is in the right place—who wouldn’t want to focus on building workable code rather than creating documentation? However, in the world of distributed teams, detailed documentation is critical for both remote and hybrid teams. This also encompasses documentation that goes beyond just the codebase. Documentation involving decisions, requirements, process, etc. also provides context for teams that are adjacent to developer teams and help them understand progress and goals. Documentation is also helpful for addressing the skills gap and talent shortage. New hires or newly-promoted team members need to get up to speed quickly and provide value as quickly as possible. In addition to training, good documentation can help them do that and help their teams be more productive more quickly.

    The third value of the Agile Manifesto said that we should focus on customer collaboration rather than contract negotiation. I believe this needs a slight but essential adjustment for today’s hybrid teams. Specifically, teams should be building with end-users,—not just “customers”—in mind, which means that the needs of whoever will be using the end application regularly should be paramount. This is particularly important today given how many end users are involved in buying decisions.

    I think the final value of the Agile Manifesto—responding to change over following a plan—needs no edits whatsoever. This remains great guidance for developer teams everywhere, and it seems the Manifesto should take this advice and adapt to the many changes that have occurred since it was first written.

    Many of the 12 principles of the Agile Manifesto continue to be outstanding guidance that is applicable for most developer teams. There is, however, some nuance that should be considered for a couple of them. To wit:

    “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”—This is true, but today, remote and hybrid teams are a reality. That’s not changing any time soon. Teams need to optimize for asynchronous communication and collaboration.

    “Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a constant pace indefinitely.”—Sure, but given the myriad demands on software teams, developer burnout needs to be monitored and managed or overall quality and productivity will suffer.

    “Simplicity—the art of maximizing the amount of work not done is essential.”—I couldn’t agree more, but in my mind, this means that teams should leverage automation where possible to maximize “work not done.”

    Conclusions

    In general, it seems that the Agile Manifesto holds up pretty well, provided that today’s workplace realities are taken into account. If software teams recognize that tools and processes create conditions for greater productivity and that documentation is critical in a remote/hybrid world, their remote Agile teams will be much better off. In addition, teams should build with end-users in mind, not just customers. With this added nuance and recognition of a changing world, today’s software teams can continue to thrive while leveraging this decades-old advice.

  • Secure Software Summit: Measuring and Mitigating OSS Risks

    Secure Software Summit: Measuring and Mitigating OSS Risks

    Measuring and mitigating the security risks in open source software is becoming a major issue in the software development community. Attacks on open source software (OSS) are on the rise; open source supply chain attacks grew 650% in 2021—a staggering number.

    2021 and early 2022 saw significant attacks which impacted more than 1,000 organizations. The most prominent recent OSS attack was the Log4Shell vulnerability disclosed in December 2021. Attacks on the OSS supply chain and on OSS impacted nearly every single software organization on the planet. Everybody is scrambling to continuously patch systems as new vulnerabilities continue pouring in.

    To properly navigate this environment and keep our systems safe, we as a community need to understand the security risks of OSS and be better prepared to handle them, collectively and individually. 

    Open Source Risks

    There are a number of risks that make OSS more vulnerable without proper security measures. 

    OSS is Free; OSS Security is Expensive

    To start, we must recognize that open source software may be free to download and require no payment, but it comes with a massive security cost. 

    A significant portion of open source software is developed by volunteers in their free time. Most of the prominent open source volunteers actually have full-time jobs, and the code they contribute to projects like the Log4j logging libraries are written on nights and weekends; on their own time and not on the company dime. Not surprisingly, security isn’t high on their priority list. They don’t have the resources to invest in a secure development life cycle or have the time and bandwidth to maintain their code. This may mean that security vulnerabilities remain unaddressed for long periods of time, sometimes indefinitely. 

    All Bugs are Shallow, but There Aren’t Enough Eyeballs

    In software development, Linus’s law states “…given enough eyeballs, all bugs are shallow.” However, this is only true if those eyes are looking in the right place. In the case of open source, all code is open but there aren’t enough eyes paying attention to it to spot all the bugs. In addition, organizations may not consider the security consequences of their decision to consume a particular piece of open source software and adding it as a dependency. Dependency trees become more and more complex; harder and harder to monitor and secure, as you keep adding more third-party components to a core codebase. Each new dependency might bring additional nested dependencies—perhaps hundreds or thousands, depending on the size of the library added or the role it plays in the application architecture. For example, the Kubernetes project—one of the most successful and widely adopted OSS projects—has several thousand dependencies, many of which have additional secondary and tertiary downstream dependencies. 

    So Much Code. So Many Choices. Not all are Good.

    The beauty of the open source ecosystem is that there is a solution for everything. The problem with the open source ecosystem is that there’s not just one solution, but several dozen. For example, if you need a JSON parser for your application, there are several to choose from. If your application is running a critical service or infrastructure, then picking the right JSON parser—one that is secure and comes from a responsible project or maintainer—becomes very important. A JSON parser will serve as a critical component for your application, so it will impact the security of your entire system. If you chose the wrong JSON parser, you might pay a big price down the road in security risk and subsequent breaches. 

    Proposal: An Entry Gate Security Risk Assessment

    One potential solution to enhance OSS code security is the creation and implementation of a security risk assessment gate as part of your OSS ingestion pipeline. Putting this process in place will help you understand whether consuming a certain piece of software is high risk and avoid software that does not meet your risk criteria. 

    Here is an example of how this might work. Consider five different JSON parser libraries in GitHub, all of which have a high number of stars and appear to be highly rated. How can we assess the risk of these libraries and quickly and easily eliminate ones that are too risky? One of these JSON libraries is hosted under a user account. Most of the commits are done by a single developer without reviews. This alone indicates that using this library in a critical system presents a high risk. This lone developer could commit malicious code without review. The maintainer’s token access could be leaked, allowing attackers to insert malicious code. The developer might also turn over committer control of their library to a malicious actor without notifying users (who might not pay attention to the notification anyway). Identifying this risk required no more than a few minutes of review of the repository and cost little in time or effort. 

    A Broader View of Security Posture for OSS

    More broadly, we need to understand the full security posture of an organization or developer and their open source code prior to adding it to critical systems. Because OSS is a complex ecosystem and many pieces of OSS are complex, the risks they generate are similarly complex. Our traditional, myopic view of these risks is that we just need to find all the security vulnerabilities in our codebase and fix them. This does not resolve all risks and may lull us into a false sense of security.

    We need to understand the full comprehensive security posture of an open source project. That involves understanding a broad range of risks beyond code vulnerabilities. Here are some of the key risks to consider beyond ensuring that your codebase is well-secured and patched. 

    Supply Chain Attacks on the Build Pipeline

    Attacks like SolarWinds and CodeCove taught us that supply chain risk is a very big deal and that tools are just as much at risk as code. Attackers have found it much easier and much more reliable to attack build pipelines, rather than finding vulnerabilities in code.

    Security Risks Revealed by Continuous Security Testing 

    Continuous security testing is critical to maintaining the security of a codebase. There might not be code vulnerabilities today, but as developers continuously modify code and iterate on applications, vulnerabilities may be introduced. This can mean testing code as part of the CI/CD process with every new code push. For example, we have found numerous critical bugs in OpenSSL libraries. All were caught on a master or main branch and never made it to a production release. 

    Project Maintenance and Capabilities

    One of the really big risks with open source is the quality and frequency of project maintenance. Issues raised may not be addressed in a timely fashion. Maintainers of a project may have a long backlog of issues, indicating that they are not actively maintaining a project. It is not uncommon for critical open source projects used in millions of applications and comprising a significant dependency to the global software supply chain to not have been updated in a year or more. In addition, maintainers of OSS projects may not know how to respond to security incidents and fix security vulnerabilities. This is particularly true if the maintainers do not work at a large organization, do not have cybersecurity or secure development experience or have never suffered through the toil and trauma of addressing a major vulnerability in their code on short notice. 

    Tools for Measuring and Addressing Supply Chain Risk

    A number of efforts are underway to create free tools to help developers and application security teams better measure and address supply chain risk programmatically and analytically. 

    SLSA—Supply Chain Level for Software Artifacts

    As a tool to help organizations better address software supply chain risks, Google created a novel framework that we call Supply Chain Level for Software Artifacts (SLSA). The project has been underway at Google for more than a decade and has been open sourced as part of the OSSF. Contributors include VMware, Citibank and Datadog, among others. Google hopes to standardize SLSA and make it an industry benchmark for evaluating supply chain risk. Right now, SLSA is focused primarily on build process risks but the hope is to expand it in the near future to other areas, such as vulnerability and maintainer risks.

    OSS Security Scorecards Project

    In order to scale the process of assessing, identifying and mitigating risks in OSS, we need robust automation. In 2020, as part of the OSSF, Google released a new project called Open Source Security Scorecard. The project aimed to automate the analysis and trust aspect when evaluating the security posture of open source code. Further, the Scorecards can provide specific security recommendations. The Scorecard process can incorporate many other automated tools, including checking against vulnerabilities using the Open Source Vulnerability (OSV) database and running dependency checks using open source dependency trackers like Dependabot. The scorecards project also makes it easier to report new vulnerabilities and get those to the right people in a project. 

    Conclusion: What is not Measured is not Mitigated

    No one denies that OSS has become crucial to the vast majority of applications running the world’s technology foundation. The span of OSS continues to grow, as more and more languages and frameworks emerge and specialization increases. No one wants to reinvent the wheel and rewrite fresh code to perform the same tasks with every new application. The model of reuse, dependency and distributed software development works and speeds innovation. That said, the open source community and the supply chain used to build OSS need a more rigorous approach to reducing risk. Given the scope of the problem, automation and standardization are helping to scale toward the stated goal. We are in the early stages of creating the tools and infrastructure to automate the risk identification, measurement, assessment and mitigation processes. Security frameworks like SLSA and new processes like entry gating new OSS code with security scorecards will direct organizations down that path and move us toward a more secure open source future.

    About Secure Software Summit
    The Secure Software Summit brought together the world’s leading innovators, practitioners and academics of secure software development to share and teach the latest methods and breakthroughs on secure coding and deployment practices. If you are about developing, releasing and securing software, delivering new features fast and building things right from the start, click on the link below to get access to all of the sessions from this summit.

    https://go.shiftleft.io/secure-software-summit-2022-replay

     

  • Secure Software Summit: The State of OSS Supply Chain Security

    Secure Software Summit: The State of OSS Supply Chain Security

    The open source software (OSS) supply chain is under attack. As evidenced by the recent Log4Shell vulnerability, the OSS supply chain is increasingly a focus for attackers seeking to exploit weak links in security. A number of research reports have recorded a significant increase in so-called ‘next-gen software supply chain attacks‘ over the past decade. These attacks look to compromise key links in the software supply chain including source code, build systems, CI/CD tools and code testing tools, among others. Next-gen attacks are happening across all languages, all types of software and are increasing in their frequency. Common forms include typosquatting, malicious code injection and tool tampering (like the CodeCov incident). In 2021, 64% of organizations reported they were affected by software supply chain attacks, according to container security company Anchore. 

    The increasing wave of next-gen attacks has driven new regulations mandating specific steps to inventory and make more transparent the content of software applications and the software supply chain. Specifically, executive order 14082, issued by U.S. president Joseph Biden, mandated a long list of security practices, including the inclusion of a software bill of materials (SBOM) with every application run by the U.S. federal government. The National Institute of Standards and Technology (NIST) is creating reference architectures and templates for application security as a result of the executive order. In other words, software security regulations will likely impact every company that produces code or sells products running on code (including SaaS products). 

    To be clear, some of the problems experienced by the OSS software supply chain are not exclusive to open source. The recent supply chain attacks on Microsoft Exchange Server and the SunBurst attack on the SolarWinds network management tool targeted proprietary code bases. That said, OSS is produced and maintained very differently than proprietary software. This does result in some unique risks. Here, we will cover those risks as well as some promising initiatives underway to address them.  

    Why Is the OSS Supply Chain A Juicy Target?

    There are two primary reasons OSS supply chains are a tempting target. One is its omnipresence and the other is its weakness. It is not an exaggeration to say that OSS is omnipresent. According to Sonatype, 98% of all codebases contain OSS. This is likely an underrepresentation. You simply cannot build code from scratch today without using OSS components. It is too labor-intensive and expensive. The ubiquity of OSS makes the supply chain that creates, distributed and manages it an obvious target. At the same time, that widespread use translates into an increasing burden on companies to maintain, update and patch all OSS dependencies in their applications. 

    This is an impossible task; it is far easier to insert OSS code to solve an immediate problem than to, over time, make sure that code is updated and not vulnerable. For this reason, Sonatype estimated that 92% of all applications contain outdated or vulnerable code. We have seen the impact of this inability of organizations to maintain their OSS code through attacks like the one perpetrated against Apache Struts software run by credit reporting company Equifax, a breach that exposed the sensitive personal and financial information of hundreds of millions of Americans.  

    In recognizing that OSS is a fruitful channel for attacks, attackers have focused more specifically on the supply chain because it is often the weakest link. Elements of the supply chain reside in the gap between responsible organizations and may be less strictly policed or less actively secured. Continuous integration/continuous deployment (CI/CD) tools, OSS libraries that can easily access other systems, package managers and open source developer tooling are all components of the supply chain that are widely used but which are often the sole responsibility of many part-time maintainers who write and maintain code as an unpaid avocation. 

    OSS-Specific Challenges

    The three key benefits of OSS each create unique security and supply chain challenges. Specifically, OSS is open, free and transparent. 

    The Two Sides of Open

    Linus Torvalds said that many eyeballs make all bugs shallow. To a degree, this is true. But what if those same eyes are looking for bugs to exploit rather than fix? Not everyone on the internet is nice and many seek to exploit the fact that anyone with a GitHub account can contribute code to critical libraries. This attracts a lot of great contributors, but also a lot of bad actors and bad behavior. 

    That behavior is often hidden or delayed. For example, we often see bad actors ask to contribute to projects or libraries run by single maintainers that are relatively popular and used by big applications. Often, maintainers struggle to keep up with the demands of these projects. A bad actor will offer to help, submit a handful of good PRs and gain commit privileges to the repository. At this point, the bad actor will then stealthily insert malware, such as a cryptojacking script. 

    Free Puppy. Not Free Beer. 

    Free is a good price. There are no purchase orders. This makes it super easy to use. Free also serves as a great distribution strategy. However, sometimes free is not worth it. If an organization depends on an application and it becomes critical, they might require fast patching, better compliance and official support relationships. Free OSS comes with no guarantee, and this can be a major liability. An application’s maintainers might be writing the code and applying fixes in their spare time on weekends because they already have a full-time job. In fact, research has shown that a significant percentage of the critical software powering the internet is maintained by volunteers; this was the cause of the Heartbleed vulnerability in the OpenSSH encryption libraries. 

    Free code also may not be updated for years. If a vulnerability is disclosed, you may not be able to get permission to merge fixes you write. If the code is abandoned and the author no longer checks comments or requests, then you are stuck and may have no choice but to fork the project. A hard fork on short notice can cause chaos and its own problems. This is what happened with a popular Golang library called JWT Go, which made its way into dependency trees across the Golang ecosystem. JWT Go was even included in Kubernetes, one of the most popular and important open source projects. When a vulnerability to JWT Go emerged, everyone realized the maintainer had abandoned it. Multiple forked versions emerged with different patches for the same bug. Chaos ensued. 

    It is a common refrain to ask companies to fund volunteer maintainers. There are numerous programs for this, including GitHub Sponsors, Open Collective and Tidelift. These programs are helpful but not sufficient. You can’t just hand people a small amount of money and get more secure code and professional-grade maintenance. Even if you offered full-time salaries, it might not translate into well-supported, secure code. 

    The Illusory Benefit of Transparency

    The transparency of OSS is real, but the idea that this code is getting audited and reviewed constantly is an illusion. Yes, the source code is distributed and open. But the sheer quantity makes it impossible to review all the code you use. If you are using millions and tens of millions of lines of code, you don’t have time or sufficient eyeballs. Even if you are spending considerable resources on code review, it is still easy to miss problems and vulnerabilities and relatively simple for malicious contributors to projects to hide malware or other bad inclusions. Reviewing the source code also does eliminate all risks. When most OSS code is consumed, it is running as a package compiled by another entity or developer. 

    Further, there is no guarantee that the contents of the package comes from the same source code you audited. We have actually seen this attack often with package repositories like npm and Maven, where malicious actors insert tampered builds or take over packages and insert code that is not listed as part of the package. 

    The CodeCov incident is one of the more recent and egregious examples. This free service for checking the code coverage of unit tests used BASH script running inside of a CI server for installation. The script was compromised and attackers inserted malware that could steal code secrets and export them. Those secrets might be tokens used to publish packages in npm or Maven and exposing those secrets allowed attackers to then compromise those packages. In another example, a PHP server managing source code distribution for the PHP project was compromised and multiple commits injected malicious code into the source code repository. The incident was caught quickly, but it demonstrated how attackers could tamper with source code directly, bypassing code review and security processes.  

    But All is Not Lost

    Innovative efforts to better secure open source and its supply chain are gaining momentum. The SBOM is gaining acceptance and it is getting easier to automatically generate SBOMs from CI/CD pipelines or build systems. A new initiative to make it easy to cryptographically sign code, called Sigstore, aims to remove barriers to code signing; ubiquitous code signing would make it much harder to compromise the supply chain by attacking the source code provenance problem at its root. Lastly, the Open Source Security Foundation has garnered tens of millions of dollars in funding from major technology companies and is in the process of spinning up mechanisms to issue grants and otherwise provide a more robust financial support mechanism for OSS. These developments point to a brighter future where the OSS Supply Chain is far more robust and companies can more easily audit, track and trust all the OSS dependencies in their applications and the full supply chain that moves OSS code from developer boxes up into production environments. 

    About Secure Software Summit
    The Secure Software Summit brought together the world’s leading innovators, practitioners and academics of secure software development to share and teach the latest methods and breakthroughs on secure coding and deployment practices. If you are about developing, releasing and securing software, delivering new features fast and building things right from the start, click on the link below to get access to all of the sessions from this summit.

    https://go.shiftleft.io/secure-software-summit-2022-replay