Tag: open source

  • Securing the Software Supply Chain with Behavioral Analysis

    Securing the Software Supply Chain with Behavioral Analysis

    Lately, software supply chains find themselves in a very interesting and uncomfortable position—the industry spotlight—and not in a good way. While significant and costly breaches such as  SolarWinds or Kaseya make front-page news, supply chain attack tactics (e.g. typosquatting or dependency confusion) that target package managers such as npm, PyPI or WinGet can poison downstream applications or compromise developer credentials.  

    The software engineering community has been steadily increasing its use of third-party and open source solutions to help create new offerings. In fact, over the last five years, there has been a 259% increase in the use of open source.

    One reason for this is the simple fact that creating new applications takes time and can strain budgets that are already tight. With open source code, software engineers gain fast access to pre-existing code that’s already demonstrated a proven ability to address the very unique challenge they are looking to solve. By removing the need to reinvent the wheel, the development timeframe is accelerated, the costs are lowered and teams are freed to focus on the creation of new and unique features that the customers demand.

    At this point, turning to third-party code may seem like a no-brainer. However, it can also open the door to potential risks. This is especially true with engineers who, rather than checking software to ensure it is secure and compliant, take the providers’ assurances as gospel. 

    They are not alone. Other groups will try to conduct their initial due diligence at the onset if that’s possible. Soon after they then ease into a “set it and forget it” approach, where they fail to update the code over time. As a result, while the code may appear secure initially, vulnerabilities can emerge over time, and if the software is not continually updated, bad actors can leverage an unprotected window of opportunity. In fact, according to Enterprise Security Group’s “Modern Application Development Security” eBook, 48% of organizations regularly push vulnerable code. The same research also found that 60 percent of organizations have had production application vulnerabilities exploited.

    Organizations conduct static application security testing (SAST) and dynamic application security testing (DAST) to assess pre-production code vulnerabilities. They will also scan with AV before releasing software, examine signing certificates, and code quality. Yet, despite these efforts, they remain challenged by gaps in security processes and the ability to sufficiently inspect large, complex and multi-layered software packages. What’s missing is the ability to conduct in-depth analysis to pinpoint behavioral differences between compiled software versions.  

    Malicious tampering can often involve changing how the software interacts with the system on which it is deployed.  Differential behavior analysis highlighted below makes it easier to detect those unusual changes. For example, having an application suddenly add a pre-install script that downloads files via certutil is highly unusual. If the addition was planned, reasons for the change would be well documented and it would be associated with a significant new feature. For this type of change to happen in any other scenario is suspicious and deserves further investigation before the software is released. In this case, such action would have caught ua-parser-js developer’s account compromise.

    supply chain

    With the software supply chain at risk and with shared code the new normal, what’s needed is software assurance. 

    Software assurance allows teams to inspect every application layer within an installation package for the presence of software tampering, malware and other indicators of software supply chain attacks.   

    Begin the software assurance process with the deep inspection step through recursive package decomposition. The resulting software bill of materials (SBOM) becomes so much more than a build manifest (a list of components that should be in the final build). It becomes the means to determine exactly which components have unacceptable build quality or those that contain unwanted software behaviors. This enables developers to verify the code they are working with and take appropriate action to protect their customers and partners from the majority of software risks. These assessments include:

    • Software grading: The software package and every included component can be graded based on the types and severity of security issues found. Using a grading system makes it easy for software buyers to gauge the level of risk posed by a software package and determine whether additional deployment measures can be taken to mitigate those risks. For developers, component grading makes it easier to prioritize their remediation efforts and aids in the selection of higher-quality third-party components.  
    • Application hardening: Activating specific build or compiler options can mitigate vulnerabilities and protect applications against attack—but only if they are implemented correctly. Assessing whether these mitigations are correctly implemented ensures that these protection mechanisms will work as expected. 
    • Binary behavior analysis: This allows teams to fully understand how each software component interacts with the system on which it will run. The ability to understand third-party component behaviors, even when developers have no access to the source code, delivers a more comprehensive understanding of supply chain risks. Checking for unusual software behaviors or unexpected behavioral changes between one version and the next is often the only way to identify software tampering by a malicious actor. The ability to have this final check before distribution enables software developers to validate that the software being released is trustworthy.

    For companies that are using software to create business value or producing software that is mission-critical for their customers, turning a blind eye or making software assurance assumptions can no longer be tolerated. In today’s climate where increasingly sophisticated threats loom, inspecting software and monitoring updates for signs of tampering, malicious or unwanted additions is essential and must become a part of standard software development practice. 

  • CI/CD Is Still All About Open Source

    CI/CD Is Still All About Open Source

    One of the lessons I’ve learned in my years at the helm of devopsy.kinsta.cloud is that the heart of DevOps is CI/CD. And at the heart of CI/CD is open source. While the names have been changed to protect the innocent (and the obsolete), these lessons are as true today as they ever were. CI/CD is still all about open source.

    When we first started devopsy.kinsta.cloud, folks said that DevOps was just a symptom; CI/CD was the cure. For many of you, that cure was delivered via open source—probably using Jenkins. Jenkins, which itself was a fork of the open source Hudson project, was created by Kohsuke Kawaguchi.

    Jenkins was a CI tool at heart and later morphed into a CI/CD tool. Many people think that this fork in the road may have hurt the continued evolution of continuous delivery in the long term. But that is an argument for another devopsy.kinsta.cloud article (or maybe even a panel discussion at an upcoming DevOps live event).

    Regardless of where you stand on that issue, as an open source project, it is hard to argue with the success of Jenkins. Driving a lot of that success is the Jenkins plug-in architecture. There are literally thousands of plugins that allow Jenkins to work with just about anything. That is the engine that powered Jenkins, yes; but its secret superpower was and is open source.

    That said, Jenkins has grown a bit long in the tooth over the years. It’s not that it doesn’t do what it always did, it’s that what we do and how we do it has changed. Microservices, Kubernetes and even cloud have changed the very fabric of the tapestry in front of which Jenkins sits. The open source community that supports Jenkins should receive enormous credit here: It has tried mightily to keep up with the many changes. 

    Jenkins X sought to bring Jenkins to Kubernetes, for instance. But ultimately, nothing lasts forever and, as it says in the Bible, “For everything there is a season, and a time for every purpose under heaven: A time to be born, and a time to die.” Now, I am not saying Jenkins is dying, but I think its time as the leader of the pack is behind it.

    The good news is there are a host of new CI/CD tools that have risen to fill the vacuum left by Jenkins. Some of them are entirely open source; some are based on open source. Solutions such as Buildkite, Harness, Codefresh, GitLab, GitHub, CircleCI, JFrog and Tekton to name but a few. 

    Another major development has been the rise of GitOps. Weaveworks has led the way in adoption, not to mention coining of the term GitOps itself, but clearly, they are not alone. What these solutions all have in common, though, is open source.

    I used to think it was sort of a chicken-or-the-egg question when it came to CI/CD and open source. But I realize now that it is not. I firmly believe that, if not for open source, CI/CD would not be the force it is today. Make no mistake, it is truly a force. CI/CD has become the dominant method for software development.

    Of course, you could say “Why stop at CI/CD? Open source has become the dominant software development foundation in observability, OS, cloud and so much more!” Yes, I agree—we are living in the open source age. This rising tide may in fact lift all boats, including CI/CD.

    This wave is forcing all vendors to either go entirely open source or at least support open source. What’s more, for the foreseeable future, I don’t see that changing. Open source has also wrought other changes in the market. How software tools, including CI/CD, are distributed and consumed. Who is hiring enterprise software salespeople anymore? Doing large POCs? No, open source has resulted in getting software into the hands of developers and DevOps teams directly and delivering delight. If your product works, they buy it. If it doesn’t work, they won’t, either—and they will let you know, too!

    This meritocracy is another benefit of open source and is helping to speed the development of better, next-gen CI/CD tools in the market. And at the end of the day, we are all better off for that.

  • WhiteSource Report Finds NPM Vulnerabilities Fixed Fast

    WhiteSource Report Finds NPM Vulnerabilities Fixed Fast

    WhiteSource today published a report that found most of the vulnerabilities that affect node package managers (NPMs), widely employed to deploy JavaScript applications, are addressed long before they are assigned a Common Vulnerabilities and Exposure (CVE) in the National Vulnerability Database (NVD).

    The report, based on an analysis of the vulnerabilities that WhiteSource tracks in its own vulnerability database and revealed at the AWS re:Invent conference, found more than 90% of vulnerabilities in NPM packages are fixed before the vulnerability is published in the NVD.

    Susan St. Clair, director of product management for WhiteSource, said that fact suggests that, overall, the open source community is staying on top of vulnerabilities as they are disclosed rather than waiting for them to be assigned a CVE number.

    The WhiteSource report comes on the heels of a breach of UA-parser.js, a widely employed NPM based on a JavaScript library that detects browser, engine, OS, CPU and device type/model from User-Agent data. Unknown malicious actors found a way to inject malware into the NPM that modified the library. That vulnerability has since been remediated.

    In fact, the WhiteSource report noted 83% of the CVEs assigned to the most widely used versions of vulnerable NPMs could be remediated before the publication of the CVE.

    In the wake of a series of high-profile breaches of software supply chains, there is a lot more focus on how open source software is actually secured. Commercial software products have a distribution list that they typically notify of a security patch. Open source updates, on the other hand, need to be tracked by developers that use that software. The level of attention individual developers might pay to any number of open source updates can vary widely.

    Many developers are also hesitant to immediately update their applications because they are not sure what impact that additional code may have on their application, said St. Clair. For example, their application may have dependencies that are no longer backward compatible with the updated code, she noted.

    Developers also need to determine the degree to which that vulnerability may affect their applications because they often don’t use every component of an open source library, so the fix being offered may not be required.

    Open source project maintainers often depend on their community to help uncover vulnerabilities and create the fixes as quickly as possible. When a vulnerability is detected in an open source library, the more common practice is to report it to the NVD even after the fix for that issue has been made available.

    In general, St. Clair noted the most important thing for the open source community to remember is to remain proactive when it comes to tracking vulnerabilities and then implementing DevSecOps best practices. The one thing that is certain is that a wide range of malicious actors also are tracking those vulnerabilities closely. It’s not uncommon to see a wave of exploits being created once a vulnerability is disclosed via the NVD.

    The challenge, of course, is that the more open source software that is employed, the more difficult it becomes for individual developers to stay on top of every vulnerability in an application portfolio that only continues to grow. On the plus side, vendors have pledged more than $10 million to help better secure open source software. Hopefully, the return on that investment will become apparent sooner than later.

  • Logz.io Extends Open Source Observability Platform

    Logz.io Extends Open Source Observability Platform

    Logz.io has enhanced its observability platform to include both unified dashboards and support for distributed tracing to monitor service performance enabled by the open source Jaeger agent software being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    At the same time, Logz.io is making available a preview of an anomaly detection capability that leverages machine learning algorithms to build an artificial intelligence (AI) model to monitor data.

    The company has also added support for OpenSearch, a fork of the open source ElasticSearch software project that is no longer available via an Apache license. OpenSearch was created by Amazon Web Services (AWS) and Logz.io as an alternative search engine.

    Finally, Logz.io has added a security event management capability that makes it easier to classify, prioritize and collaborate on mitigation of cyberattacks as they are discovered.

    The Logz.io observability platform is entirely based on open source software. In addition, the company has made contributions to enhance both Jaeger and the OpenTelemetry project, including contributing an OTEL Span Metrics Processor that makes it easier to aggregate metrics.

    Logz.io CEO Tomer Levy said that, in general, it’s still early days as far as observability is concerned. As a concept, observability traces its lineage to linear dynamic systems. Observability at its most basic level measures how well internal states of a system can be inferred from knowledge of its external outputs. The overall goal is to make it easier to query machine data in a way that enables DevOps teams to proactively discover the root cause of issues before they cause further disruption. That’s a major advance over monitoring platforms that typically only provide predefined metrics to identify when a specific platform or application is performing within expectations. The issue, however, is most IT teams really don’t know what questions to ask to take full advantage of an observability platform, said Levy.

    However, as more organizations are exposed to observability platforms, it will become easier to advance both DevOps and DevSecOps as more platforms continue to leverage AI to provide remediation capabilities and automate processes based on the massive amounts of data collected via these platforms, said Levy. There is simply no way the average IT team will be able to manage all the microservices being strewn across a distributed computing environment without the aid of an AI model trained using observability data, he added.

    As DevOps continues to evolve, it’s more than likely that the best DevOps engineers are not going to want to work for organizations that have not invested in some form of observability. The odds of being consistently successful as a DevOps engineer decline sharply as IT environments become more complex.

    On the plus side, it should soon become a lot easier as the overall level of application instrumentation increases to manage those complex environments and to do so at a level of scale that doesn’t require a small army of DevOps engineers.

  • Nvidia/ARM Wavering | Google Outage Outrage | Backblaze IPO on Fire

    Nvidia/ARM Wavering | Google Outage Outrage | Backblaze IPO on Fire

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    Nvidia Can’t be Allowed to Buy Arm

    First up this week: Nvidia’s faltering attempt to buy Arm: Regulators around the world are alarmed by the idea. And now the UK—the legal home of the company—has taken the next step, which could nix the deal.

    Analysis: RISCy home for ARM

    This matters because ownership of the ARM ISA and the core designs would be a huge strategic stick to beat Nvidia’s competition with. Arm doesn’t make chips—it licenses technology to Nvidia and many of its competitors—AMD, Apple, Broadcom, Samsung and Qualcomm, to name but five. So this deal really shouldn’t go through, regardless of Nvidia’s fluffy PR reassurances.

    ARM chips aren’t just found in 99% of smartphones, but are an increasing fixture in the datacenter—especially where “performance per Watt” is important. They’re even found in exascale iron, including the Riken Fugaku, the world’s fastest supercomputer.

    Helen Thomas: Nvidia faces tough Arm wrestle

    It is one thing for a regulator to give your deal a thorough going-over. It is another for it to imply publicly that your assurances [aren’t] credible. … Nvidia got both from the UK’s Competition and Markets Authority. … The CMA’s full phase 1 report doesn’t make for easy reading.
    …
    The transaction, which was announced in September 2020, is bogged down in regulatory reviews around the world: … Arm’s “neutral” business model has always made it a difficult acquisition. … Arm licenses its designs to other chipmakers, providing scope to restrict rivals’ access.
    …
    Arm was a political talking point in 2016, when its sale to SoftBank … was waved through with embarrassing ease. There is now a much more muscular political take on UK national security concerns with a particular focus on tech.


    But if it’s owned by a Japanese company, why is the UK getting involved? Glad you asked—Aighearach answers:

    What did they buy? Shares of a UK company. … Then it is a UK company! … Buying shares is buying ownership, but it doesn’t move the company to another jurisdiction.
    …
    This blathering about “sold to a Japanese firm” is getting old.


    Nvidia claims, “The transaction will … boost competition and innovation.” Falmari finds that hilarious:

    Did Nvidia’s spokesperson previously work as a stand-up comedian? Because as jokes go, “boost competition” is a whopper. [Do you think] that’s what a company like Nvidia wants—to boost the competition?


    Google Cloud Outage has Ripple Effect

    Google’s load balancers all went offline for two hours on Tuesday. Aside from the knock-on Google Cloud failures, DevOps tools such as BigCommerce, Datadog, Deno, Discord, Egnyte, Evernote, Linear, Netlify and Toggl were reported to be affected. But perhaps the worst impact was felt by the poor, unfortunate users of Pokémon GO.

    Analysis: Paralysis

    In theory, putting all your eggs in one basket is okay—so long as that basket can reliably stop your eggs smashing on the pavement. But this isn’t the first time we’ve seen Google Cloud issues such as this; I can’t imagine it’s the last.

    Steven J. Vaughan-Nichols: Google glitch triggers major internet outage

    The result? Many of us have seen 404 errors when trying to get to some of our favorite web pages. … There appears to be more major sites having problems than not.
    …
    The specific problem seems to be with the GCP load balancers. These, working with Google’s Cloud Delivery Network (CDN) [are] designed to stop website failures and Distributed Denial of Service (DDoS) attacks by putting your website behind a single anycast IP.


    Are you feeling a touch of déjà vu? jeffbee, too:

    This is a sort of repetitive outage for Google. They’ve wiped out the GSLB configs before.

    A year ago there was also that big outage where they blanked out the whole Gmail delivery config and started rejecting all mail. … Config safety is not their strong suit.


    The #3 IaaS provider won’t get to #2 if it keeps this up. Here’s u/totopapa:

    I’m trying to convince my company to switch from AWS to GCP but an outage like this one is definitely a showstopper.


    Backblaze IPO Burns Brightly

    Backblaze is now NAS:BLZE. In its first week, the stock jumped 60%.

    Backblaze is well known for its tiny-margin consumer backup product and its open-source “storage pod” hardware—but is less well known for its DevOps-focused “B2” cloud storage service.

    Analysis: Can it Avoid Death-by-Packrats?

    Backblaze is trying to grow this newer DevOps business, without losing its classic consumer chops. But many of its former competitors have essentially dumped their “unlimited” offerings—it’s the old tragedy-of-the-commons problem.

    Jordan Novet: Backblaze [goes] public on small valuation and minimal funding

    In an era of blockbuster market debuts preceded by astronomical private financing rounds, Backblaze … a cloud service for backing up data … is a throwback to a time when sub-billion-dollar companies routinely went public to raise capital and build their profile … to win the trust of customers.
    …
    Backblaze shares climbed 24% in their debut on Thursday and another 12% on Friday, lifting the price to $22.31 and giving the company a market cap of around $650 million. … By the standards of its peers, Backblaze is a micro-cap.
    …
    One of Backblaze’s competitors is Amazon, the world’s fourth most valuable U.S. company. … Backblaze currently stores two exabytes of data — that’s 2 million terabytes — and has a half-million paying customers.


    Donovan Jones: Backblaze Targets $100 Million IPO

    The firm provides cloud-based storage services to small businesses and individuals worldwide. BLZE has grown revenue and gross profit impressively through the 2020 pandemic period and the IPO appears fairly valued, so is worth a close look.
    …
    The firm’s financials show strong revenue and gross profit growth. … The firm’s dollar-based net revenue retention rate for 2020 was 114%—a solid result. … The market opportunity for providing cloud-based storage services to small businesses and individuals is large and expected to grow at a material rate. … As for valuation, compared to partial competitor Dropbox, management is seeking the same Enterprise Value/Revenue multiple. [However] the combined revenue growth rate and EBITDA percentage rate … was 3% as of June 30, 2021, so the firm needs improvement in this regard.


    But there’s a long way to go, opines John Kozubik (aka rsync):

    Of the ~500,000 customers, 70,000 are B2 customers and 430,000 are “backup” customers—which are flat-rate “unlimited” users. … Roughly 85% of the user-base are paying $7/mo for “unlimited backup.”
    …
    These flat-rate users are very fickle and price-sensitive. … Roughly 85% of the user-base are paying $7/mo for “unlimited backup.” … It’s a fools errand to pursue this market and I assume the pivot to B2 was to build something more sustainable with a much higher price and usage ceiling.


    The Moral of the Story: Success is a journey, not a destination


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Kristin Wilson (via Unsplash)

  • What Developers Need to Know About Edge Computing

    What Developers Need to Know About Edge Computing

    More and more companies are living “on the edge” as they move from “I’ll think about edge computing” to “I don’t think our business can optimally grow and compete without edge computing.” Developers will play a key role in helping organizations use the edge, but their effectiveness will depend on having edge-aligned processes, products and people in place.

    Edge computing takes place near the physical location of either the user or a data source. It’s one way that organizations can use and distribute a common pool of resources across multiple locations. The edge model provides users with faster, more reliable services while expanding the potential of hybrid cloud computing for organizations. Customers and businesses alike will benefit from the way in which edge computing optimizes the use and application of artificial intelligence and machine learning by running data-intensive applications closer to the data source.

    While it is in many ways a question of architecture, applications are key to effectively exploiting the edge. Here are some of the things developers—and their organizations—need to know.

    Edge computing and cloud computing are not mutually exclusive. The best solutions will leverage both platforms. For example, centralized computing might be used for storage and compute-intensive workloads, while edge computing would be used to exploit data at the source, in near-real-time. Infrastructure and company culture need to be in place to support this, but developers must be thinking in terms of building applications that can cross and bridge computing platforms.

    Consistent tooling is essential. Developing for legacy IoT was very specific and required specialized tools and skills. The modern edge, in contrast, is part of a technology continuum, and developers need tools that they can use across that continuum, including:

    • Provider/enterprise core: The “non-edge” tiers, owned and operated by public cloud providers, telco service providers or large enterprises.
    • Service provider edge: Tiers are located between the core or regional data centers and the last mile access.
    • End user premises edge: Tiers on the end-user side of the last mile access can include the enterprise or the consumer edge.
    • Device edge: Standalone (non-clustered) systems that directly connect sensors/actuators via non-internet protocols. This represents the far edge of the network.

    Containerization enables flexibility, scale: Linux container technology wasn’t built for the edge, but the model fits perfectly with edge requirements for modularity, segregation and immutability. 

    Keep an “open” mind. Open APIs open up new opportunities. Developers using open APIs can access real-time data programmatically, making it possible to build applications that can leverage data as it’s “born.” Likewise, a focus on open source development tools and platforms will help to ensure that organizations can continue to leverage the edge as the model evolves. 

    What’s different is the devices. Developing for the edge is just like developing for any other platform–except that developers must write consistent and secure code that can be seamlessly copied to all device types. 

    Edge computing may not start with developers, but developers will play a huge role in organizations’ ability to leverage the edge. A focus on flexibility, openness and the future will help ensure that developers build applications that fully exploit and expand the edge advantage. 

  • Automated Refactoring with Moderne

    Automated Refactoring with Moderne

    Jonathan Schneider and Olga Kundzich are co-founders of Moderne, a Seattle-based startup. Moderne received $4.7 million in seed funding in July 2021 to commercialize OpenRewrite, an open source automated refactoring tool for code (initially Java) that Schneider started while at Netflix. Moderne’s technology reduces the tedium and time it takes for software remediation; a big problem in today’s world of cloud-native development. The video is below, followed by a transcript of the conversation.

    Announcer: This is Digital Anarchist.

    Alan Shimel: Hey, everyone. Welcome to another segment of TechStrong TV. We have a new company to detail for you, and we have two cofounders here to tell you all about it. It’s a great open source story as well. Let me introduce you to – and I’m gonna do my best not to mess it up – Olga Kundzich, Kundzich, Kundzich – three times the charm – and Jonathan Schneider. I think I got that one on the first try. Olga, Jonathan, welcome to TechStrong TV. 

    Olga Kundzich: Great to be here. 

    Jonathan Schneider: Thanks. It’s good to be here. 

    Alan Shimel: Thank you. So the only rules is only one of you can talk at a time, but both of you can talk as much as you’d like. But why don’t we start off with maybe a little bit about each of you. Olga, if you don’t mind, why don’t you go first and maybe share with the audience a little bit of your background and how you came to be here today. 

    Olga Kundzich: Great. So my background, I was working at Pivotal on a Spinnaker continuous delivery solution, and that’s where Jonathan and I had met. We worked together there for about a year and a half. Prior to that, I worked at EMC in engineering for, in different functions for a number of years. So when we were working together on Spinnaker, we talked to a lot of people in the industry. There was a lot of interest around that product, and kind of we realized that everyone had sort of the same problem. We would start a conversation about some old feature, and it will come back to, well, all my application is still stuck on this old, aging framework, and let me talk to you in a year when I migrate all of this back.

    So and the problems were highly repeatable and kind of same framework, same third-party vulnerabilities, same libraries, APIs, so after reflecting on this we realized that this is really not a coincidence. It’s a consequence of modern application development practices. Our software right now is 90 percent third-party dependencies, open source, APIs, so the problem’s kind of highly repeatable. And that’s kind of led, what led us to, back to technology that Jonathan developed earlier at his time with Netflix and kind of take it, develop a lot of more functionality into that sort of large-scale distributive factoring that will help us solve these type of problems, the framework migrations, API migrations, CVE patching that cross all of these open-source and third-party dependencies. 

    And I will let Jonathan speak about his background and a little bit on the open-source framework.

    Jonathan Schneider: Yeah, so Netflix had this engineering culture called freedom and responsibility, so product engineers were sort of, had the freedom to do whatever they wanted really. They could use their, a different build tool, they could use their particular Java style, they could use whatever libraries they wanted, and so forth. And so as a central team, if you’re on an engineering productivity team or you’re on a platform team and you’re trying to effect some change inside the organization, we couldn’t put up gates in front of product engineers that would, say, break their build if they didn’t comply with a certain thing. Instead, you tried to build tools that would help bring them along on a journey with you, and I kept hearing the same thing over and over again from engineers which is, hey, I got a lot on my plate. Happy to move forward to this new pattern that you want to see, if you do it for me. If you just do the work for me, I’m happy to commit it. 

    Alan Shimel: Nice of them. 

    Jonathan Schneider: And of course, you know, you think there’s two of us and 700 of them, and that doesn’t scale super well. Right? So that’s where we started working on a way of actually automating the source code transformation. When we developed that technology, one of the consequences of that freedom and responsibility is that there was not a consistent style of the code or consistent patterns, necessarily. So that refactorization we built had to make transformations that were, looked like they were a developer writing in that code base stylistically consistent, hundred percent accurate changes. And so we got some interesting results out of that, so when Olga and I were working together and you would hear, “Oh, I’m still stuck on Spring Boot 1 and I’m trying to get to Spring Boot 2,” or, “I’m stuck on JUnit 4. I’m trying to get to 5,” we just found another application of this at a larger level in a highly repeatable problem space. 

    Alan Shimel: Fair enough. You know what, I’ve spent the last 20 plus years talking to founders. I’ve founded my own companies. This is what happens. You see a real problem. Right? See a real-world problem. There’s got to be a better way we could solve it. Sometimes you find out solving that problem is not as easy as you thought. Sometimes it is, very rarely. But you work through it and that’s what startups are all about. In your case, though, I mean, look, Spinnaker, which is now part of CDF, right, was managed by CDF, but Google, Netflix were instrumental in the development of Spinnaker. But Spinnaker is an open source tool, and you guys went with an open source route in what you’ve developed. Why don’t we talk about that a little bit? 

    Jonathan Schneider: Sure, yeah. There’s really two parts to our product offering. There’s OpenRewrite, which is GitHub slash OpenRewrite. OpenRewrite is that core refactoring technology we’ve been talking about that can make stylistic automatic and consistent changes to a code base. On OpenRewrite we built recipes, so we have a recipe to migrate you from Spring Boot 1 to 2, from JUnit 4 to 5, to do code cleanup things, to do security vulnerability fixes. We want those recipes to always be Apache licensed and open source. Framework authors are going to develop the recipe, so the Spring team will develop the recipe that will move you from Spring Boot 2 to 3, or the Quarkus team at IBM or Red Hat would develop the recipe that’d move you from one version to another.

    What Moderne does commercially is we take those recipes and we run them at massive scale, so I could take that recipe and run them against tens or hundreds of millions of lines of code, and you can see that, the impact of that recipe running against your whole organization’s code base. 

    Alan Shimel: Fair. And so you mention – how did – it’s not Modern. It’s Moderne. Correct? 

    Jonathan Schneider: Yeah, we call it Moderne. Yeah.

    Alan Shimel: Moderne. So that’s the name of the company.

    Jonathan Schneider: It’s the name of the company, yeah. 

    Alan Shimel: And OpenRewrite is the open source aspect of it or –

    Jonathan Schneider: That’s right. 

    Alan Shimel: – owner of the solution. So let’s get into the mind of the entrepreneur here a little bit, Olga. Having worked at Pivotal and EMC for years before this, you saw this issue. You saw this problem. There’s got to be a solution. When did you say, you know what, this is something worthy of starting a company, right? When I was starting companies, one of the big questions is, is this a feature or a product? Right? Is it just a feature of someone else’s product, or is it truly a product in and of itself? And that makes a big difference in, obviously in what you’re building and what your aims and all of that. Olga, I’d like to hear from you. Feature, product, how did – what was kind of your thought process in saying, you know what, this could be its own company?

    Olga Kundzich: I think this, something like this is essential for managing modern cloud-native applications. These are becoming, the applications sort of become a glue code teaching all of these third-party components. They evolve and change at their own pace, and developers really right now are just restitching those things back together, upgrading CVs, upgrading versions, and it takes a really significant amount of time just to – sort of anecdotally we hear from people that 20 to 30 percent of their time, engineering time, is just now dedicated to these tedious migrations. So if we can automate this, we can unlock innovation, productivity of engineering organizations, and help a lot of businesses actually innovate.

    Alan Shimel: Got it. Jonathan, what about from your end? Well, let me ask you both. Is this your first time starting a company? 

    Jonathan Schneider: It’s first time starting a company, yeah. I think it’s real special to me. I think one thing I love about this business is that we’re really after helping organizations fix their code, and our outcome and our future success is tied directly to how much code we’re able to fix, how much help we’re able to provide. We’re not about reporting and saying like, you know, here’s the problems that you, engineering team, need to fix. When we partner with an organization, we’re kind of in the trenches with them getting these issues off their plate.

    We talked to one organization recently – and this is why I think it’s a product and not a feature. We talked to one organization recently and they say we just look at our top ten issues reported by a series of static analysis tools they have in-house, and it says, well, it’s gonna, these tools estimate it’ll take 1,700 developer days to fix these top ten issues. So that’s like 4.8 years’ worth of developer time to fix those top ten issues that they have, and those aren’t false positives. Those are just literally top ten critical problems they have right now. So we developed recipes for a lot of those, and their engineers said, you know, “We estimate it would take us five days with these recipes to apply that change and be done with it.”

    When I think about 1,700 developer days or 4.8 years of time, what that really means to me is infinity, or like it means that the engineering team is never going to get through that backlog of issues. Five days seems like it is possible, and that’s, there’s this line we have, the, basically software grows until it can no longer be maintained, and I think our business is in keeping that software in a maintainable state so that the business can continue to innovate and thrive. 

    Alan Shimel: Absolutely. So companies launch now – we didn’t even mention the website. Shame on –

    Jonathan Schneider: Yeah.

    Alan Shimel: What’s the website? 

    Jonathan Schneider: Yeah, so the website is moderne.io, so it’s modern with an E.

    Alan Shimel: Can you spell that, Jonathan? M-O- –

    Jonathan Schneider: Yeah, it’s M-O-D-E-R-N-E, so modern with an E, dot I-O.

    Alan Shimel: Dot I-O.

    Jonathan Schneider: Yep, that’s right. We do have a restricted beta going on right now that you can just sign up for that on that website, and we’ll do an invite to you and you can see how we can apply these recipes across tens of millions of lines of code. 

    Alan Shimel: Absolutely. Are you working with CDF and the Spinnaker community?

    Jonathan Schneider: We’re really in an adjacent product space now. I think there’s a lot of value to operational automation, what Spinnaker is doing, what different APM vendors are doing. I think we were working in that and we saw just an adjacent pain point that we just decided to go after instead. 

    Alan Shimel: Good stuff. Well, guys, I want to wish you nothing but a lot of success with this. It’s always, you know, I gotta be honest with you. I always enjoy interviewing new companies coming out of stealth and beta, first-time entrepreneurs, ’cause you make the world go around. And good luck. Keep us posted. Come back on and tell us more as you, as the story continues to unfold here. 

    Jonathan Schneider: Yeah, look forward to talking again soon. 

    Alan Shimel: All right. 

    Olga Kundzich: Thank you.

    Alan Shimel: Jonathan and Olga from Moderne, M-O-D-E-R-N-E, dot I-O, here on TechStrong TV. We’re gonna take a break. We’ll be right back.

    [End of Audio] 

  • Open Source is the Engine for Observability at Scale – Observability at Scale TV EP 2

    Open Source is the Engine for Observability at Scale – Observability at Scale TV EP 2

    As today’s organizations seek advanced monitoring and observability to ensure the performance of their microservices and cloud native systems, issues of cost, complexity and tool sprawl have emerged as pervasive challenges. Join our hosts Alan Shimel and Mitch Ashley along with panelists Dotan Horovits (Logz) and Constance Caramanolis (Splunk) as they dive into the challenges that DevOps practitioners are frequently forced to compromise on and how they transform modern observability. The video and a transcript of the conversation are below. 

    Alan Shimel: Hey everyone, welcome to Observability At Scale, one of the latest video series or video show series, over here on TechStrong TV part of TechStrong Group. I’m your host, Alan Shimel. And let me just say that Observability At Scale is sponsored by our good friends at Logz.io, it’s L-O-G-Z and many thanks to them for sponsoring this. This is a great subject.  

    I recently in KubeCon about two weeks ago. And I will tell you, it was pretty much all Observability all the time. I did 40 interviews there in two days. And I wouldn’t be surprised if somewhere between a third and 40% of the people I interviewed were about Observability. And many of those being at KubeCon CNCF Cloud Native Con, many of them were about open-source Observability. 

    Whether we’re talking about Prometheus or OpenTelemetry or several of the other Observability-related open-source projects out there. That seems to be the subject of today’s episode: Observability and open source. And I think we’ve got a great panel to discuss this. Let me introduce you to them. First of all, we have Dotan Horovits. Dotan is with Logz. Actually Dotan, you introduce yourself. I don’t want to put words in your mouth. Go ahead.

    Dotan Horovits: Gladly. Glad to be here, Alan. And my name is Dotan Horovits, and I’m a developer advocate at Logz.io. Logz is a cloud-native platform for observability. And the nice thing that I love about Logz.io is that it’s all based on popular open-source tools that we’re going to talk about extensively today like the ELK Stack and Prometheus and OpenTelemetry and Jaeger and so on. So that’s about Logz and myself.

    Alan Shimel: Thank you, Dotan. Next up, we have Constance Caramanolis, and I hope I did that justice. Hey, Constance, how are you? Maybe a little bit about yourself for our audience?

    Constance C.: Good morning. I’m good. Hi, everyone. I’m Constance. I’m a principal software engineer at Splunk. Right now I’m on OpenTelemetry Governance Committee for I guess another five days until elections are over. But I’ve been involved in open source for about four or five years now, both on Envoy OpenTelemetry and on the product with Con community.

    Alan Shimel: Excellent. Thank you, Constance, for coming on today. We appreciate you having you here. And then last but not least, is my co-host of Observability At Scale. He’s also CTO here at TechStrong Group and a principal analyst at TechStrong Research. It’s Mitchell Ashley. Hey, Mitchell, how are you?

    Mitchell Ashley: Good, good, good. Great to be here and very thankful to Dotan and Constance for joining us. I look forward to a great conversation.

    Alan Shimel: Yep. Thank you. So guys, as I started the show off I was out in Los Angeles, I guess it was two weeks ago now for KubeCon Cloud Native Con. And first of all Linux Foundation did an amazing job creating as safe an environment as possible for us to do an in-person event. And it was great to be in-person they got. I don’t want to belabor it but this stuff is good on Zoom, it is better in-person. 

    One day maybe we’ll run this as a live panel from a KubeCon, and that would be just great. But it was all Observability all the time, it seemed. Several things with Observability. Number one, the hairs on the back of my neck stand up when I go to a major show. And there’s something that’s so hot, so hot, that I just feel I can never live up to its promise. And that is what I was afraid with you all. 

    Are we in a overhype cycle around Observability. And then secondly, so much of Observability is based on open-source underpinnings, if you will. Right. There’s a lot of great open-source programs. Maybe that’ll save us from the usual hype cycle that we get here, I don’t know. Anyway, I’m interested in your comments on that, is Observability being overhyped at this point?

    Dotan Horovits: If I may take the first pass on this one. I think that as many trends, and we’ve seen the quite a few of those thought out the years, you have so many vendors trying to piggyback on the trend and leverage that for the relevant offerings that it creates a bit of a blur and confusion about what exactly Observability is about.

    Is it about tooling or methodologies? Is login in its own right for vendors that just provide, let’s say, login of vendors just provide metrics is that Observability? And can you rebrand for Observability yes or no? You have two of these, is that enough? Is signals the whole story about Observability at all? Or is the discussion about Logz metrics traces do justice to Observability?

    I think these are the sorts of conflict and confusion that I see and sense in my conversations with the user community, with customers of ours, with partners of ours. And, again, lucky enough in my position as an evangelist, I use my time to try and clarify and try to separate between the tooling and the methodologies and the essence of things. But definitely, a lot of confusion around that.

    Alan Shimel: A lot of confusion. I did an interview or a panel discussion yesterday with some folks. And one of the things, there was a good line that came out of there was actually around the definition of DevOps, but it’s equally, I think, applicable here, which is, what is Observability? And all of these kinds of things you mentioned, Dotan is, where is the value? 

    Show me where the value is in and I’ll show you what it is, who’s using it, who’s doing it, who’s not. Where is the value? Constance, one might say you were into Observability before it was cool, before it was hyped.

    Constance C.: True. Yeah, right, because that was one of the selling points of Envoy, was making things transparent to the networking to everyone around. And one of the features that made people really love the project was that we would joke, there’s a staff for that. You would send a request, there would be staff for the connection, and everything related to it, and so that made things really easy to debug as a consumer of it. 

    Dotan, you’re definitely right in terms of so much confusion because observably can almost be like the block-chain. And no offense of block-chain, but that’s like the key word that gets thrown around in terms of getting really hip and cool. I think the nice thing about this attention is that it is forcing a lot of people to start thinking more about why they’re doing things.

    There’s a lot of just doing things because you’re either told to or your thought about and now sort of be like, hey, if you’re going to do something, we’ve always asked how do we know at work, but it’s like, oh, let me just write a test. Now it’s a little bit more like how do I systematically know it works by either adding instrumentation so I can consistently monitor it or observe it. 

    I think that is a really good thing that we need to start moving towards, because it’s a little bit more of a, we still check something in, throw it over our shoulders and be like, whoa, good day, everyone. And we’re starting to think a little bit more that sustainability. 

    And I also think there’s a second part too, especially when it comes to these projects that were mentioned earlier, we’re not just thinking about tools anymore, we’re sort of seeing them as products and starting to actually say why this could matter to you, why you should actually adopt them. 

    And that’s something that as engineers, it’s not in our normal toolset in how we think about things, we’re starting to get better at, because we need to start. We’re also selling things for open source, the currencies, commits and contributions and not necessarily money.

    Alan Shimel: I think that’s super important. To the question, is it overhyped? Will it be overhyped? Yes, yes. And whatever the next question is, yes. But I think what Observability has going forward is two big things. One is it started from open sources, it started from a community. Second was trying to solve a real problem around as we move to micro-services’ architecture containers.

    How do you manage this stuff? How do you understand what’s happening? I remember Constance, you’re on a podcast we did together, we were talking about how do you know what happened where, what’s doing what, and what do these things do, to use generic terms, but what is it? So we’ve created this flexible, malleable component containerized architecture. 

    Now how do we understand it? We operate it and improve it over time. I guess the other thing we were talking to, you said software engineers, developers, operations people, they’re not into marketing hype. They’re pretty real folks. They want to know, are you solving a real problem for me or is it a checkbox and you’re calling it monitoring observability?

    So we have a pretty healthy skepticism in the market too that will fair it out, I think in most cases, what’s real, what’s not. Where the value is, to your point, Dotan. Okay, go on.

    Dotan Horovits: If I may just add on that, one of the benefits of the open source tools that we’ll talk about and standards, by the way, it’s not just tools, it’s also standards, such as OpenMetrics and others, is that it’s usually driven by real needs of real companies. Actually, today, I had an episode with a gentleman, senior SRE, site reliability engineer, from Google in charge of the entire identity services package of Google. 

    So when you log into your Google account, it goes through their micro-services. And many of the tooling that they use, and they’ve been using for a couple of decades, is the predecessor of what we currently talk about at the CNCF and many others. So obviously, Kubernetes itself, the containerization is a successor of Borg, their internal containerization orchestration system. 

    And then you have Google sensors that evolved into open sensors that now is emerged into OpenTelemetry together with OpenTracing. And you have the Dapper project that very, very fascinating research paper about distributed tracing that was, I guess, the forefather of Zipkin open source and the Jaeger open source and many others in the industry. 

    The nice thing is that Google’s scale is such that where we are at today was where they used to be a couple of decades ago. So in a way, these are well rooted within realistic problems at scale in micro-service architecture, in containerized environment, cloud native environments and technology. 

    So I feel much more confident basing Observability standards, tools and practices on the open source rather than on some vendor initiatives that again, might be inspired by some strategy and might have some forecasting behind it. But much less solid ground around Observability practices hands on.

    Constance C.: Two things I want to add to that, one is like, I think, I guess this is one of the things I want to keep on advocating for, is that I think we should get more just for nitpicking by cognitive doesn’t actually mean containerization. I think it’s just —

    Alan Shimel: And it doesn’t mean Kubernetes necessarily either.

    Constance C.: Exactly. Yeah, that was actually one of the points I was trying to make is cognitive is like any of the things there. But it’s more just I want us to dump that because a lot of people when they think about, oh, let me modernize my stack, I’m going to go straight to containerization. But that might not be the problem that they actually dissolve. 

    And so that’s just one thing I want everyone to be aware of when people are talking about cloud native is like, what is the problem I need to solve when you’re upgrading your stack, remodeling your house, thinking about that problem there? And I guess, I don’t know, Alan, if you want to talk to us now, but the vendor influence. The vendor influence is really interesting.

    Alan Shimel: I was going to bring that up. Let me jump into that. So let me just also lead off with this stuff. Look, Gartner has been doing a hype cycle for as long as I’ve been in technology, 25, 30 years. And the hype cycle is real. That is the way our industry works. Things get really, really overly high, they do the trial, disillusionment, and then they become productive, or whatever that end stage is. 

    What’s a little different about Observability is it’s in this new foundational era of open source, where you have, as I said, several really good open source projects that are driving so much of the Observability innovation out there. None of those projects are controlled by a single vendor. 

    They’re owned and managed by Linux Foundation and maybe some other foundations, which creates this whole golden era of coopetition, to your point Constance, among all the vendors. And this is a very interesting thing. So everybody starts with that same base. When Mitchell and I were doing vulnerability scanners 20 years ago, you could use the open source that was Nessus.

    Nessus was still open source then. You could use the Nessus scanning engine but there were proprietary scanning engines that people were using, some of them really based on Nessus some not. Not the case here. Here, you got a company like Logz, and Dotan, I don’t mean this in a bad way. But you got a company like Logz, for instance, almost their entire value prop is built upon open source. 

    Built upon that open source. I spoke to the Splunk people, their observability stuff, built upon open source. I’m just made mentioning that because we’ve got the two of you here, but there’s a lot of observability vendors out there. And they’re all built upon the same open source foundational tools. 

    I don’t know if we’ve had this kind of coopetition in a market like this in years past and in times past where, look, everybody starts at that same level and what you do with it. Who makes the best observability solution? I don’t know, I guess it depends on really what your needs are. 

    But it’ll be very interesting to see how in the real world laboratory of market conditions, people all come from the same open-source base, and what the value add is on top of that.

    Dotan Horovits: I would challenge two points there. First of all, we started the discussion from the CNCF and from the Linux Foundation, and we can add on top of that, the Apache foundation and others. But let’s bear in mind that many popular tools out there still are controlled by vendors. We can talk about the ELK Stack, ElasticSearch —.

    We can talk about the Grafana that is highly popular, and it’s controlled by Grafana Lab that actually rebranded itself to Grafana to be synonymous with the open source. So there is another beast out there, open sources that are very popular and very well adopted and very mature, but then again, not controlled by a foundation and this various implications. 

    I guess the most prominent implications are when these open sources are then being relicensed to some degree. So users one day wake up to find that the open source on which they built the entire system around is suddenly being pulled of the open source [crosstalk].

    Alan Shimel: One day we woke up and Nusses was an open source anymore, it wasn’t like that. But that’s exactly the point that I was making with foundational open source and why CNCF and these kinds of foundations —, Apache, whatever, it’s so great. And I’m not picking on Grafana, I happen to like Grafana, I like Loki as well, which is another Grafana project. 

    But the very fact is, if you have one company that’s controlling an open-source project, other companies are very often hesitant to contribute. They may use it for now, but they’re hesitant to contribute. And so in the long run, and I’ve been in open source, Mitchell will tell you, we’ve been in open source 25 plus years. 

    Over the long run the foundational model where you can have coopetition among various vendors prove superior to what I call the big brother open-source model where you have one vendor who owns that, and can pull it at any time or calls the shot. If I’m a VC-funded company, a unicorn or a public entity, and I’m investing 10s of millions, hundreds of millions of dollars, I want to be very careful that I am not dependent on an open-source project that is controlled by a single company. 

    This may be getting off observability and I’m sorry for getting on my soapbox, but I think this goes to the reason why CNCF and that community is so vibrant. It’s so much vitality, why the projects coming out of there are so popular and really changing our game. I look at observability because companies feel like everyone gets dealt the same hand, what you do with it is up to you. 

    No one’s going to pull the rug out from under you. And Constance, you’re on these boards so I’m sure you probably feel that way, but I’m interested in what you all think?

    Constance C.: The vendor has such a dirty word in like CNCF like KubeCon, Cloud Native Con, and open source. Okay, I’m putting on my vendor hat. Well, I’m actually putting on both hats, the vendor hat and also the contributor hat. One thing that the vendors have is that there’s a little bit of superpower. 

    And maybe we don’t as vendors or as leverages, is that we actually get to see a broad spectrum of how things are used. And if we’re doing a good job in terms of like, hey, we’ve seen these things like 10 times over and over so that we think this is really important and bring this to the community that could really help the community. 

    I think sometimes the messaging gets lost, because we’re just like, hey, we need this thing here and not explaining why about there. I think that’s sometimes why vendors get a bad name. It’s such a dirty word within open source, because you feel like, oh, you’re controlling how we want to use things, you’re going to make us buy these things there. 

    If you look at our stats, I can’t remember what the stats are on top of my head, a lot of our talks come from vendor companies because a lot of things we use, they’ve contributed so much time into it. I’m only saying this because I’m trying to make it less of a forbidden word. Because they do have valuable insights, and they do contribute a lot. 

    Then there’s also the end user part who has a lot of different use cases that vendors wouldn’t necessarily see. I also think for like some of the examples you mentioned, it might just be a matter of time before they move over to more CNCF. Because a lot of like, OpenTelemetry is still new, two-ish years. I guess, everything within CNCF are about six, seven years, so really, really new. 

    I think it’s just more like continuation of time, as CNCF is, as you can still see, it’s so new that it’s like iron itself out. It’s starting to be, you could say, the de facto place where a lot of people look to, to donate their projects and be a part of that community. 

    KubeCon and Con keeps getting crazy, crazy numbers of attendees and participation, it’s going to become more and more projects start coming to it, because there’s more of that trust, and it’s a little bit more of a matter of time. I like your, what was it?

    Alan Shimel: Coopetition.

    Constance C.: Coopetition. I’m starting to see it less and less. My scope is OpenTelemetry, I will say I wanted OpenTelemetry be like, we’re all vendors, you’re little skeptical. And as I’m leaving the GC now is like, oh my goodness, just seeing so many different people’s, so many different companies’ perspectives on what customers are targeting, what problem we’re trying to solve.

    And start to view these people at first, I’m like, who are you? It’ll now be like, setting up one on one fairly regularly. Talk about careers and life. We go to open source to collaborate with others who are working on projects that are similar to us or probably they’re similar to us. And this was like another avenue to see the more people and get to know them.

    Dotan Horovits: It’s important to say I work for a vendor — that also contribute to OpenTelemetry and to Jaeger and to others. So definitely I agree with you entirely that the know-how within these companies and the broad perspective of different use cases across different verticals, industries, geographies, gives a very valuable perspective.

    It just requires the right mindset of the vendor, because even if you look at the evolution, specifically around observability, players such as Datadog, New Relic and even Splunk started off with a proprietary agent that did very intelligent things. 

    I agree entirely that the ability with a proprietary agent to fetch the right telemetry, to enrich it with the right metadata to do the right aggregations, to fine tune it to the backend tool, storage and analytics and visualization, that’s perfect. But then again, it is very vendor locked. You can’t correlate across different vendors across different signals, assuming that not the same vendor serves all signals. 

    And that’s what the driver that brought us to OpenTelemetry. And gladly is to say, and by the way, today OpenTelemetry is the second most active project under the CNCF according to the DevStats, CNCF DevStat. It’s really amazing to see. That essentially brought the vendor to their realization that the locking through agents is no longer the future path. 

    And they’re aligning with OpenTelemetry and even contributing know how around the agents back to OpenTelemetry. And now the differentiating factor is within the collector, doing intelligent processing, aggregation and the power of visualization based on that. And that’s the right play, and that’s the play that evens the playing field because everyone gets the same raw telemetry and the raw signals. Now, it is up to you to draw the most value out of that and bring it to the end.

    Alan Shimel: That’s exactly what I was trying to say.

    Mitchell Ashley: With your point, yeah.

    Alan Shimel: That was exactly. I’ll take it one step further. It’s not just the CNCF. We do another video show on here with the open mainframe project of all thanks. Another Linux Foundation project. And it’s really an umbrella for 26 different open mainframe-related projects. It’s the same thing. You got Broadcom and IBM and Rocket and BMC, and many others all working together to be able to do like DevOps on mainframes, project Zoe and stuff.

    The CDF, Continuous Delivery Foundation, same thing. You got all the CD vendors in there.  Yes, there are the Grafanas there. Are the big brother open source players left there? This is the foundational era of all, and mean no offense, why like foundation. 

    But this is the foundational era of open source. You just said, Dotan, that’s it. That’s it. You can’t beat that. And that is why the Splunk and the New Relics and the rest of the world have come around to recognize that as well.

    Mitchell Ashley: Maybe [crosstalk] Alan. May be just kind of fanboy for a minute. What’s really awesome about this conversation is that we’re coming to it from multiple points. It isn’t just about let’s start an open source project, and that grows up into its evolution and becomes popular, etc. That’s one point of entry. There’s other points of entry, the proprietary. 

    We started here, now this has sprung up and we need to, maybe in some cases, companies are forced to. You have to, to be competitive, because something like OpenTelemetry has really become so foundational, to use that word, Alan, that that’s critical. You have to be part of that community, and maybe even have to contribute to be able to do that. I’m referring into a genuine vendor here. 

    And of course, tomorrow, someone else will start. And they’re going to start from where we are today. So what’s cool about this open source, it isn’t one thing, it isn’t one way of starting and one place of starting. You may be, like you were contributing to some early projects when you were at Lyft Constance. 

    Or you may be starting the next company, Dotan, that you’re going to build on OpenTelemetry and maybe one or some other projects, you’re going to join Jaeger, whatever, starts something new. But my point is, it’s not one thread through it, it is multiple threads. 

    But having an underlying activity that is bigger than any one of us, that we are contributing to, that it sort of advances the scrum as we move down the field, so we’re heading in a direction that sort of the rising tide lifts all boats kinds of things, yes, we’re all going to compete. And some people are going to only be interested in sort of their pure open source work. 

    And that’s what I’m going to do. And that’s how kind of a developer I’m going to be or what I’m going to contribute to. Others are going to say, how do I productize? How do I commercialize? How do I build something on top of that? Because I have some great ideas too, to develop it. And that’s what’s cool, how far open source is come from, when Alan and I tried to build some products on earlier open source stuff and to where we are today.

    Dotan Horovits: And that’s the beauty. The beauty is that, many talk about open source in the context of community in the greater good and everything, but then it’s important to say it is also a viable business model. It’s [crosstalk] one that you can build a viable business model, unlike some that claim that they cannot sustain it going forward. 

    And use that as an excuse maybe to take it off the open source or things like that. Actually, just look now, last week, or this week that chromosphere became a unicorn raising 200 million with a one point something billion dollar valuation. And that’s all based on M3DB or M3 that spun out of an internal Uber project, Uber engineering that developed its own time series database, scalable Prometheus, if you’d like, database in terms of time series storage.

    And now it’s a company in its own right with its own open source, in addition to a commercial offering. Now they’re expanding from metrics to also distributed tracing, as far as I understand. So it is happening. It is accelerating, actually, innovation, including business innovation, including new startups. It has sprung up around observability. 

    And it’s a very exciting thing to see that as the open source by polar signals that there is now around a continuous profiling in observability. You see that from all angles, and you see startup and innovation. And I think it’s fueled by and facilitated by the open source. And that’s something that they think is great.

    Alan Shimel: Absolutely. So here for me was the key sort of switch that switched, that enabled this. There was a time when we used to talk about the only open source business model that ever succeeded was Red Hat. That was a common thing you used to hear. And why? Because you look at some of the early open source commercial models, and they were about what we offer support and training. 

    And that was like, how do I monetize open source? You do support and trainings really. Well, that’s not a great model. That’s been proven, not a very lucrative model. Then we started doing some open core, premium modules. So I’m going to give you bare bones and if you want the good stuff, you pay me for it. And that was the model. 

    And look, there’s still companies doing that sort of model, it’s viable. Some people have problems with it, is it really open source? I think it is, but another story. But today, what we have is, Dotan, what you’re talking about. And I think, frankly, SAS has been the great enabler of this. Because now, I can take this open source project, set it up as a SAS app that’s going to do it for you. 

    You know what? People understand that these things cost money to develop, maintain, and deliver. And if I’m getting it as a SAS app like that, and I’m paying monthly or subscription, whatever your subscription is, that’s the way business is done. That’s the way software is done today, whether it’s observability, or anything else in this cloud native world. That has been the enabler.

    Yeah, it’s open source, but I’m getting it as a SAS app, this is what it cost, and I’m good with that. I don’t feel like I’m getting something that I should be getting for free. Ron Gula, who started Tenable Network skewed. I don’t know if you guys know Ron. But Ron always used to say, look, with open source that’s free as in freedom and free as in beer. 

    We’ve concentrated on the free isn’t freedom with observability. It’s not necessarily free as in beer. And I think that’s a key mind view that we need that’s enabled, quite frankly, Logz success. Dotan, right? People say yeah, maybe built on open source, but I’m getting real value out of it. And they’re packaging it all up for me and building on top of that. I’m willing to pay for that.

    Dotan Horovits: I would like to even reiterate what Constance said before because when you look at the vendors, the vendors actually bring value here. The vendors, say, okay, I will make the open source accessible to you. But then again, open source projects, and Constance probably can say a lot about that from Envoy and others, very focused on the core functionality, the core value proposition.

    But for a much more mature organization, it needs an entire wrapping around it that provides DSSO and the authentication, the certifications and all the administrative aspects. The open source developers are not even bothered with that. Even a more powerful UI and an accessibility and user management control agents who are back whatever. 

    And this is the wrapping that vendor with again, vendor that serves enterprises and knows all these needs can augment the open source with these capabilities. I think Splunk is doing it magnificently with enterprises and Logz.io and many others.

    But the idea is that okay, you have the open source that you know and love but the open source beyond having it in a managed version also needs some additional layers to get it to be a commercial grade or enterprise grade. So it can actually be acquired that whatever you run is certified, meets the criteria that you need to run it within your organization at scale. 

    Constance C.: So I guess I’m going to say I’m very disappointed all three of you. I guess you didn’t watch my keynote.

    Alan Shimel: No, I didn’t. I apologize. But tell us.

    Constance C.: Okay, so I don’t necessarily know if it should be offered. Some things shouldn’t be offered as like the vendor implemented for you. But the main thing, one of the points I was trying to make that CNCF is more than Kubernetes. That’s why I was calling out the Cloud Native. We don’t think the — hold it. But the next thing for CNCF you have to keep.

    Actually let’s use it like the hype train. CNCF is on this hype train. And it’s great, because it’s celebrating all this hard work. I’m going to say as a community, we actually deserve quite a bit of attention, because we’ve done a lot of great work. But for us to keep on having celebrations like KubeCon Cloud Native Con, and community based, and just having these fun discussions, we need to make sure that we think of CNCF as a whole product. 

    And each project needs to think as a product. And we actually need to start thinking about integrations. Because as CNCF as a whole, we have great projects here. But there’s no glue between them. And so once you drop one project, sometimes you get some other project for free. But if you’re trying to integrate multiple things at once, you get a hard time.

    You don’t get any freebies once you implement, say Kubernetes and you’re trying to implement Envoy, both things there. No overlap, and so we need to start thinking about from a more together. I don’t honestly know that should be offered to SAS, I actually think that – I know, there’s a bit of debate about should there be CNCF Stack. 

    But I think that we should probably actually have something easily set up 10 projects that were really great together. All look great. And that can help things there. The thing that I really like about where observability space maybe for the value add as vendors, is like the key distinction. For OpenTelemetry, it’s about instrumentation data collection. That’s where our boundaries are specifically set. 

    We will never do anything about visualization. We’re trying to make you gather your data and then after, you can choose how you want to interact with that. There’s a little bit of that boundary that I think are being set around projects of where things go, because you can either try and almost be just kind of gobbling everything up, everything around you. Or you set your boundaries and then you have to let something else track there.

    Alan Shimel: Sorry, I think that’s a CNCF decision. They have to make, like for instance, how many service match projects do you are under CNCF.

    Constance C.: Well, so that’s a no kingmakers rule. Like we do have an explicit rule about there being no kingmakers, because we do want to offer things right. Because I’m biased that I love Envoy, but for some people Linkerd works great. And that’s amazing because there’s all these things there. Some people really love Prometheus, I think open source is great. 

    Metrics is not necessarily equal there. There should be different option because things work better for other there. But it doesn’t mean that we can’t maybe say have like, I don’t know, a few CNCF Stacks. Like these 10 things work great here, these 10 things work there. And this is actually going to be more on the engineers, because CNCF does have more of the model of the open governance so you choose your own adventure. 

    But then you see a little more of, it’s this great thing about open source, you choose your own adventure but we also need to think a little bit more like PMS, because marketers and salespeople are thinking about what the future will be. Because if we just implement the next thing, like go faster build all those things there, you’re not going to have sustainability to project.

    Because I think the thing that we’re taught, everyone’s like, okay. This is almost sold as this is the last big project you’ll ever adopt. 

    Alan Shimel: Agree.

    Constance C.: But we’re not. And if we don’t think about the future, sorry, everyone, you’re going to have another rewrite in 5 to 10 years.

    Alan Shimel: Guys, we’re about out of time, I got to ring the bell here as I said in the beginning of our talk. But you know what, look. I didn’t take the prerogative of taking the last word because I’m the host. We are in a golden age of open source innovation right now. And we have been for the last couple years. And I think CNCF in similar kind of foundations have ushered this golden aged. 

    And there’s a lot of companies including Logz and Splunk, and many, many others who are constantly innovating and pushing what we can do off of this foundation, and I mean foundation of open source. And I don’t know how long it’ll last until we go into a post-cloud native world, I don’t know. I may not be here, but I do think when we look back on this, it is a golden age.

    And it’s going to be very interesting to see what the next short-term, because who knows long-term, what the next short-term future holds and especially for observability. Observability is kind of the poster child right now for what we can do. That’s my poster post right there. Anyway, on that note, I think we’re going to close this one out. 

    Constance, thank you so, so, so much for joining with us today, this was great. I apologize, I had that little network outage there and I don’t have you guys up on my usual monitor. So I keep looking over here to my computer. But thank you for joining us. Dotan, as always, a pleasure to have you on my friend. I look forward to being able to see you soon. Yep. And then of course, Mitchell. You want to take the last word, Mitchell, and we’ll wrap it up.

    Mitchell Ashley: I’m going to leave it right where you said it. You just said it perfectly. Let’s end there.

    Alan Shimel: Okay, then. Hey, we’ll be back in just a couple of weeks, two weeks I think, with our next Observability At Scale. It’s a great series if you’re interested in observability. Check it out. But for now, this is Alan Shimel, TechStrong Group. Have a great day.

    [End of Audio]

     

  • DevSecOps Tech Radar Highlights Diverse Tooling Adoption

    DevSecOps Tech Radar Highlights Diverse Tooling Adoption

    The DevSecOps Technology Radar showcases the opinions cloud-native groups have about DevSecOps tools. To review, The Technology Radar is a periodic report from the Cloud Native Computing Foundation (CNCF), a burgeoning host for new open source projects that are becoming pivotal underpinnings to cloud-native architecture at large.

    Just like previous radars, the DevSecOps Technology Radar was assembled using feedback from the CNCF end-user community, which is a group of over 155 organizations that use cloud-native technologies to build and distribute their services, spanning startups and enterprises. The Radar report organizes popular tools into adopt, trial and/or assess status, based on a technology’s active use and its maturity.

    ‘Adopt’ means these tools are stable and come highly recommended for use in production systems.

    ‘Trial’ means users have had some success with these tools, and it is recommended that others consider them.

    ‘Assess’ means these are promising tools that are good for solving specific needs.

    So, why was DevSecOps selected as this Radar’s theme? According to the Radar team, DevSecOps was chosen because it is highly relevant to current development focuses. “We realized that more than half of the tools we were using in CI/CD were actually security-oriented,” said Sergiu Petean, head of DevOps at Allianz Direct. This goes beyond DevOps—”this is definitely an uppercase ‘Sec’,” he said.

    “More than half of our time is spent discussing designs around security,” said Keith Nielsen, director, cloud architecture at Discover Financial Services. Financial services especially must find innovative methods to balance speed-to-market with security. Nielsen also cited the rapid rate of change in this space, which requires companies to continually reevaluate their security posture.

    DevSecOps Radar Results

    So, where did the CNCF end-user community place popular DevSecOps tools? The Radar ended up rating 16 tools across the three different levels:

    • Adopt: Istio, Sonarqube, Artifactory, Hashicorp Vault, Calico/Tigera, Terraform, ArgoCD and OPA.
    • Trial: XRay.
    • Assess: Cilium, Harness, SonaType Nexus, Hashicorp Sentinel, GitHub Actions, Linkerd and Trivy.

    The above results were based on 252 votes from end users across 35 tools, with most respondents coming from large companies across multiple sectors.

    Out of these tools, Terraform collected the highest number of ‘Adopt’ votes. This technology aids the move toward infrastructure-as-code (IaC) with declarative configuration files, which is important for DevSecOps automation. Tied for second place were Hashicorp Vault and Artifactory, indicating that secrets management, access control and continuous updates for security are pressing requirements.

    Other open source tools on the Radar are providing impressive ways to unify security controls. For example, OPA is a universal layer for applying policies that could help unite fragmented authorization across applications. Or, service mesh options like Linkerd and Istio place a sidecar proxy around applications to centrally manage access control, observability and networking across a large microservices ecosystem. Such projects are programmable to fit into the DevSecOps pipeline.

    Analysis

    In general, the radar team was somewhat surprised to see such a high number of security tools in use today. Some tools compete in the same arena, making hunting for the right tool challenging. “The sheer number of products is a bit staggering … what we’re seeing is a highly fractured strategy from different companies,” explained Nielsen, who said he expected more consolidation on capabilities.

    After evaluating the DevSecOps tooling landscape, The Radar team organized their core findings into three main takeaways:

    Usability is compromised at the expense of security. Security features are necessary, but they often come at the cost of usability. The DevSecOps Radar team found that the developer experience for most tools is poor. Many disparate tools must be stitched together to have a truly automated pipeline which is cumbersome to implement. “The tools are getting better, but still there’s not a cohesive, prescriptive way to develop for the cloud,” said Nielsen.

    The pace of change in the security space is rapid. Major cloud providers continue to rapidly release new services, which has led to an explosion in new tools. Simultaneously, the speed of innovation is accelerating within companies, opening up unique security needs. Combined, this means the security landscape is constantly shifting. “Security tools are trying to keep up with the services and how you host them and run them,” Nielsen said. “We definitely weren’t expecting to see so many newcomers in the security space,” Petean agreed. Though DevSecOps is spoiled for choice these days, “it’s always better to have options,” he added.

    Microsegmentation is important but presents a significant challenge. Many great tools enable companies to improve how they segment traffic, such as service mesh tools like Linkerd and Istio. However, implementing cloud-native microsegmentation is bumping against legacy implementations for established companies. Between API gateways, service mesh, edge firewalls and Kubernetes federated firewall functions, DevSecOps has much complexity to deal with, Nielsen said.

    The DevSecOps Technology Radar: In Review

    In recent years, cloud-native technology has placed increasing emphasis on solving the security puzzle. Companies continue to reevaluate their security approach in light of cloud-native strategies, novel vulnerabilities and new compliance requirements. DevSecOps is proving to be a useful, if not necessary, practice to maintain both security and rapid software delivery.

    With new security tools and unique approaches coming out so frequently, it’s easy to have “analysis paralysis.” However, DevSecOps must start somewhere. And, for those who trust the opinions of the CNCF Technology Radar, you now have some helpful insights on where to begin.

  • What 30 Years of Linux Taught the Software Industry

    What 30 Years of Linux Taught the Software Industry

    Linux has become the largest collaborative development project in the history of computing over the last 30 years. Reflecting on what made this possible and how its open source philosophy finally imposed itself in the industry can offer software vendors valuable lessons from this amazing success story.

    The web may not have reached full adulthood yet, but it has already crafted its own mythology.

    August 25, 1991: Linus Torvalds, a 21-year-old university student from Finland, writes a post to a Usenet group: “Hello everybody out there using minix — I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) for 386 (486) AT clones […]”. A few weeks later, the project, which will eventually be known as Linux, is published for the first time.

    This is the starting point of an epic that few could have foreseen.

    Fast-forward 30 years and the Linux kernel isn’t only running on most of the web servers and smartphones around the globe, but it also supports virtually all of the much more recent cloud infrastructure. Without open source programs like Linux, cloud computing wouldn’t have happened.

    Among the major factors that propelled Linux to success is security. Today, the largest software companies in the world are taking open source security to new levels, but the Linux project was one of the first to emphasize this.

    How Linux Became the Backbone of the Modern IT World

    Brief History

    Open source predates the Linux project by many years and is arguably as old as software itself. Yet, it is the success of the latter that propelled this movement in the 1990s. When it was first submitted for contribution in 1991 by Torvalds, the Linux kernel was the GNU project’s ‘missing link’ to a completely free software operating system, which could be distributed and even sold without restrictions. In the following years, and as the project started to incorporate proprietary licensed components and grow in popularity, a clarification on the meaning of “free software” became necessary.

    This led to the coining of the term “open source” as we use it today, thanks in part to Eric Raymond’s seminal paper The Cathedral and the Bazaar, a “reflective analysis of the hacker community and free software principles.” Open source was chosen to qualify software in which the source code is available to the general public for use or modification from its original design, depending on the terms of the license. People may then download, modify and publish their version of source code (fork) back to the community.

    Open source projects started gaining traction in the late nineties thanks to the popularity of software like Apache HTTP Server, MySQL and PHP to run the first dynamic websites on the internet.

    Facts and Figures

    Today, not only is Linux powering most of the digital era, but open source has become the leading model for how we build and ship software. Though most people don’t realize it, much of the technology we rely on every day runs on free and open source software (FOSS). Phones, cars, planes and even many cutting-edge artificial intelligence programs use open source software. According to the Linux Foundation, 96.3% of the world’s top one million servers run on Linux and 95% of all cloud infrastructure operates on it. Other infrastructure also relies on open source: 70% of global mobile subscribers use devices running on networks built using ONAP (Open Network Automation Platform).

    Linux adoption is very high in professional IT, where it’s become a de facto standard, especially with the advent of the cloud era. In fact, 83.1% of developers said Linux is the platform they prefer to work on. This success is due, in large part, to the community that contributed to its source code since its creation: More than 15,000 developers from more than 1,500 companies. Linux went on to become, arguably, the biggest success story of the free software movement, proving that open source could lead to the creation of software as powerful as any sold by a corporation.

    The Linux Foundation, a non-profit technology consortium founded in 2000 to support the collaborative development of Linux and OS software projects, is itself a big success. It now has more than 100 projects under its umbrella, spread across technology sectors like artificial intelligence, autonomous vehicles, networking and security. Several subset foundations have also emerged over the years, including the Cloud Foundry Foundation, the influential Cloud Native Computing Foundation, and the recently announced Open Source Security Foundation. The Foundation estimates the total shared value created from the collective contributions of its community to a whopping $54.1 billion.

    All these achievements may not have been possible without the embrace of open source by the enterprise world, which may represent its biggest win.

    Enterprise Adoption

    Companies began to realize that many open source projects were easier and cheaper to implement than asking their developers to build the basic pieces of an internet business over and over again from scratch.

    Twenty years ago, most businesses ran atop proprietary software from Microsoft, Oracle and IBM, and the idea of collaborating on big software projects might have sounded laughable to them. Today, these companies, along with relative newcomers such as Google, Facebook and Amazon, are not only employing thousands of full-time contributors to work on open source projects like Linux, they also regularly choose to open source some of their state-of-the-art projects; from Google Brain’s machine learning platform TensorFlow and container orchestration platform Kubernetes to Facebook’s React.

    There’s no question that open source software created a new wave of business opportunities. As more companies took an interest in open source projects, they realized they didn’t necessarily have the in-house expertise to manage those projects themselves and turned to startups and larger companies for help.

    Even Microsoft, which famously warred against the very concept of Linux for nearly a decade, made a strategic shift to embrace open source in the 2010s, led by CEO Satya Nadella. The IT giant finally joined the Linux Foundation in 2016 and acquired GitHub, the largest host for open source projects, two years later. It has since become one of the biggest sponsors of open source projects.

    As a consequence, the stakes have been raised for open source software, which is the engine powering the shift toward the cloud for virtually every company. In this context, security is becoming a topic of the utmost importance, and the commitment to secure the open source ecosystem is growing fast.

    Setting a Standard for Security and Trust

    Open Source Security Issues

    Following the OSS adoption boom, the sustainability, stability and security of these software packages is now a major concern for every company that uses them.

    The Census II report on structural and security complexities in the modern-day supply chain “where open source is pervasive but not always understood” revealed two concerning trends that could make FOSS more vulnerable to security breaches. First, the report said it is common to see popular packages published under individual developers’ accounts, raising the issue of security and reliability. Second, it is very common to see outdated versions of open source programs in use, meaning they contain fewer security patches.

    The 2021 OSSRA report agrees: “98% of the codebases audited over the past year contain at least one open source component, with open source comprising 75% of the code overall.” The report also noted that 85% of the audited codebases contained components “more than four years out of date”.

    This highlights the mounting security risk posed by “unmanaged” open source: “84% of audited codebases containing open source components with known security vulnerabilities, up from 75% the previous year. Similarly, 60% of the codebases contained high-risk vulnerabilities, compared to 49% just 12 months prior.” Not only is the security posture affected, but there are also compliance issues that can arise from unsupervised integration of open source content because licenses can be conflicting or even absent.

    Because large corporations are now a big part of the open source ecosystem, their sponsorship is a welcome source of financing for many people whose work had been done for free until now, yet it may not be enough. The open source community is well-known for its commitment to independence, its sense of belonging and its self-sufficiency, and expecting contributors to voluntarily address security issues is unlikely to succeed.

    This is where the experience of building Linux over 30 years and coordinating the work of thousands of individual contributors may be an example to follow.

    Linux Foundations

    In Linux kernel development, security is taken very seriously. Because it is an underlying layer for so many public and private software ‘bricks’ in the digital world means that any mistake can cost millions to businesses, if not lives. Since the beginning, it has adopted a decentralized development approach with a large number of contributors collaborating continuously. Therefore, it has consolidated a strong peer-reviewing process as the community development effort grew and expanded.

    The last stable release at the time of writing is 5.14, released on August 29th, 2021, only a few days before the 30th birthday of the project. The most important features in the release are security-related: One is intended to help mitigate processor-level vulnerabilities like Spectre and Meltdown and the other concerns system memory protection, which is a primary attack surface to exploit. Each Linux kernel release sees close to 100 new fixes per week committed by individuals and professionals from the likes of Intel, AMD, IBM, Oracle and Samsung.

    With such broad adoption and long history, the Linux project has reached a level of maturity that few, if any, other FOSS projects have seen. The review process and release model have built confidence for numerous downstream vendors. Although the world is not perfect and it is arguably difficult for them to keep up with such a high rate of change, they can at least benefit from strong security enforcement mechanisms and they can adapt their security posture in concordance to their “risk appetite”: Vendors are able to do the calculus of determining how old a kernel they can tolerate exposing users to.

    Pushing the Boundaries of Open Source Security

    Heartbleed and the Fragility of OS Security

    In April 2014, a major security incident affecting the OpenSSL cryptography library; disclosed as “Heartbleed.” The developer who introduced the bug acknowledged that, though he was working on the project with a handful of other engineers:

    “I am responsible for the error because I wrote the code and missed the necessary validation by an oversight. Unfortunately, this mistake also slipped through the review process and therefore made its way into the released version.” OpenSSL, an open source project, is widely used to implement the Transport Layer Security (TLS) protocol. In other words, it’s a fundamental piece used to secure a large part of the web.

    Open source was seen as fundamentally secure for a long time because the more people examine a line of code, the better the chances of spotting any weakness. Additionally, this model prevents “security by obscurity,” whereby the bulk of the protection comes from people not knowing how the security software works—which can result in the whole edifice tumbling down if that confidential information is released or discovered externally.

    This incident was a major turning point for a large share of the biggest web corporations: They realized that many open source technologies underpinning their core operations could not be “assumed to be secure” anymore. Any human error could have huge implications; therefore, a specific effort had to be made to improve the security in this specific space.

    A New Era for Open Source

    As we advance in an era where open source is omnipresent in codebases, tooling, networks and infrastructure and is even in fields other than software, security awareness is starting to take hold. But it needs a lot more work.

    A big part of the challenge, to begin with, is for the industry to understand the scope of the problem.

    Google just announced that it will be committing “$100 million to support third-party foundations that manage open source security priorities and help fix vulnerabilities.”

    The Secure Open Source (SOS) pilot program, run by the Linux Foundation, will reward developers for enhancing the security of critical open source projects that we all depend on.

    In doing so, Google leads the way in enlarging the financial sponsorship of big players like companies and governments — which are increasingly sponsoring open source both directly and indirectly. However, they also recommend that organizations “understand the impact they have on the future of the FOSS ecosystem and follow a few guiding principles.”

    What could these principles look like?

    A Roadmap to Safely Use and Contribute to Open Source

    The Linux Foundation proposed a specific Trust and Security Initiative which describes a collection of eight best practices (with three degrees of maturity) open source teams should use to secure the software they produce as well as by a larger audience to “raise the collective security bar.” Here they are:

    • Clarifying the roles and responsibilities, and making sure everyone is aware of their security responsibilities across the organization.
    • Setting up a security policy for everyone; in other words, a clear north star for all members of the organization.
    • ‘Know your contributors’ is defined as a set of practices to make risk-based decisions on who to trust and fight offensive cyberwarfare techniques, such as the poisoning of upstream code.
    • Locking down the software supply chain: This has become a preferred target as adversaries clearly understood that they can have a bigger and more effective impact with less effort than targeting individual systems.
    • Provide technical security guidance to narrow potential solutions down to the more appropriate ones in terms of security.
    • Deploy security playbooks to define how to do specific security processes, specifically incident response and vulnerability management processes, like creating roles and responsibilities or publishing security policies. This may feel formal, antiquated and old-school but having pre-defined playbooks means that teams can focus on shipping software and not learning how to do security, especially at the least convenient and most stressful time.
    • Develop security testing techniques with automated testing strongly recommended since it scales better, has less friction and less cost to the teams and aligns well to modern continuous delivery pipelines.

    However, the authors of the guide are aware that some major challenges are still facing the industry and, as such, need to be addressed. They mention:

    • The lack of open source security testing tools
    • The fact that open source package distribution is broken
    • The fact that the CVE format for vulnerability disclosure is also broken
    • The lack of a standard for a security build certificate which would allow any consumer to transparently verify that a product or component complies with the announced specifications

    “The types of verification can and should include the use of automated security tools like SAST, DAST and SCA, as well as verification of security processes like the presence of security readmes in repos and that security response emails are valid.”

    A scheme like this could have a significant and lasting effect on the security quality of open source software and the internet at large.

    The Linux project, born 30 years ago, is present in all layers of the modern software stack today. It is used by all the largest server clusters powering the modern web and any business going digital will use it at some point. This unparalleled longevity and success have demonstrated that the open source model was compatible with the requirements of enterprise-grade services and economically viable. Now that open source is all the rage in the software industry, a consensus and action plan on how to ensure the sustainability of this ecosystem becomes urgent. The top priority for businesses that depend on it is to adopt strong application security guidelines, like the ones promoted by the Linux Foundation, which have proven their value.

    One last note on the nature of open source: As businesses are now much more bound by the common use of open source components to build upon, they should not fall into the “tragedy of the commons” trap. This would mean waiting until others take action; for instance, to improve the global software security landscape. This might be one of the biggest challenges confronting our highly collaborative industry.