devopsy.kinsta.cloud

Tag: open source

Why the Linux Kernel is Still Important

Last year, the Linux kernel turned 30. If you’re someone who’s been immersed in the Linux world since Y2K like I am, it may feel a bit surreal that so much time has passed since the kernel’s inception.

As a training architect at A Cloud Guru (ACG), I teach courses about all things Linux and specialize in hands-on lab-based learning. Before joining ACG, I worked as a Unix systems engineer at GE and IBM as well as a technical account manager and customer advocate for Red Hat. I’m hugely passionate about Linux, just as so many other engineers and enthusiasts are, not just because of its importance to our careers but its impact overall.

In its 30 years of existence, the Linux kernel has had a massive impact on the modern computing landscape—revolutionizing what’s possible for operating systems and allowing countless tinkerers to get their hands dirty in the process. Linux has also become the foundation of paradigm-shifting innovations over the years due to its ever-evolving nature.

Here are some of the key reasons why the kernel is so valuable and why it will continue to be a major player in the computing landscape for years to come.

My Own Linux Journey

My first exposure to Linux was in college in 1999. I took an introductory Unix Shell Scripting class and Fedora was installed on the lab servers because it was free and easily scalable. I view Linux as my entry point into customizable operating systems. After learning basic Unix commands on Linux, I ultimately went on to pursue a career supporting multiple commercial Unix vendors.

As Linux made its way into larger enterprise-level companies, I quickly returned to working with Linux distributions. Something I’ve always loved about Linux is the capacity to fine-tune your system to support the applications and distros that work best for your projects. Linux improves the functionality of whatever applications you’re running. The Linux kernel has literally changed how the world processes information, which is why I’m so invested in the software.

Now, it’s my job to share my knowledge and passion for Linux with other technologists. At ACG, I develop courses to help aspiring Linux experts learn how to optimize their systems. Additionally, I contribute to Linux this Month, an ACG-hosted web series that provides monthly updates from the global Linux community.

The fact that I am able to build a career around teaching Linux and staying up-to-date with Linux news shows the vast use cases and applications of the kernel. What makes Linux so unique and evergreen is its open source nature—Linux innovations are only limited by the creativity of the technologists who use and adapt it.

The Open Source Effect

When Linux first arrived, it was mostly a hobby for enthusiastic engineers and computer science students who could contribute by developing code. The steep learning curve associated with fitting Linux to your machine was a barrier to entry for more novice programmers.

Over time, this has changed considerably. Online forums, workshops and classes have made Linux more accessible to the average internet user. The free sharing of ideas has come to epitomize the open-source community, and for software engineers, Linux is at the heart of this community. This democratization of Linux has had incredibly positive impacts on the computing world.

Now, Linux is everywhere. Enterprise-level companies use Linux distributions to process the biggest production workloads in the world. It has replaced proprietary commercial Unix operating systems in very large companies with better stability and less downtime. Because Linux systems can be as small or as large as you want, it’s also now being used in our homes for smart and mobile devices as well.

The open source nature of Linux is incredibly beneficial for these enterprise-level companies. Linux distributors leverage the contributions of the entire open source community. This wide range of contributors produces a more stable product with more features but also ensures that the OS keeps growing and solving real-world problems that are beneficial to a wide range of users.

Linux Kernel Runs the Cloud

Potentially the most impactful outcome of the kernel is its importance to the infrastructure of modern cloud computing. Linux’s scalability has paved the way for supercomputers and server farms to function efficiently while requiring relatively lightweight computing resources. In fact, Linux supports about 90% of the public cloud workload.

Without Linux, the cloud as we know it would not exist. This is, in part, because Linux has become so ubiquitous—its use cases are nearly limitless. Because it has been time-tested, many engineers and IT professionals have a solid grasp of Linux fundamentals, making it an attractive choice for enterprise companies dealing with the cloud.

Cloud-based software and products are increasingly becoming the norm in the engineering world. Unsurprisingly, major cloud providers such as AWS, Azure and Google Cloud are all supported by Linux as well. Linux is unique because it is a shape-shifter that can conform to the needs of any given engineering environment, and it’s incredibly stable because of the army of contributors that fortify weak points in the software.

Linux Forever

The concept of Linux is just as important as the Linux product. The concept allows a free and open source operating system to be refined, reinforced and replicated across an endless web of contributors. Thirty years is a long time for software to be relevant, especially with the ever-shortening tech product cycles. Because it was designed with the intent to be changed and updated by an open source community, Linux has no foreseeable expiration date.

Had Linux not achieved the prominence it has today, we would see more commercial Unix vendors attempting to solve some of the problems that Linux addresses, but none would address them all. Additionally, customers would have to choose which OS to invest in based on which addresses some of their use cases, but none would be as beneficial as the Linux OS.

Linux is always growing and will become even more popular within the next few years. As more people become familiar with Linux and learn to use it, I see major potential for growth in the mobile computing space, within personal computers, and across small and large companies. In fact, we are already seeing it filter into home gaming systems and Raspberry Pi projects. With Linux, the sky’s the limit!

March 17, 2022
Secure Software Summit: The State of OSS Supply Chain Security

The open source software (OSS) supply chain is under attack. As evidenced by the recent Log4Shell vulnerability, the OSS supply chain is increasingly a focus for attackers seeking to exploit weak links in security. A number of research reports have recorded a significant increase in so-called ‘next-gen software supply chain attacks‘ over the past decade. These attacks look to compromise key links in the software supply chain including source code, build systems, CI/CD tools and code testing tools, among others. Next-gen attacks are happening across all languages, all types of software and are increasing in their frequency. Common forms include typosquatting, malicious code injection and tool tampering (like the CodeCov incident). In 2021, 64% of organizations reported they were affected by software supply chain attacks, according to container security company Anchore.

The increasing wave of next-gen attacks has driven new regulations mandating specific steps to inventory and make more transparent the content of software applications and the software supply chain. Specifically, executive order 14082, issued by U.S. president Joseph Biden, mandated a long list of security practices, including the inclusion of a software bill of materials (SBOM) with every application run by the U.S. federal government. The National Institute of Standards and Technology (NIST) is creating reference architectures and templates for application security as a result of the executive order. In other words, software security regulations will likely impact every company that produces code or sells products running on code (including SaaS products).

To be clear, some of the problems experienced by the OSS software supply chain are not exclusive to open source. The recent supply chain attacks on Microsoft Exchange Server and the SunBurst attack on the SolarWinds network management tool targeted proprietary code bases. That said, OSS is produced and maintained very differently than proprietary software. This does result in some unique risks. Here, we will cover those risks as well as some promising initiatives underway to address them.

Why Is the OSS Supply Chain A Juicy Target?

There are two primary reasons OSS supply chains are a tempting target. One is its omnipresence and the other is its weakness. It is not an exaggeration to say that OSS is omnipresent. According to Sonatype, 98% of all codebases contain OSS. This is likely an underrepresentation. You simply cannot build code from scratch today without using OSS components. It is too labor-intensive and expensive. The ubiquity of OSS makes the supply chain that creates, distributed and manages it an obvious target. At the same time, that widespread use translates into an increasing burden on companies to maintain, update and patch all OSS dependencies in their applications.

This is an impossible task; it is far easier to insert OSS code to solve an immediate problem than to, over time, make sure that code is updated and not vulnerable. For this reason, Sonatype estimated that 92% of all applications contain outdated or vulnerable code. We have seen the impact of this inability of organizations to maintain their OSS code through attacks like the one perpetrated against Apache Struts software run by credit reporting company Equifax, a breach that exposed the sensitive personal and financial information of hundreds of millions of Americans.

In recognizing that OSS is a fruitful channel for attacks, attackers have focused more specifically on the supply chain because it is often the weakest link. Elements of the supply chain reside in the gap between responsible organizations and may be less strictly policed or less actively secured. Continuous integration/continuous deployment (CI/CD) tools, OSS libraries that can easily access other systems, package managers and open source developer tooling are all components of the supply chain that are widely used but which are often the sole responsibility of many part-time maintainers who write and maintain code as an unpaid avocation.

OSS-Specific Challenges

The three key benefits of OSS each create unique security and supply chain challenges. Specifically, OSS is open, free and transparent.

The Two Sides of Open

Linus Torvalds said that many eyeballs make all bugs shallow. To a degree, this is true. But what if those same eyes are looking for bugs to exploit rather than fix? Not everyone on the internet is nice and many seek to exploit the fact that anyone with a GitHub account can contribute code to critical libraries. This attracts a lot of great contributors, but also a lot of bad actors and bad behavior.

That behavior is often hidden or delayed. For example, we often see bad actors ask to contribute to projects or libraries run by single maintainers that are relatively popular and used by big applications. Often, maintainers struggle to keep up with the demands of these projects. A bad actor will offer to help, submit a handful of good PRs and gain commit privileges to the repository. At this point, the bad actor will then stealthily insert malware, such as a cryptojacking script.

Free Puppy. Not Free Beer.

Free is a good price. There are no purchase orders. This makes it super easy to use. Free also serves as a great distribution strategy. However, sometimes free is not worth it. If an organization depends on an application and it becomes critical, they might require fast patching, better compliance and official support relationships. Free OSS comes with no guarantee, and this can be a major liability. An application’s maintainers might be writing the code and applying fixes in their spare time on weekends because they already have a full-time job. In fact, research has shown that a significant percentage of the critical software powering the internet is maintained by volunteers; this was the cause of the Heartbleed vulnerability in the OpenSSH encryption libraries.

Free code also may not be updated for years. If a vulnerability is disclosed, you may not be able to get permission to merge fixes you write. If the code is abandoned and the author no longer checks comments or requests, then you are stuck and may have no choice but to fork the project. A hard fork on short notice can cause chaos and its own problems. This is what happened with a popular Golang library called JWT Go, which made its way into dependency trees across the Golang ecosystem. JWT Go was even included in Kubernetes, one of the most popular and important open source projects. When a vulnerability to JWT Go emerged, everyone realized the maintainer had abandoned it. Multiple forked versions emerged with different patches for the same bug. Chaos ensued.

It is a common refrain to ask companies to fund volunteer maintainers. There are numerous programs for this, including GitHub Sponsors, Open Collective and Tidelift. These programs are helpful but not sufficient. You can’t just hand people a small amount of money and get more secure code and professional-grade maintenance. Even if you offered full-time salaries, it might not translate into well-supported, secure code.

The Illusory Benefit of Transparency

The transparency of OSS is real, but the idea that this code is getting audited and reviewed constantly is an illusion. Yes, the source code is distributed and open. But the sheer quantity makes it impossible to review all the code you use. If you are using millions and tens of millions of lines of code, you don’t have time or sufficient eyeballs. Even if you are spending considerable resources on code review, it is still easy to miss problems and vulnerabilities and relatively simple for malicious contributors to projects to hide malware or other bad inclusions. Reviewing the source code also does eliminate all risks. When most OSS code is consumed, it is running as a package compiled by another entity or developer.

Further, there is no guarantee that the contents of the package comes from the same source code you audited. We have actually seen this attack often with package repositories like npm and Maven, where malicious actors insert tampered builds or take over packages and insert code that is not listed as part of the package.

The CodeCov incident is one of the more recent and egregious examples. This free service for checking the code coverage of unit tests used BASH script running inside of a CI server for installation. The script was compromised and attackers inserted malware that could steal code secrets and export them. Those secrets might be tokens used to publish packages in npm or Maven and exposing those secrets allowed attackers to then compromise those packages. In another example, a PHP server managing source code distribution for the PHP project was compromised and multiple commits injected malicious code into the source code repository. The incident was caught quickly, but it demonstrated how attackers could tamper with source code directly, bypassing code review and security processes.

But All is Not Lost

Innovative efforts to better secure open source and its supply chain are gaining momentum. The SBOM is gaining acceptance and it is getting easier to automatically generate SBOMs from CI/CD pipelines or build systems. A new initiative to make it easy to cryptographically sign code, called Sigstore, aims to remove barriers to code signing; ubiquitous code signing would make it much harder to compromise the supply chain by attacking the source code provenance problem at its root. Lastly, the Open Source Security Foundation has garnered tens of millions of dollars in funding from major technology companies and is in the process of spinning up mechanisms to issue grants and otherwise provide a more robust financial support mechanism for OSS. These developments point to a brighter future where the OSS Supply Chain is far more robust and companies can more easily audit, track and trust all the OSS dependencies in their applications and the full supply chain that moves OSS code from developer boxes up into production environments.

About Secure Software Summit
The Secure Software Summit brought together the world’s leading innovators, practitioners and academics of secure software development to share and teach the latest methods and breakthroughs on secure coding and deployment practices. If you are about developing, releasing and securing software, delivering new features fast and building things right from the start, click on the link below to get access to all of the sessions from this summit.

https://go.shiftleft.io/secure-software-summit-2022-replay

March 15, 2022
Linux Foundation Lists Top Open Source Libraries

The Linux Foundation today published a report that provides access to eight lists of the top 500 open source libraries being used by organizations as part of an ongoing effort to help better secure software supply chains.

The Census II of Free and Open Source Software—Application Libraries report is based on usage data from providers of software composition analysis (SCA) tools such as Snyk, the Synopsys Cybersecurity Research Center (CyRC) and FOSSA. The report itself was compiled by the Harvard Lab for Innovation Science.

The eight lists are made up of four that include version numbers and four that are version agnostic. The lists are separated into npm and non-npm packages because npm packages would dominate any ranking created. The top non-npm packages being employed include maven, nugget, Go and cargo.

Frank Nagle, assistant professor at Harvard Business School, said the report is intended to provide organizations with some guidance in terms of knowing what open source packages are used in the wake of recent vulnerabilities discovered in the widely employed Log4j software used to manage logs within Java applications.

Many of the organizations impacted by that vulnerability were unaware of just how widely deployed Log4j was within their enterprise IT organizations. IT organizations should evaluate the sustainability of those projects in terms of the number of contributors and maintainers that are working to ensure the security of those libraries.

One of the biggest challenges is to find a way to standardize naming schema for software components and the overall management of versions of different libraries, noted Nagle.

Ultimately, the goal is to convince more enterprise IT organizations—and the vendors that support them—to make more resources available to secure those libraries, noted Nagle. Much of the most widely used open source software is developed and maintained by only a handful of contributors. Many of those contributors argue that, while they made that software available for free, it is the responsibility of the organizations employing it to secure it.

The Linux Foundation is moving to become a conduit through which it will make available additional resources to help better secure open source software. The Open Source Security Foundation (OpenSSF), a consortium hosted at the Linux Foundation, earlier this week announced that 19 additional organizations have joined OpenSSF to help identify and fix security vulnerabilities in open source software and improve tooling, training, research, best practices and vulnerability disclosure practices.

Brian Behlendorf, executive director at OpenSSF, said in addition to making resources available, it’s clear there is a need for some form of a third-party audit of open source software as organizations are more broadly employing that software across their software supply chains.

New premier member commitments to OpenSSF are now coming from 1Password, Citi, Coinbase, Huawei Technologies, JFrog and Wipro. New general member commitments are being made by Accuknox, Alibaba Cloud, Block, Inc., Blockchain Technology Partners, Catena Cyber, Chainguard, DeployHub, Gravitational Inc., MongoDB, NCC Group, ReversingLabs, Spotify and Wingtecher Technology. New associate members include Institute of Software, the Chinese Academy of Science (ISCAS), MITRE and OpenUK.

Other efforts led by the OpenSSF include an Alpha-Omega Project to better secure open source security posture, scorecards that identify risks of the dependencies within more than one million projects, multi-factor authentication tokens and a Project Sigstore initiative to sign, verify and protect open source code.

It may be a while before open source software achieves the level of security the IT industry strives for. On the plus side, however, there have never been more resources allocated to achieving that goal.

March 2, 2022
Without a Universal Language, Business Systems are Doomed
Imagine a world without DevOps. Everything that could be automated isn’t. Any job you dream up isn’t possible and, instead, you have to dig 10 to 15 clicks deep into a friendly but oppressive software interface to make every single change. Oh, and there’s no way to run searches.

Picture that and you have a good sense for what managing most business systems (CRM, ERP, HCM, MAP and other three-letter acronyms) are like today. They’ve been designed to make things easy on users, not efficient for administrators and it forces admins to work out of a schematic island—the interface—where no-code conventions rule.

Yes, there are developer modes, but they aren’t the cure-all you’d imagine. Your business likely has 800+ business applications and whatever you build in one you have to rebuild in all. Not all have dev modes, and no two dev modes use the same language.

This is all to say that, unless business systems get a universal language that works across all of them, businesses will be in deep trouble. I talk to lots of companies about this and many already are.

The Cost of Chaos

If you think about all your business systems together, they constitute a sort of “product” upon which the entire business runs. When that product is built out of 800+ services that can only be governed individually, you have a lot of problems.

As examples, companies I talk to have experienced the following horrors: A minor change caused sales leads to go unrouted for six months. Those leads grew stale, and the company wasted both the money spent acquiring them and lost millions in potential business. Another found that changes in their CRM created an error in the ERP and invoices didn’t go out until after the quarter’s end. (You should have seen the investors’ reactions). Another had a minor error trigger a dormant email campaign that blanketed their database with out-of-date offers.

This list could go on and on, and the effects are real. From managers who lost out on rockstar candidates because the HR system was undergoing maintenance and they couldn’t send an offer to arcane DOS-based internal systems that have become so enmeshed they are irreplaceable—business systems are vital, and most of them are a mess.

In fact, business applications teams face all the same challenges that software developers do, but benefit from none of their tools. They can’t search through one formal representation of all their applications’ code. (To say nothing of searching across applications.) They can’t easily back up, save versions, collaborate or evaluate PRs except within each system’s frustratingly isolated dev mode.

As a result, they often can’t:
- Simply see a blueprint of how things are set up today (especially across applications)
- Analyze impact across systems (there are tools, but they’re always app-specific)
- Save versions and revert to a prior state
- Run “find and replace” searches for settings buried 10 to 15 clicks deep
- Figure out who did what, where and why (a real problem for SOX compliance)
This would be all well and good if business systems were due for a sudden and rapid consolidation, but they’re not. They’re so not. The SaaS market is still growing at 19% year-over-year and some sub-markets are growing faster, like MarTech, at 25% year-over-year. It’s given rise to tragicomic diagrams like this one by Chiefmartec’s Scott Brinker. Companies are buying and relying on epic levels of SaaS.

Moreover, today’s business systems accumulate in vertical stacks. The financial applications team owns the ERP and other financial applications like Stripe and Zuora. The Salesforce team owns Salesforce. But business happens horizontally. Leads go from marketing to sales to finance to customer success. All those siloed applications aren’t really built to allow administrators to create a reliably error-free “product” out of all that. Unless, of course, we get a universal language.

A Plea: It’s Time for Standards

I find it pretty unlikely that any one SaaS vendor is going to lead the charge for universal conventions for managing and configuring business applications. Each SaaS company views itself as the center of its own universe, with all other apps revolving around it. Their app stores reinforce that viewpoint.

It’s going to take an open source third party to build such a system, but here’s what having a universal language for managing your business systems can mean: You get DevOps across all your business applications.

To the Salesforce developer who couldn’t care less about NetSuite, that’s fair. But what if I told you that with this standardized language, everything you know about managing Salesforce could be applied to NetSuite if you were pulled into that project? Or vice versa? It’s a lot like suddenly being able to speak an entirely new language.

For the heads of departments whose purview spans applications and those who are already rearranging their teams to figure out how to be more agile and support the business, this is (potentially) for you. It doesn’t connect your applications (you’ll still need a Workato or a Boomi, for example), but it’ll give you and your team one view into how multiple applications are configured, and the ability to make changes and push them into production without going 10 to 15 clicks deep into the interface.

It allows you, in essence, to apply all of those DevOps practices you’ve been growing accustomed to in each dev mode across multiple systems. You can run searches, use a Git versioning tool, practice CI/CD, automate documentation, conduct impact analysis and scale your efforts.

Imagine a business systems world with DevOps. That’s possible, and I’d love for you to check it out and give us feedback.
February 22, 2022
The Past, Present and Future of OpenStack

OpenStack, described as the world’s most widely deployed open source cloud software, will mark its 12-year anniversary milestone later in 2022. In September 2012, the OpenStack Foundation launched as an independent organization to manage and promote the technology—a watershed moment for a project announced just two years earlier.

Today, with contributions from more than 34,000 people at 550 companies, OpenStack is one of the three most active open source projects, alongside the Linux kernel and the Chromium web browser, according to the group now known as the Open Infrastructure Foundation (OIF). More than 25 million OpenStack cores are running in production worldwide.

It’s easy to forget that only a few years ago many predicted OpenStack’s demise. Initially, OpenStack was positioned as a general-purpose private cloud platform that offered an open source alternative to Amazon Web Services (AWS), but it never stood much of a chance in that matchup.

Then, after OpenStack regained momentum and pivoted to become a platform for telco network function virtualization infrastructure (NFVI), it was rumored that it would be overtaken by the new ‘next big thing’—Kubernetes, the open source orchestration and coordination system for containerized applications.

The Kubernetes-as-OpenStack-killer talk was silly, in retrospect, because the technologies are complementary, not competitive. In fact, more than 70% of OpenStack users reported running Kubernetes on top of OpenStack for container coordination, according to the OIF.

And OpenStack adoption should continue to flourish in 2022. Gartner forecasted that global spending on cloud services will exceed $480 billion this year, up from $313 billion in 2020, as more organizations invest in hybrid cloud: A mix of pay-as-you-go public clouds and private clouds that can offer lower, more predictable operating costs and greater customization. As the hybrid cloud market expands and open source continues to become more firmly entrenched in development and IT organizations, OpenStack is more relevant than ever.

If you’re not familiar with OpenStack, here are 10 things you should know about its past, present and its future.

1. OpenStack began in early 2010 as a collaboration between Rackspace and engineers at NASA’s Ames Research Center in Mountain View, California. Rackspace wanted to rewrite the infrastructure code running its cloud servers and the NASA team wanted to standardize the space agency’s websites. Their original mission was “to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.”

2. The first OpenStack design summit was held in Austin, Texas in July 2010, and the project was officially launched at the O’Reilly Open Source Convention (OSCON) later that month.

3. A whopping 80% of OpenStack clouds are used in production, while 13% are under development and 8% are in the proof-of-concept stage, according to the OIF. With that level of adoption, no wonder OpenStack is considered the most widely deployed open source cloud software.

4. An OIF survey showed that nearly half of the organizations using OpenStack have deployed at least 1,000 cores. The breakdown: 29% of organizations have 1,000-9,999 cores, 13% have 10,000-99,999 cores, 6% have 100,000-999,999 cores and 1% have one million or more cores.

5. An enormous corporate ecosystem supports OpenStack, including IBM, Red Hat, HP Enterprise, Intel, Huawei, NEC, Cisco, VMware, AT&T and Canonical, whose Ubuntu Linux distribution is the most popular operating system for OpenStack deployment (used by about 40% of OpenStack clouds globally).

6. OpenStack has a great resource guide available on openstack.org with all kinds of helpful information for developers, including for those new to the platform.

7. One of the main qualities people love about OpenStack is that it’s a modular architecture with interconnected components including, for example, services that expose API endpoints and handle basic cloud functions, a dashboard, SQL databases that store records created by OpenStack services and message queues for facilitating inter-process communication.

8. OpenStack jargon includes several special terms. A tenant, for example, is a user or group of users with isolated cloud resources. (OpenStack is multitenant.) An image is a template containing an OS for instance provisioning. A flavor is a template defining cloud resources for instance provisioning. An instance is a virtual machine (VM) provisioned from the image and flavor. And so on.

9. The platform has expanded through the years to include a panoply of core services, such as the Keystone identity service that manages domains, projects, roles, groups and user accounts, the Glance service that manages the catalog of cloud images, Nova for instance provisioning, scheduling and termination, the Neutron network service and the Cinder block storage service.

10. Beyond these core services, several newer services are seeing increased adoption. For example, according to the OIF, use of Ironic, a project that provisions bare metal, nearly tripled between 2016 and 2021. Others with growing traction include Octavia, which provides load balancing capabilities, Manila for file system sharing between instances and OpenStack Charms which provide rich life cycle management capabilities for services.

As these 10 points show, far from being on its deathbed, OpenStack has survived, sometimes against the odds, and thrived as a unique force in open source and cloud computing. Given its journey over the last 10 years, it will be interesting to see what lies ahead in the next decade.

February 18, 2022
Codenotary Launches Cloud Service to Generate SBOMs

Codenotary has launched a Codenotary Cloud platform that can automatically generate a software bill of materials (SBOM) and make it easier to discover what components have been included in an application.

Moshe Bar, Codenotary CEO, said that capability can also play a key role in identifying which components in an application might contain, for example, the Log4jShell vulnerabilities that were recently discovered in Java applications.

That capability can reduce the time required to remove insecure artifacts by as much as 80% simply because organizations are not dependent on scanning tools to search for vulnerable components, added Bar.

Codenotary Cloud is based on an immutable open source immudb database that cryptographically attaches an identity to each software artifact. In the wake of a series of high-profile breaches, there’s now a lot more focus on the integrity of software supply chains. In fact, the Biden administration recently issued an executive order requiring all federal agencies to, among other things, make sure an SBOM is available for all the software they use.

Codenotary claimed Codenotary Cloud can scale to millions of integrity verifications per second across source code, builds, repositories, Docker container images and Kubernetes deployments in a way that doesn’t require DevOps teams to upload data to a centralized platform, noted Bar. That makes it possible to notarize and verify the provenance of all the code being employed, he added.

Codenotary Cloud can also be fully integrated with most vulnerability scanners and popular cloud-native continuous integration/continuous delivery (CI/CD) systems running on any cloud or on-premises host via a managed service. Pricing starts at $5,500 for a workgroup of 10 developers.

It’s not clear to what degree the Biden administration’s executive order will improve overall software security. There have been multiple vulnerability issues over the years that have not resulted in any material change in how organizations manage their application development processes. However, the executive order issued by the Biden administration, coupled with a recent summit on the state of open source software security, means more senior business and IT executives are at least aware of growing concerns.

The one thing that is certain is there will be more zero-day vulnerabilities discovered. Developers today routinely reuse open source software and free components maintained by a small number of volunteer programmers. Like any other developer, the amount of security expertise those individuals have is limited. Many organizations reuse that code without contributing anything meaningful back to the project—either in terms of financing or vulnerability identification and remediation support. Contributors to these projects freely donate their time and expertise to build these components. Many of those contributors feel the onus for securing the software they created is on the organizations that decide to deploy that software.

Regardless of who is responsible for security, it’s apparent cybersecurity teams are going to be reviewing application development processes. Theoretically, that should benefit all concerned and lead to DevSecOps best practices being more widely adopted.

February 7, 2022
WhiteSource Tightens Code Scanning Tool Integration with Azure DevOps

WhiteSource has added the Microsoft Azure DevOps platform to the list of continuous integration/continuous delivery (CI/CD) platforms its open source vulnerability scanning tools natively supports.

Susan St. Clair, director of product management for WhiteSource, said while developers are always encouraged to scan for vulnerabilities, it’s more effective if organizations implement scanning by default every time there is a merge request. That approach reduces friction because organizations become much less dependent on developers to scan for vulnerabilities within the context of a larger DevSecOps workflow, she added.

WhiteSource also provides a merge confidence feature that uses crowdsourced data to show how likely it is that an open source component can be updated without breaking the build. Merge confidence includes data on upgrade age, adoption and compatibility to create a confidence score.

The WhiteSource integrations make it possible for DevOps teams to detect all open source components being used and automatically enforce security policies directly from their repository. DevOps teams are provided with vulnerability and misconfiguration alerts and license violations along with detailed remediation guidance, including suggested fixes and prioritization advice, within an existing workflow versus being required to switch to a tool that has a separate user interface (UI) they need to learn.

Should a merge request introduce a new error, the developer is given immediate feedback to resolve any newly introduced vulnerabilities before the request is completed. That approach to separating feature branches and mainline branches prevents interruptions to workflows. The enterprise edition of the WhiteSource tool also automatically generates pull requests in the repository to update vulnerable open source components to the lowest non-vulnerable version.

WhiteSource also supports code repositories such as GitHub, GitHub Packages, JFrog, Bitbucket and GitLab. The Azure DevOps platform is gaining traction as more application development projects are being managed via the cloud following the onset of the COVID-19 pandemic, noted St. Clair.

As more organizations begin to embrace DevSecOps workflows each of them will need to decide how far left they want to shift responsibility for application security. In theory, each developer is now being held more accountable for every one of their applications that gets deployed in a production environment. In practice, most developers lack the expertise required to ensure applications are secure.

There are, of course, more concerns being raised about the security of software supply chains that include open source software in the wake of the recent disclosure of zero-day vulnerabilities in the widely-used Log4j logging tool. Many of the maintainers of smaller open source projects lack the resources required to ensure there are no inadvertent vulnerabilities that could be exploited by cybercriminals. More challenging still, many developers may have used an older version of that software that has known vulnerabilities.

It’s not clear how open source software will ultimately be made more secure. In the meantime, however, it’s clear the onus for ensuring the security of that software is on the DevOps teams that employ it.

February 4, 2022
Heterogeneous Hardware Needs Universal Software

As hardware advances and diversifies, we’re entering what many see as a new golden age of computer architecture. However, the idea of an ever-expanding hardware landscape can be daunting for software developers, because while hardware diversity is wonderful for price and innovation, it can lead to spiraling software complexity on the back end. So many software tools are specific to the hardware they run on that a world where CPUs, GPUs, FPGAs, ASICs, DSPs and more are freely intermingled feels overwhelming; an invitation to interoperability headaches and spiraling tech overhead. The question on the minds of developers is how to navigate this constantly changing ecosystem to take advantage of the new world of hardware acceleration without driving themselves crazy in the process.

The challenge for developers is that, if we look historically, every time a new piece of world-changing silicon shows up, it often comes with a new software stack of tools. With heterogeneous hardware innovation moving at a nearly untrackable pace already, keeping up with the software end is beginning to feel untenable. In response, a new philosophy is emerging: We should be able to have a common set of tools, libraries, frameworks and compilers that can address a wide variety of accelerators in a “silicon agnostic” way.

Anyone doing DevOps work today should be asking, “Are the tools we are adopting ready for the heterogeneous hardware future?” Rather than using proprietary stacks exclusive to individual vendors’ concepts of how acceleration should work, we should be thinking about abstraction layers that allow us to move to open software stacks that work across multiple vendors and architectures.

This philosophy is central to a number of movements—oneAPI being one example—of asking for libraries, compilers and analysis tools that are more versatile. It’s incumbent on any future-conscious developer to analyze the tools they’re using and ask if they are preparing for the future or locking them into a single solution that excludes you from other innovations.

When it comes to navigating the changing developer landscape, it’s particularly important to look at the multiple-vendor piece of the puzzle. One of the signature strengths of computing and electronics is its ability to interoperate. Without compatibility between platforms, we end up locked into single solutions that don’t take advantage of the diverse landscape of innovation available to us. If systems are built from the ground up with mixing and matching in mind, we don’t have to change our software tools to adapt to new technologies and we can use the same languages, libraries and analysis tools we are used to.

It’s important we take on this work now because hardware diversity is not just at an all-time high, its momentum is only growing. In the past when the hardware landscape was more limited, switching to different toolsets for different CPUs was a feasible, if inelegant solution. Today’s computers, however, have computational capabilities from multiple vendors. If my tools don’t stretch across all of its ecosystems, I end up in a complex mess very quickly.

A particularly relevant example can be found in supercomputing, one of the most exciting and fast-moving areas of innovation in the developer community. What is the most exciting thing about supercomputing? It’s heterogeneous, allowing multiple architectures to take on specialized hardware acceleration for the different tasks for which they are best suited. The Aurora supercomputer, for example, is planned to achieve up to 2 exaflops of performance—a staggering number (the first supercomputer I helped design had 1 teraflop—or 2 millionths of Aurora’s performance). The amazing capabilities of these machines are only possible because of advancements in heterogeneous memory technologies, storage capabilities, sophisticated CPUs and GPUs all working together seamlessly.

The advancements we’re seeing in supercomputers, as always, will ultimately filter down into lower-level systems. In the near future, regardless of the type of work you do, enormous computing power is coming online. We need to make sure that our libraries and tools can take full advantage of it when it arrives.

Two great places to learn more about activities to help software be open, multi-vendor and multi-architecture for heterogeneous programming are: https://sycl.tech/ and https://www.oneapi.io. There are numerous online tutorials, a link to the SYCL book (free PDF download) and much more.

February 4, 2022
1.65 Million Devices Enrolled – Techstrong TV

Fleet is an open-source device management platform Mike founded alongside Zach Wasserman, co-creator of OSquery. The company recently raised a $5M seed round. In this interview, we’ll learn more about what Fleet does, how they got 1.65M devices enrolled in less than a year, and why they believe open source is the future. The video and a transcript of the conversation are below.

Recorded Voice: This is Digital Anarchist.

Alan Shimel: Hey everyone welcome to another segment here on Techstrong TV. I’ve got a first-time guest, new company, open-source project I want to introduce you to. Say hello to Michael McNeil. Michael is a founder of Fleet or FleetO for Osquery is some we know. Hey Mike welcome to Techstrong TV.

Mike McNeil: Hey Allan, thanks for having me.

Alan Shimel: My pleasure. So Mike I, I, you know I guess we should jump right into it. Before we even get into who is Mike McNeil, let’s talk about Fleet and, and I guess to talk about Fleet we probably need to start with Osquery and then you know that kind of gave rise to Fleet.

Mike McNeil: Yeah, for sure. So back in ’14 my cofounder, Zach, he and a guy named, Mike Arpaia and a small team at Facebook built this tool called “Osquery.” They were kind of trying to DIY their own security and they started realizing, “Hey it would be really great instead of running all these little scripts all the time, if we could have a standard way to talk to any computer, whether it’s a Mac, it’s Linux, Windows and it would be great if we could do it in SQL.” And that’s where Osquery came from. So it’s kind of a _____ [crosstalk] utility.

Alan Shimel: Yep, but I mean look we’re being modest, it’s had a fair amount of success, yes?

Mike McNeil: Yeah, it’s gotten pretty wide adoption at this stage.

Alan Shimel: I mean can you give us some numbers to just put it in context? Like how many downloads, users, what have you, whatever you want.

Mike McNeil: I’ve seen, so I’ve seen a wide number of stats on Osquery itself, but I can say it is used by a lot of the Fortune 100. It turns out the people that have this problem have a lot of computers, right, it’s a scale.

Alan Shimel: It’s a scaled – you know it’s a problem and it’s at scale.

                                 So now, all right so we’ve got Osquery, now let’s talk about Fleet.

Mike McNeil: So one of the problems with Osquery by itself is you have to go build a server to receive all the data from Osquery. So you can use Osquery to say, “Hey, Linux server” or “Hey, Windows laptop” or “Hey, Mac laptop,” “Hey code machine, tell me some information about you right now.” You can also schedule those queries, right, to run in the background on a certain cadence. But you got to have somewhere to collect that data and send it to where you want it to go. So Fleet is that, it’s an Osquery management server originally and lately it’s become a lot more than that.

Alan Shimel: Okay. Now we’re going to get into what that “lot more” in one second, but while we’re, while we’re doing foundational stuff, Mike, let’s talk about you. Give us a little bit of your background and how you got to be here.

Mike McNeil: Yeah, so let’s see a few years ago I started a open-source framework called “Sails.js,” kind of in the early days of no js back in 2012. Got mixed up in open-source that way. Published my first open-source modules. Then lo and behold in like 2013 Sails.js kind of took off. We started to build a community. Went through Y Combinator in 2015. That’s actually where I met Sid Sijbrandij from GitLab. That’s how – that was actually later on the same folks that ended up coming up with the idea of to build a company around Osquery. So, so we kind of got together and with Sid’s help and with and obviously with Zach’s help we build Fleet. We were able to take this kind of stem of a new open source idea. Apply the open-core business model and then run with it.

Alan Shimel: Excellent. So Sid’s a friend. I mean he’s been on our show many, many times over the years. You know he’s – well since they’ve gone public he’s gotten a little bit more busier I guess. I should mention he’s also an investor in Fleet, right?

Mike McNeil: That’s right.

Alan Shimel: Yep. Okay, so now let’s talk about you know what, what’s Fleet doing and I mean recently it’s, it’s doing more than it sort of original sort of scope if you will, right? We’ve, we’ve added to the mission. And, and Fleet is – excuse me, Sid isn’t the only investor, right? You’ve raised some more money at seed rounds, there’s all kinds of stuff going on. But you tell us Mike what’s happening.

Mike McNeil: Yeah. So we’re up to 22 people now. We just announced the seed rounds led by CRV. We have support from Mike Arpaia, so one of the other creators of Osquery, Nico Waisman, CSO at Lyft, and Jack Naglieri, the CEO of Panther Labs.

Alan Shimel: Very cool. What, what’s, what’s you know beyond the amount of money what’s the money being used for?

Mike McNeil: Yeah, yeah so when we started talking to users of Fleet early on and really trying to like understand what were the bigger problems, so what you’re trying to solve. We started to realize that there were these jobs that needed to be done, right, that kind of just were – they weren’t really satisfactory solutions for it yet out there. So a lot of people were using Fleet to kind of sniff out or like spyglass, right, to see like, “What’s going on on this machine right now?” Maybe it’s for incident response, maybe they’re just trying to kind of like diagnose a weird issue with a server. There’s all these different use cases where device data is useful. But one of the strongest is people needed peace of mind that their MDM is working, that their EDR tool’s working, and that the actual agents are – that they think are installed are actually there. So that’s been one of the biggest use cases we’ve seen driving Fleet so far in the adoption we’d seen last year.

Alan Shimel: Absolutely. What about the go-to-market though, Mike? Like talk to us. How, how you know I mean at some level open source you know once you have an open-source product project like this the go-to-market is, “Hey man give it a try and if you like it you know maybe you’ll want a commercial version or added features or whatever. But here’s our GitHub, you know repo, download it and you know ask the community if you a question.” But is that the deal with Fleet? Talk to us about that.

Mike McNeil: Yeah, so that’s how it all starts, right? And for a lot of users that’s going to be enough. You know we try to make sure that Fleet is enterprise ready. The free version is low. Like we don’t want anyone that needs to use the product to not be able to use the product there, but we are adding additional features and we’ve added some additional features last year around role-based access control that are, that are especially useful for a larger team to – or it makes sense for them to upgrade. That’s kind of been the, been the pay wall. The other thing is support, so if you need a little bit more support or an SLA that’s included with the paid subscription.

Alan Shimel: Got it. What about going forward are there other… So this sounds almost like a little bit of an open-core model, right, where you’re going to get an open version and then there will be you know premium features. What other premium features do you see perhaps coming down the pike here?

Mike McNeil: So a lot of options for that. What’s, what’s currently paid is the team’s feature. So if you want to have multiple, multiple different teams at your organization, like maybe you have one group that manages the laptops, the productivity end points, right, and you got another group managing your production servers, maybe you’ve got another group managing a few of the remaining core servers or maybe some laptops from an acquisition. That segmentation you can do in a Fleet Premium using the team’s feature.

                                 You can also do self-hosted auto updates. So if you want to use – you know a lot of folks are using Chef Ansible, something like that to deploy out the latest version of Osquery to all of their hosts. But if you are someone who would rather have that be taken care of automatically Fleet offers auto updates out-of-the-box, they come from fleetdm.com. If you want to host your own version of that and use the register yourself and kind of decide when the update happens that’s currently a paid feature of Fleet Premium as well.

Alan Shimel: Excellent, very cool stuff. Mike are there any metrics in terms of how many Fleet downloads users or anything? Like give us a sense of, of just how widespread this is now?

Mike McNeil: Yeah, so not everybody has usage analytics turned on. Obviously, you know some folks it makes sense not to do that, but between, between the data that we have and what we know about our customers, kind of a conservative estimate is 1.65 million devices enrolled.

Alan Shimel: Wow, that’s crazy, huh when you think about it from a yeah.

Mike McNeil: Yeah.

Alan Shimel: From just a project you guys started up. That’s great. You know we didn’t mention for people wanting information where, what’s the URL for the Fleet project?

Mike McNeil: Yeah, so it’s fleetdm.com and the D-M stands for “device management.”

Alan Shimel: So it’s fleet device management, fleetdm.com.

Mike McNeil: That’s right.

Alan Shimel: Excellent. And then you probably don’t know this, but what, for someone that wants the open-source version they get it from there as well or do they got to go to like a GitHub or some other repo?

Mike McNeil: Yeah, so there’s a, there’s a few ways to build Fleet. Look if you want to run it locally on your laptop that’s the easiest way to kind of see what it looks like and try it out real quick. We have an easy-to-use installer on the website, you can get going in like five minutes, you’ve just got to have Docker installed.

                                 And if you’re going to production we’re actually we just finished a Terraform in fig, so you can take that deploy straight to AWS quickly. We’re working on one for Google Cloud and Azure now.

Alan Shimel: Yep, that seems to be the trail. Hey man, Mike, I want to thank you for coming onto Techstrong TV and enlightening our audience about Fleet. You know what it’s an open-source world so these are the kinds of projects that may not be household names in everyone’s house, but 1.6 million devices will tell you otherwise, right? Keep up the great work and come back and visit us.

Mike McNeil: Thanks Alan, thanks for having me.

Alan Shimel: My pleasure. Mike McNeil here for Fleet, fleetdm.com if you want to check that out.

                                 We’ll be going to take a break and we’ll be right back.

[End of Audio]

February 3, 2022
Embedded Connectivity Solves the Right (and Left) Problem

In Steven Bellovin’s book, “Thinking Security: Stopping Next Year’s Hackers,” (2015) he wrote:

“Companies are spending a great deal on security, but we read of massive computer-related attacks. Clearly, something is wrong.

The root of the problem is two-fold: We’re protecting (and spending money on protecting) the wrong things, and we’re hurting productivity in the process.

InfoSec is hurting productivity in the organization it is supposed to be protecting.”

Bellovin clarified what he meant by “the wrong things.” He observed that security measures tend to stick around even when the threat they were intended to address has faded into the past. Those security costs have only increased since 2015 when the book was published, and today represent a trillion-dollar drag on the global economy despite the emergence of security-integrated models of software delivery such as DevSecOps, shift left and shift right.

The 2021 IBM Security Threat Intelligence Index reports the No. 1 attack vector was a surveillance technique known as scan-and-exploit. Defenders face an asymmetric engagement with such network threats. The successful attacker needs only to exploit a single vulnerability whereas defenders must secure 100% of the network’s ports of entry 100% of the time. The fundamental problem is the anonymous and permissionless nature of the internet. This makes it both useful and easily abused. Anyone anywhere may conduct reconnaissance and launch attacks at a massive scale with little cost or risk. How can we make it prohibitively difficult or costly to successfully surveil and attack our own applications? What if we could reliably defend against the entire category of active scanning tactics?

There are no secure networks, only isolated networks. Taking this back to first principles, a network’s purpose is to convey packets of data from sender address to recipient address. The network has no built-in ability to identify either party or to discern the legitimacy of any activity. Active scanning techniques identify potential targets that are then probed by scripts that may be intrusive or stealthy or both. We could simply reject all incoming network traffic with a firewall, but application servers must be continually reachable in order to listen for incoming requests. Any interruption of availability causes errors like “connection refused” and “service unavailable” for the user.

This necessary network exposure leaves the upper layers of the network and the application server itself vulnerable to intrusion and denial of service attacks. “Private” enterprise applications are similarly exposed to the famously-vulnerable group that are VPN users’ devices, and so are by no means immune to this problem. This presents the problem of automatically discerning which incoming connections are legitimate, continually, forever. Are we trying to solve the wrong problem?

Ideally, we would identify the sender before they connect to the application server. That way, we can know whether they have permission to connect at all. This is something we simply can not do when the sender is known only by a network address. We can’t ask for a credential because that introduces yet another application layer with all of the same problems. This is precisely why VPNs do not go far enough: A superficial barrier is erected for users that prevents the application server from checking for permission until after surveillance and subsequent attacks have occurred.

Embedded connectivity-as-code solves the right problem. This represents a shift in thinking about the relationship between the application and the network. Now, the application brings its own point-to-point overlay with a verifiable identity on both ends. This has significant implications for the security of the application that can be summarized in terms of shifting security left in the development timeline and shifted right in the life span after deployment.

Shift left: OpenZiti makes embedding connectivity-as-code convenient for developers and transparent for auditors and users. As open source software, it fits into DevSecOps processes that scan the source code. The management side of the OpenZiti network is cloud-native and API-driven and fits into the DevOps paradigm of continuous integration testing. For example, a network-as-code may be continually created and destroyed by a pipeline to exercise any imaginable configuration of endpoints, services and policies, or to facilitate dynamic scanning of the data edge and fabric APIs.

Shift right: OpenZiti allows us to shift right security to measure and monitor applications after development, in production, with a novel ease and without compromising security. Operators maintain perfect visibility of deployed applications and need not cope with the complexity of hopping through security gateways or bastions or worrying about external malicious actors and inbound connections. This best-of-both-worlds reality is a natural effect of an access control scheme that is based upon verifiable cryptographic identity, not a weak identifier like an IP address. Importantly, this means it is always knowable which identity is accessing what services, when they were accessed and to what extent.

OpenZiti gives applications superpowers! To name a few: Identity-driven addressability, private domain name system, shelter from unauthorized connections, portable and programmable micro-segmentation, end-to-end encryption, solves the IPv4 shortage without IPv6, smart routing to choose optimal routes across the internet and is transparent to end users.

Learn more about OpenZiti by checking out our Quickstarts or engaging in conversation on Discourse.

January 31, 2022