devopsy.kinsta.cloud

Category: DevOps and Open Technologies

Editorial Calendar for February 2016

  • GitBOM Tool Automatically Identifies Software Artifact Components

    GitBOM Tool Automatically Identifies Software Artifact Components

    An open source GitBOM tool, discussed at the Open Source Summit Europe conference this week, can automatically track every source code file incorporated into each built artifact.

    Nell Shamrell-Harrington, a principal software engineer for Microsoft, told conference attendees via a video link that the GitBOM tool, based on a compact Artifact Dependency Graph (ADG) technology, can verify which artifacts are being used across any programming languages, environments and packaging formats without any effort made by developers. To do this, GitBOM repurposes the directed acyclic graph made available in Git version control software.

    All that is required is for GitBOM identifier—a unique, content-addressable reference—to be inserted into the artifact at build time. That identifier then creates a Gitoid that can be consumed by ADG, Shamrell-Harrington said. Each Gitoid then views an artifact as a binary large object (BLOB) through which of all the components that went into it, including source files, dependencies and object files can be identified.

    Support is also included for the Software Package Data Exchange 2.3 format for sharing software bill of material (SBOM) information discovered by GitBOM.

    GitBOM can also detect potential vulnerabilities at runtime, regardless of the depth, in every software artifact that might be impacted to make it simpler for IT teams to remediate issues. That approach makes it simpler to find, for example, every instance of the Log4j vulnerability that might exist in an application environment. GitBOM only includes the minimum information required to create a fingerprint to enable efficient runtime scanning for a known-vulnerable artifact.

    The goal is not to replace SBOM tools but rather to make it easier for them to collect data about artifacts in the most frictionless way possible for developers, Shamrell-Harrington said. GitBOM enables any metadata to be linked to a specific set of corresponding software artifacts to make it easier to discover exactly how they were constructed. Artifact IDs can then be extended to produce GitBOM documents that can be consumed by other build tools.

    GitBOM is being shared at a time when the focus on securing software supply chains has never been higher. In the wake of the Log4j vulnerability disclosure, the Biden administration is requiring every federal agency to construct SBOMs to enable developers to find instances of vulnerable software components. Most large enterprise organizations are generally following suit.

    However, collecting SBOMs is only the first part of a bigger challenge. Organizations will then need to acquire tools to analyze SBOMs and provide feedback to application developers. In effect, eventually, organizations will be able to accept or reject software based on the components being used to create it. Armed with those insights, developers then have the option to upgrade software components to bring applications in line with any policy an organization defines.

    It may be a while before SBOMs proliferate across IT organizations, but the immediate issue at hand is simply finding the least-intrusive way possible to create them.

  • Linus Torvalds Signals Support for Memory-Safe Rust Language

    Linus Torvalds Signals Support for Memory-Safe Rust Language

    Linus Torvalds told attendees of the Open Source Summit Europe conference today that he is hopeful the foundational work needed to add support for the Rust programming language will be laid in a forthcoming version 6.1 of the Linux operating system.

    It would take several additional releases of Linux to add full support for Rust, but the ability to more easily build and deploy applications written in Rust will go a long way toward improving the overall state of application security on Linux platforms.

    The Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation, has called for replacing programming languages like Java that are not memory-safe as part of a larger plan to secure open source software supply chains. Rust is a memory-safe language that employs a compiler to track the ownership of values that can be used once and a borrow checker that manages how data is used without relying on traditional garbage collection techniques.

    Earlier this week, the Rust Foundation allied with OpenSSF and JFrog to address other security issues that might arise as applications are developed in Rust.

    It’s not clear just how many developers are adopting Rust, but Torvalds said he personally was looking forward to learning the Rust syntax—using a recently-acquired laptop based on an Arm processor—should the maintainers of Linux approve the effort. However, Torvalds also left open the possibility that adding support for Rust might not be as successful as initially hoped.

    Torvalds also bemoaned the overall state of cybersecurity transparency at the operating system level. Many of the bugs that are encountered at the hardware level cannot be shared with the entire open source community, so any fix to those efforts is not subject to the same level of peer review as the rest of the operating system. The overall situation is improving, but it’s still very painful from a development perspective, he said. Secrecy is counter to the open source ethos, he said.

    As new applications are built using a memory-safe programming language, the overall security posture of an organization will steadily improve. The challenge, of course, is that the number of developers that know how to build applications using those languages is still relatively small. Of course, replacing trillions of lines of code that have already been constructed using a variety of legacy non-memory-safe languages represents a gargantuan task that might never be fully completed—but shifting to a modern programming language is a major step in the right direction when it comes to application security.

    There may even come a day when the way applications are coded is a lot more secure by default than it is today. That may not necessarily resolve every application security issue, but as the programming languages become inherently more secure, the pressure on developers to become cybersecurity experts should ease—assuming, of course, they are willing to learn a new way to write code.

  • Rust Foundation Allies With OpenSSF and JFrog to Secure Code

    Rust Foundation Allies With OpenSSF and JFrog to Secure Code

    The Rust Foundation announced today it is working with the Open Source Security Foundation (OpenSSF) and JFrog to help maintainers secure open source software created using the Rust programming language.

    Rebecca (Bec) Rumbul, executive director at the Rust Foundation, said even though Rust is a memory-safe language, there are still security issues that maintainers need to be able to address. The goal is to provide the resources those maintainers need to better secure their software supply chains.

    As part of that effort, the Rust Foundation is adding a dedicated security team to help maintainers of open source projects built using Rust to better secure their software supply chains. That team is being underwritten by JFrog and the Alpha-Omega Initiative funded by Google and Microsoft which was recently launched by OpenSSF. JFrog last week announced it is joining the Rust Foundation at the Platinum level alongside Amazon Web Services (AWS), Google, Huawei, Meta, Microsoft and Mozilla.

    The first initiative for the security team will be to conduct a security audit and develop threat modeling tools that will provide the foundation for a set of best security practices for maintainers, said Rumbul. The team will also help advocate for those security practices across the Rust landscape, including the Cargo package manager and Crates.io directory projects.

    The OpenSSF has previously called for replacing programming languages like Java that are not memory-safe as part of a larger plan to secure open source software supply chains.

    In general, the Rust Foundation is committed to enabling Rust developers to achieve their scale, security and sustainability goals, said Rumbul. Given the early stages of the Rust programming language, there is no better time to start addressing security issues that application developers can’t ignore, she added.

    In the longer term, The Rust Foundation is also committed to providing developers with Rust security education and is also working to come up with a fair approach to certifying which projects have achieved an appropriate level of cybersecurity maturity, said Rumbul.

    It’s not clear whether developers are abandoning older programming languages in favor of a memory-safe language such as Rust. Rust employs a compiler to track the ownership of values that can be used once and a borrow checker that manages how data is used without having to rely on traditional garbage collection techniques.

    Of course, replacing trillions of lines of code that have already been constructed using a variety of legacy languages is a gargantuan task that might never be fully completed. However, as new applications are built using a memory-safe programming language, the overall security posture of an organization will steadily improve. The challenge, of course, is that the number of developers that know how to build applications using those languages is still relatively small.

    Nevertheless, there may come a day when the way applications are coded is a lot more secure by default than it is today. In the meantime, however, the number of vulnerabilities that exist with application environments is still too numerous to count.

  • DevOps World 2022: Using SBOMs to Secure the Software Supply Chain

    DevOps World 2022: Using SBOMs to Secure the Software Supply Chain

    As supply chain attacks become increasingly prevalent, visibility is emerging as a necessity in cybersecurity. One way to improve visibility and secure your software supply chain is with a software bill of materials (SBOM). 

    An SBOM is a list of all the components within a codebase. By providing insight into open source and third-party components, SBOMs improve overall visibility and are integral to the identification of security or license threats. To properly secure the software supply chain, organizations must have an understanding of all the components in their applications; it is especially important to do so with containerized applications and those built using cloud-native, microservices-based architectures. That’s why SBOMs are becoming a cornerstone of a comprehensive application security strategy. 

    The Need for Visibility

    Gartner’s recent report Innovation Insight for SBOMs emphasizes the need for constant visibility into the software development life cycle and how SBOMs can help organizations and developers achieve that goal. While Gartner recognizes that open source software is a foundational component of modern software development, it also noted that it can be a major contributing factor in software vulnerabilities. 

    “While reusable components and open source software have simplified software development, this simplicity has exposed a critical visibility gap: Organizations are unable to accurately record and summarize the massive volume of software they produce, consume and operate,” the report said. “Without this visibility, software supply chains are vulnerable to the security and licensing compliance risks associated with software components.” 

    To mitigate the associated risks, Gartner’s report not only emphasizes the importance of SBOMs but the need to automate the SBOM creation and validation process. To achieve supply chain security at scale, organizations should automatically generate and verify SBOMs and use that data to constantly evaluate oncoming risks. 

    This ties directly into the continuous integration and continuous deployment (CI/CD) process, which uses SBOM data and assessments to ensure threats don’t slip under the radar and wreak havoc. You can more easily identify security vulnerabilities at any stage in the pipeline and remove them—because an SBOM gives you more granular visibility.

    The importance of SBOMs has even been noted by the U.S. federal government. As stated in president Biden’s Cybersecurity Executive Order 14028, any company that sells software to the federal government will be mandated to provide a complete SBOM. 

    The initiative has also spread to other branches of the government, with the Cybersecurity and Infrastructure Security Agency also emphasizing the significance of SBOMs. 

    “CISA will advance the SBOM work by facilitating community engagement, development, and progress, with a focus on scaling and operationalization, as well as tools, new technologies, and new use cases,” said CISA in a statement on its website.

    SBOM Requirements

    The U.S. Department of Commerce and the National Telecommunications and Information Administration (NTIA) have outlined the minimum elements required. For one, data fields with baseline information about each component are required. Also, automation support is needed, as SBOMs necessitate machine readability for support at scale. Finally, the need for practices and processes was outlined, calling for frequent SBOM generation, including transitive dependencies and widespread availability of this information to those who need it. 

    On many fronts, SBOMs have become non-negotiable throughout the software industry. In order to effectively detect and respond to threats, organizations must take preemptive steps to ensure visibility is maximized, and SBOMs accomplish just that.

    Want to learn more about this and other related topics? Join CloudBees for DevOps World 2022, held in Orlando, Florida, at the World Marriott Center. Use discount code DW22 when registering and get access to all the fantastic offerings, including keynotes, sessions, training and other interactive activities. We can’t wait to see you there for the DevOps Remix!

  • How’s Facebook Work? They Don’t Know! | Cali. Pay Law | NASA RISC-V Launch

    How’s Facebook Work? They Don’t Know! | Cali. Pay Law | NASA RISC-V Launch

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Facebook engineers admit they’ve no idea what Meta stores (or where), California requires job ads quote salaries, and RISC-V will power future NASA spacecraft. (more…)

  • Stable Diffusion Goes Public — and the Internet Freaks Out

    Stable Diffusion Goes Public — and the Internet Freaks Out

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters. (more…)

  • VMware Embraces Graph Technology to Manage Multiple Clouds

    VMware Embraces Graph Technology to Manage Multiple Clouds

    At the VMware Explore 2022 conference today, VMware launched a portfolio of multi-cloud management tools, dubbed VMware Aria, based on a graph-based data store that captures the resources spanning multiple clouds and their relationship to one another.

    Accessed via a VMware Aria Hub platform previously known as Project Ensemble, VMware Aria Graph provides a common definition of applications, resources, roles and accounts in near-real-time across a multi-cloud environment that can consist of millions of nodes.

    VMware CEO Raghu Raghuram told conference attendees the Aria Graph is at the core of an effort to enable IT organizations to move beyond “cloud chaos” to embrace a next-generation “cloud smart” approach to managing distributed computing environments that span multiple platforms.

    Based on an event-driven platform, the VMware Aria Graph aggregates data pulled from public application programming interfaces (APIs) into a data store residing on that cloud. The data store is based on a federated model so there is no need to move data into a central repository. Instead, VMware is making use of GraphQL to launch queries across a federated data model. In addition, IT teams can discover dependencies and analyze historical configuration data to discover the root cause of any issue. All operational data is now aggregated in a way that creates a single source of truth.

    VMware has also provided a set of API services that can be used to integrate the Aria Graph data store with third-party platforms.
    In general, the company is committed to accelerating “the great replatforming” of IT environments spurred by digital business transformation initiatives, said Raghuram. The challenges IT organizations are encountering are roadblocks that involve everything from a lack of skills, the weight of existing enterprise applications and fragmented operations and cybersecurity models, he noted.

    The VMware Aria graph data store will enable the company to automate a wide range of management tasks, Raghuram said. That includes using the right cloud for the right application and governing and accessing them at the lowest cost possible, added Raghuram. The ultimate goal is to enable a unified developer experience that provides a frictionless multi-cloud experience across a consistent set of enterprise infrastructure, he said.

    VMware offerings that leverage the VMware Aria Graph include VMware Aria Guardrails for enforcing IT policies, a VMware Aria Migration tool for automating assessment, planning and execution using the VMware HCX platform and a VMware Aria Business Insights tool that surfaces business insights using machine learning algorithms. The rest of the VMware Aria portfolio is anchored around extensions to existing platforms in the form of VMware Aria Cost powered by CloudHealth, VMware Aria Operations and VMware Aria Automation.

    VMware also announced today that its existing VMware Cross-Cloud Service offering for deploying its stack of software across multiple clouds has been extended to add support for the Microsoft Azure cloud.

    Overall, VMware claims there are now five million developers in its ecosystem that collectively launch 350,000 projects every week. That equates to roughly 35 new applications a minute, according to Raghuram. All told, VMware claims there are 85 million workloads running globally on its platform.

    The bulk of those workloads today still run in on-premises IT environments. In fact, Raghuram noted that cloud platforms still only account for $413 billion in IT spending each year, which is only 10% of the total IT spending that reaches $4.4 trillion each year.

    VMware, which is expecting to complete its acquisition by Broadcom in the next few months, is betting that as more of those workloads move to the cloud, the need to centrally manage heterogenous IT environments will play to its strengths.

  • CloudNativeDay: WASM to Drive Next IT Epoch

    CloudNativeDay: WASM to Drive Next IT Epoch

    WebAssembly (WASM) is poised to drive the next era of development of cloud-native applications that can truly run anywhere.

    Liam Randall, CEO of Cosmonic, a provider of a platform for building and deploying WASM applications, told attendees at a virtual CloudNativeDay event that WASM will be at the core of a new IT epoch that will enable organizations to more quickly build highly portable applications that are not tied to any IT infrastructure platform and more secure.

    WASM, at its core, is a portable binary instruction format for building software that describes a memory-safe, sandboxed execution environment. The World Wide Web Consortium (W3C) drove the development of WASM as part of an effort to create a common format for browsers executing JavaScript code. Wasm is now being extended beyond browsers and JavaScript to enable developers to create a set of universal binaries that could work on any platform without modification.

    That approach replaces the current predominant method for building software that relies on the aggregation of software components that tend to lack distinct boundaries between them. One of the issues of that approach is that it becomes relatively simple for malware to infect all the components of an application.

    The Cloud Native Computing Foundation (CNCF) is now trying to advance the adoption of a Cosmonic’s WASM runtime that makes it possible to run WASM applications on any platform versus being constricted to building applications that can only run, for example, on Windows or Linux.

    That level of portability will prove crucial as more organizations look to drive application deployments on a wide range of edge computing platforms, noted Randall.

    It may be some time before WASM becomes the preferred artifact for building applications, but it’s clear the walls that have separated one IT platform from another are finally starting to crumble. Once applications become much more portable, the longstanding goal of writing an application once and then deploying it anywhere may finally be achieved. That’s not only critical in terms of overall application developer productivity, it has major implications in terms of streamlining the management of DevOps workflows as it becomes easier to both initially deploy applications and continuously update them.

    In the meantime, as organizations continue to review their application development practices to better secure software supply chains, it’s become clearer that legacy approaches to building applications are fundamentally flawed. Rather than attempting to reengineer processes to encourage developers to better secure applications, it may be more effective to change the way applications are built. After all, considering all the processes required to secure legacy approaches to application development, it may be cheaper and easier to adopt a new app development method.

    The challenge, of course, is, as always, taking that first initial step away from the inertia of legacy application development approaches.

  • CREST Defines Quality Verification Standard for AppSec Testing

    CREST Defines Quality Verification Standard for AppSec Testing

    At the Black Hat USA 2022 conference, CREST today shared a quality assurance verification standard to improve application security testing. The standard is based on the open source framework defined by the Open Web Application Security Project (OWASP).

    Tom Brennan, executive director for Americas at CREST, said the OWASP Verification Standard (OVS) measures an organization’s ability to execute and deliver assessments related to Level 1 and Level 2 of the OWASP Application Security Verification Standard (ASVS) and OWASP Mobile Application Security Verification Standard (MASVS).

    The goal is to provide a consistent methodology for accrediting companies that provide application security testing services, said Brennan. Ultimately, organizations of all sizes will then be able to require application software providers to pass an OWASP accreditation test rather than testing the security of every application they use, he noted.

    Organizations need to be accredited through CREST Penetration Testing to apply for the OVS program, which requires companies to demonstrate that they can meet the program requirements and execute and deliver Level 1 and Level 2 ASVS and MASVS services. Level 1 determines whether or not applications store or handle sensitive data; that determination then defines whether or not they need the more rigorous controls of Level 2 or 3. Level 1 controls can be checked either automatically using tools or manually without access to source code. Level 2 ensures that security controls are in place, effective and used within the application. In addition, all organizations will need to ensure that their teams have completed CREST’s Skilled Person Register and have each signed the CREST code of conduct.

    CREST is a nonprofit organization that was originally known as the Council of Registered Ethical Security Testers. CREST works with governments, regulators and multinational organizations to improve application security using CREST OVS Mobile and CREST OVS Apps accredited services. It’s becoming more common for organizations to require OVS to better ensure application security in much the same way they require compliance with frameworks such as the System and Organization Controls (SOC) framework, said Brennan.

    OWASP is administered by a separate nonprofit foundation that works to improve the security of software by sponsoring community-led open-source software projects and training application developers. By aligning with OWASP, CREST creates a verification standard to provide a more structured approach to implementing ASVS and MASVS, noted Brennan.

    OVS, of course, is arriving at a time when there is now a much greater focus on software supply chain security in the wake of a series of high-profile breaches. It is apparent cybercriminals have become more adept at discovering and exploiting a wide range of known software vulnerabilities. The Biden administration has even gone so far as to issue an executive order requiring federal agencies to review the security of their software supply chains.

    It may be a while before application testing using OVS becomes routine, but as more organizations embrace DevSecOps best practices to secure their software supply chain it’s clear there is a greater need for application security testing standards. Especially those that organizations can extend versus reinventing the wheel.

  • Open Standards Are Key For Realizing Observability

    Open Standards Are Key For Realizing Observability

    Observability has quickly become a major focus of enterprise DevOps. Yet tool sprawl and complexity can hold some observability initiatives back. The 2022 CNCF Microsurvey on observability trends found that 23% of organizations use between 10 and 15 tools for monitoring, metrics and gathering logging and tracing data. A full 50% also noted that engineers and teams using multiple tools presented a top challenge to observability.

    I recently met with Dotan Horovits, principal developer advocate for Logz.io, to get his take on improving how modern development teams handle observability. According to Horovits, open standards and open source projects are the principal drivers to realizing interoperable observability industry-wide. Below, we’ll consider the issues facing observability and introduce OpenTelemetry, a CNCF project quickly gaining traction to unify telemetry data from various sources.

    Open Standards For The Win

    Although proprietary monitoring agents have a lot of intelligence, they inherently use tight coupling between telemetry generation and the backend storage analytics, said Horovits. This can cause vendor lock-in and lead to formatting variations in telemetry data between toolsets. “The open standards have the power to converge our industry to prevent vendor lock-in, even the playing field, and allow us to get any telemetry for any source,” said Horovits.

    Without an open standard, supporting the evolving suite of technologies at the heart of modern polyglot software stacks is challenging. Organizations constantly add to their tech stack, from SQL to NoSQL, Kafka, API gateways and serverless cloud services, and these modules produce varying data formatted in unique styles. What’s more, companies are typically using more than one cloud vendor, further necessitating the need for a decoupled method to represent multi-cloud metrics. “As soon as your IT stack becomes this complex, there is a need to aggregate all telemetry into one unified observability standard,” explained Horovits.

    Introducing OpenTelemetry

    OpenTelemetry is described as a “high-quality, ubiquitous and portable telemetry to enable effective observability.” Using OpenTelemetry (sometimes abbreviated as OTel), engineers can generate, collect and export metrics, logs and traces. OpenTelemetry works with many popular libraries and is completely vendor-neutral and supported by a rising number of tools in the space. OpenTelemetry was accepted to CNCF on May 17, 2019, and is an incubating project.

    The OpenTelemetry registry boasts hundreds of libraries and plugins to export and transform telemetry data from many different environments, from legacy to contemporary sources. By decoupling the data collection and export process, OpenTelemetry is “one agent to rule them all,” Horovits explained. Of course, he added, vendors who adopt OpenTelemetry may still provide their own logic behind it.

    The Benefits of More Open Telemetry Data

    At the time of writing, live public repository data shows OpenTelemetry is the second-most active CNCF project, behind Kubernetes. So, why is the observability community so excited about OpenTelemetry? Well, for one, the interoperability it affords could equate to efficiency and time savings. “OpenTelemetry gives you the freedom to integrate throughout the stack,” stated Horovits. Part of this is because its open source roots make it a force multiplier. “You can never achieve such vast coverage with closed source as you would with a wide-open community,” Horovits said.

    Arguably, an open telemetry toolset has also become necessary due to the sheer complexity of cloud-native architectures. Most organizations are shifting to a distributed microservices architecture supported by a dynamic, elastic container-based infrastructure. In this world, a single service might span multiple clusters. Therefore, engineers might need to consider clusters, pods, nodes and namespaces when filtering for data.

    In this complex reality, there are many dimensions, multiplying what needs to be monitored. And in general, traditional monitoring systems become ineffective at this level. Tool sprawl and lack of consolidation are problems that arise when trying to cater to many environments simultaneously. Thus, a unified, decoupled observability platform makes a lot of sense.

    Future of OpenTelemetry

    We’ve seen vast adoption from cloud vendors and solution providers around OpenTelemetry. And, the project has been moving swiftly—OpenTelemetry has had a distributed tracing specification released for about a year. The project almost has a release candidate for metrics and one for logs is in the works, said Horovits.

    Beyond that, one feature that Horovits is excited about is continuous profiling, which could help users continuously analyze a function and track usage trends across instances. Another exciting potential development is eBPF. Since it’s language-independent and runs at the kernel level, eBPF could extract observability signals without requiring application-specific hooks or integrations. This could increase support for more environments.

    For more information, you can visit the OpenTelemetry site to peruse the documentation or read up on the feature roadmap for OpenTelemetry. Horovits also gave this helpful presentation on OpenTelemetry at KubeCon 2022.