Editorial Calendar for February 2016
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: PyPI complies with a “string of subpoenas,” and LTO continues to grow, despite predictions of its demise. (more…)
This week at its Red Hat Summit event, Red Hat introduced an expanded set of capabilities to Red Hat Insights that simplify management of Red Hat Enterprise Linux (RHEL).
Managing Linux across the hybrid cloud is complex, and it’s made even more so because of current macroeconomic conditions and the challenge of finding and retaining highly skilled DevOps and IT professionals, said Gunnar Hellekson, vice president and general manager, Red Hat Enterprise Linux, Red Hat.
The enhanced Insights capabilities add a layer of abstraction to help reduce enterprise Linux complexity across the hybrid cloud and make RHEL management easier by lowering skill level requirements for managing Linux server estates, the company said.
Using a single UI, entry-level systems administrators and IT support teams can access Red Hat Insights’ predictive analytics to identify potential bugs, misconfigurations or security vulnerabilities and remediate them. Systems administrators of all skill levels can detect, assess and push fixes for potential problems without a deep understanding of Linux management systems like Red Hat Satellite Server and without having to interact with the command line, the company said.
These enhancements help organizations maintain services and systems even in uncertain macroeconomic times when hiring is constrained. Red Hat research also claimed that these enhanced capabilities enabled IT organizations to find IT issues up to 90% faster and remediate them nearly 66% faster across the hybrid cloud.
A Red Hat Insights image builder service also helps lower the skills level needed to manage Linux estates by enabling IT teams to build standardized, optimized operating system images, including the creation of their own ‘golden images’ that adhere to organization-specific needs for security and compliance. These images can be deployed across public clouds, hybrid clouds, data centers and the edge using the same console.
As the economic downturn continues, organizations are looking to do more with less and identify ways to make the most of the resources, talent and skillsets they already have. Red Hat is making the case for adding a layer of abstraction to allow less-highly-skilled IT and DevOps team members to contribute to managing Linux estates across hybrid, multi-cloud and edge deployments.
“The demands of managing Linux across the hybrid cloud are stretching the ability of resource-constrained IT departments to keep pace,” said Hellekson in a statement. “CIOs need to be able to extend existing skillsets and lower the skill barriers for overseeing Linux estates, which is exactly what the management capabilities of Red Hat Insights are designed to do.”
At its Red Hat Summit conference today, Red Hat unveiled its Red Hat Developer Hub, a developer portal that provides access to an opinionated set of development tools based on the open source Backstage project.
The portal pulls together a set of curated tools and resources so that developers encounter less friction and avoid the complexity of numerous optional tools that span disparate vendors. The goal is to increase developer productivity while reducing the volume of optional tools and information that must be sifted through to actually deliver production code.
Red Hat Developer Hub aims to streamline the development process and allow organizations to maximize existing resources and developer skills while reducing sprawl, said Chris Wright, CTO and SVP for global engineering at Red Hat. This will help organizations maintain application development velocity and drive a consistent user experience for developers based on tools and skills they’re already familiar with, Wright told attendees.
Rather than building and maintaining their own internal developer portal, Red Hat is making a case for using Developer Hub to deliver a platform approach to addressing the complexity and sprawl DevOps teams face. Red Hat Developer Hub is, in essence, an opinionated and optimized version of Backstage that aims to deliver improved developer experiences across environments, including Kubernetes and container application platforms such as Red Hat OpenShift.
Developer Hub will offer a single pane of glass to view all available developer tools and resources, developer self-service capabilities and security and compliance guardrails for cloud-native application development that comply with enterprise-wide best practices, according to the company.
Additionally, Red Hat Developer Hub includes a set of ‘golden path’ templates—pre-defined, pre-architected and supported templates for further accelerating application development.
The developer hub is based on the Cloud Native Computing Foundation (CNCF) incubating open source project Backstage, originally developed by Spotify and donated to the CNCF in 2020. Backstage is now being advanced by contributions from multiple vendors, including Red Hat.
Backstage has been gaining traction in cloud-native application environments that are especially challenging to develop using multiple tools. There’s a lot more focus on developer productivity, which Backstage addresses by creating a catalog of blueprints that developers can readily consume rather than requiring them to build these capabilities themselves multiple times over. The goal is to create scaffolds that developers can consistently reuse across multiple application development projects.
Red Hat has also developed Red Hat Plug-ins for Backstage, a bundle of six plug-ins that integrate various key systems into Backstage, helping to extend the project’s functionality, ensure a consistent user experience and improve workflow efficiency. The plugins can be used across all Backstage installations, providing even more flexibility for developers across platforms and Kubernetes distributions and include:
Red Hat said additional plugins are in the works. Red Hat Developer Hub will be generally available later this year, but Red Hat is offering a ‘test drive’ through the upstream IDP project Janus.
Red Hat today unveiled a technical preview of Ansible Lightspeed with IBM Watson Code Assistant, a generative artificial intelligence (AI) service specifically trained on domain-specific data collected via the Ansible automation platform.
At the same time, Red Hat is also making available an event-driven capability within version 2.4 of the Ansible platform that enables runbooks and other tasks to automatically execute when received from a third-party tool or application.
Both capabilities were announced at the Red Hat Summit and are part of an ongoing Red Hat effort to democratize IT automation.
First previewed within the Project Wisdom initiative launched by Red Hat and IBM Research last year, Ansible Lightspeed IBM Watson Code Assistant is based on a large language model (LLM) that was developed using Ansible data. Watson Code Assistant, currently in technical preview, is a service that automatically generates suggestions to improve code as developers create an application. IBM promised to make IBM Watson Code Assistant available later this year.
Red Hat is now applying that generative AI capability to the YAML code that DevOps teams need to create Ansible playbooks. IT teams can create a simple natural language prompt that will translate their domain expertise into YAML code for creating or editing Ansible Playbooks. Red Hat and IBM, however, will not automatically use the code from an organization to fine-tune their AI model without express consent.
Rich Henshall, director of product management for Red Hat Ansible, said as IT environments become more complex, it’s clear many are hitting a ceiling in terms of what can be automated using legacy custom scripts. Ansible Lightspeed IBM Watson Code Assistant, along with an event-driven capability, will make it much simpler to automate the management of IT environments at scale using a declarative framework, he added.
Ansible, for example, can now receive events from third-party tools, determine next steps and consistently initiate an automated response using Ansible playbooks. Ansible can also consume events from third-party monitoring, observability and IT analytics tools from providers such as Cisco ThousandEyes, CyberArk, Dynatrace, F5, IBM Instana, IBM Turbonomic, Palo Alto Networks and Zabbix with more to come, noted Henshall. Red Hat is also making available supplementary content for Red Hat OpenShift, Red Hat Insights, Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform and ServiceNow, in addition to enabling IT teams to create their own custom integrations.
At this juncture, it’s not clear why more IT processes have not been automated using Ansible or other frameworks. Many DevOps teams continue to rely on custom scripts that typically don’t scale and are generally poorly documented. Any time the individual who wrote those scripts leaves an organization, it’s incredibly difficult—if not impossible—for the original scripts to be maintained, noted Henshall.
One way or another, it’s clear that a more scalable approach to IT automation is required, especially with the rise of more complex cloud-native application environments. It’s no longer economically viable for most organizations to rely on legacy processes that require a small army of IT professionals to maintain.
Red Hat today announced a portfolio of cloud services designed to better secure software supply chains. The expanded portfolio includes Red Hat Trusted Application Pipeline to secure continuous integration/continuous delivery (CI/CD) workflows and Red Hat Trusted Content offerings to better secure software components.
Announced at the Red Hat Summit, the Red Hat Trusted Software Supply Chain combines a suite of curated open source tools into a cloud service managed by Red Hat. The Red Hat Trusted Application Pipeline service is based on sigstore, an emerging de facto standard for cryptographically signing code to ensure immutability that Red Hat helped drive, while Red Hat Trusted Content provides access to a set of software components that Red Hat has vetted.
Red Hat Trusted Software Supply Chain also provides access to existing tools such as Quay, a registry for containers, and the Advanced Cluster Security (ACS) platform for securing Kubernetes clusters.
Red Hat Trusted Software Supply Chain is scheduled to be made available in preview in the next few weeks, and will also track the provenance of components. In addition, the service will monitor emerging threats to software components and alert users of new and emerging risks. The service will also surface recommendations for remediating vulnerabilities it identifies based on best practices defined by Red Hat.
Finally, Red Hat Trusted Application Pipeline also promises to make it simpler to automate the deployment of secure code with a few clicks within the context of an integrated continuous integration/continuous delivery (CI/CD) pipeline. DevOps team can also import Git repositories and configure container-native continuous build, test and deployment pipelines, inspect source code and transitive dependencies, autogenerate software bill of materials (SBOMs) within builds and verify and promote container images. An enterprise contract policy engine can consistently enforce compliance with the Supply Chain Levels for Software Artifacts (SLSA) framework.
Finding ways to implement DevSecOps workflows is a major challenge for any organization. Red Hat is making a case for a managed cloud service based on a collection of security software and services that a DevOps team would otherwise have to implement themselves.
Tasked with securing increasingly complex software development environments, it’s not clear whether DevOps teams will opt to rely on a cloud service, but given the general lack of security expertise among software development teams, the fastest way to secure a software supply chain for most organizations is to rely on a service.
Regardless of approach, of course, DevSecOps will soon become a standard element of application development workflows. Once that occurs, it’s not clear whether DevOps will simply become DevSecOps or whether security will be assumed to be an integrated element of any set of DevOps best practices.
Sarwar Raza, vice president and general manager for cloud services at Red Hat, said regardless of approach, it’s clear that implicit trust is no longer good enough. Organizations of all sizes will need to be able to demonstrate that they have maintained a chain of custody for the software components used to build an application, he noted.
One way or another, the overall security of downstream applications should eventually improve as the processes used to construct software are increasingly locked down.
The software supply chain has become increasingly complex and dynamic with the rise of cloud computing, open source software and third-party software components and APIs. Widespread damage can occur if third-party APIs, cloud services, SDKs and open source software have security flaws. As a result, software supply chain security has emerged as a critical concern for organizations across industries. Supply chain attacks have become more frequent, sophisticated and devastating, as evidenced by high-profile incidents such as SUNBURST, Kaseya and Apache Log4j.
To address the growing challenge of software supply chain security, organizations have turned to leveraging software bills of materials (SBOMs). SBOMs are a standardized inventory of software components used in a particular product or system, including their versions, dependencies and sources. SBOMs provide transparency and visibility into the software supply chain. They can be an effective approach for identifying and mitigating security risks, compliance issues and operational challenges – assuming organizations have the right tools to fully benefit from SBOMs, including runtime discovery, in place.
SBOMs are not a new concept, but their adoption has accelerated in recent years due to a number of important factors, including the following.
First, regulatory agencies, such as the U.S. National Institute of Standards and Technology (NIST) and the Cybersecurity and Infrastructure Security Agency (CISA), have been advocating for SBOMs as a best practice for software supply chain security. NIST has released a draft guidance document on SBOMs, and CISA has mandated SBOMs for federal contractors and subcontractors.
Second, industry initiatives, such as the Open Source Security Foundation (OpenSSF) and the Software Assurance Forum for Excellence in Code (SAFECode), have promoted SBOMs as a key element of software security. OpenSSF has launched a working group on SBOMs and published the Open Source Software Security Mobilization Plan to improve the resiliency and security of open source software (OSS), including guidelines for SBOM management; SAFECode published a whitepaper on SBOMs.
Third, supply chain attacks have highlighted the need for SBOMs as a proactive and preventive measure. SBOMs can help organizations detect and respond to vulnerabilities, malicious code and unauthorized changes in the software supply chain. SBOMs can also support incident response and recovery efforts by providing accurate and comprehensive information about the affected software components.
Fourth, customer demands and market pressures have influenced the adoption of SBOMs. Customers are increasingly aware of software supply chain risks and are asking for SBOMs as part of their procurement and evaluation processes. Some industries, such as health care and automotive, have mandated SBOMs for safety and liability reasons. SBOMs can also enhance brand reputation, customer trust and competitive advantage for software vendors.
Fifth, technological advancements have facilitated the creation and consumption of SBOMs. The emergence of open standards, such as CycloneDX and SPDX, has enabled interoperability and integration of SBOMs across different tools and platforms. The development of automated tools, such as software composition analysis (SCA) and vulnerability scanners, has enabled the generation and validation of SBOMs in a timely and accurate manner. The integration of SBOMs with other security measures, such as identity and access management (IAM) and security information and event management (SIEM), has enabled an effective view of software supply chain security.
However, SBOMs are not a silver bullet for software supply chain security. They are only as good as the data they contain, and the quality of data can vary depending on the source and the method of collection, particularly around the application software stack of APIs, cloud services and SDKs. SBOM inventory is constantly changing, and being able to leverage their up-to-date data requires more than a snapshot from traditional source code static analysis approaches, but rather continuous runtime analysis and dynamic inventory.
For example, common vendor management and software composition analysis (SCA) approaches often lack source code access for mobile, web, cloud and commercial-off-the-shelf (COTS) software, as well as third-party API services.
For a more effective approach, organizations can benefit from a full-stack attack surface management (ASM) software supply chain solution that delivers continuous third-party application asset discovery and dynamic tracking of third-party vendors. These newer approaches automatically categorize assets under known vendors, allow customers to add additional new vendors, curate individual assets under any vendor, and alert on increases in policy violations and high embed rates of third-party vendors within key applications. These automated capabilities allow vendor management teams to remedy supply chain security problems more quickly and easily and much more accurately than older approaches.
It is important to note that SBOMs also require collaboration and communication among multiple stakeholders, including software vendors, suppliers, customers and regulators. They may also pose some privacy and intellectual property concerns since they can reveal information about the software components and their relationships.
Overall, organizations need to adopt a comprehensive and risk-based approach to software supply chain security, of which SBOMs and tools to best leverage their information are an important component. Organizations should establish policies and procedures for SBOM creation, management and sharing, as well as for validation and verification. Organizations should also integrate SBOMs with other security measures and practices, such as threat modeling, penetration testing and code review to best ensure the security of their software supply chains.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Sources say it’s an open source future for OpenAI, and improving AI perf with Mojo. (more…)
A survey of 418 hiring managers published this week by the Linux Foundation found 59% of respondents changed or revised their technical hiring plans for 2023 because of economic uncertainty. Among those organizations, the primary strategy for addressing these concerns was to freeze hiring (34%), followed by a focus on hiring to address specific skills requirements (32%) and staff reductions (13%).
Conducted by Linux Foundation Training & Certification and Linux Foundation Research to coincide with the Open Source Summit North America conference, the survey found 30% of respondents worked for organizations that froze hiring, while another 8% reduced headcount. Another 6% opted to do both, the survey found.
In contrast, a total of 44% said they planned to increase their hiring of technical talent, with another 12% reporting that they would still be hiring even if they had frozen some hiring or reduced headcount.
The survey revealed that organizations are looking to add technical staff with experience in cloud/container technologies (50%), cybersecurity (50%), artificial intelligence/machine learning (46%), database and data management (37%) and advanced analytics and data science (37%). The most acute need for technical expertise is in the areas of augmented reality/virtual reality, AI/ML, blockchain, continuous integration/continuous delivery (CI/CD) and DevOps and Kubernetes, the report found.
Anna Hermansen, ecosystem manager for Linux Foundation Research, said in addition to looking to recruit talent, it’s also apparent more organizations are investing in training to fill gaps, with 70% of respondents reporting they invest in training whenever a new technology is introduced. Investments in upskilling often enable organizations to add critical capabilities faster than finding and then vetting potential candidates, she noted. Well over half of respondents (58%) worked for organizations that sought training for their existing employees when they were unable to find suitable technical candidates, the survey found. Only 38% of respondents said they would rely on consultants to fill those gaps.
Upskilling, however, is not a panacea. While 55% of organizations agreed that upskilling could help, there is often a need to backfill positions and 47% of respondents said upskilling takes too long or is ineffective at training for complex roles.
In comparison, 48% or respondents said it would take more than three months to fill a technical position, with nearly two-thirds (64%) also noting recruitment is costly and time-consuming and often does not lead to the right candidate. A full 84% agreed there is a need for a streamlined way to recruit new technical staff with proven skills and 64% said that onboarding takes valuable internal resources away from other critical projects for too long.
The survey also found that 38% of organizations lost up to 10% of their new hires within four to six months, with nearly one out of three new hires, on average, departing within six months of being onboarded.
There’s no doubt that an uncertain economic outlook is adding more stress to already-flawed recruitment processes. The challenge and the opportunity is determining whether to focus on recruiting top talent versus growing a farm team that will be able to step up to handle more complex tasks. It’s not a question of choosing between those approaches as much as it is striking the right balance between them.
Spotify this week added additional plugins for its open source Backstage platform that is used to build developer portals. The new plugins make it simpler to address role-based access and access Insights, a tool from Spotify that tracks Backstage usage trends.
In addition, Spotify is also enhancing a Soundcheck plugin for Backstage that is used to visualize and track development of software components. Forthcoming capabilities include a no-code interface that makes it possible to programmatically create checks of code without writing any code.
DevOps teams will also be able to view, export and understand how teams and components are doing compared to established best practices and monitor trends, graphs and historical views, and receive notifications when levels change.
Finally, Spotify is committing to adding additional integrations with third-party tools and platforms such as Snyk, Sonarqube and the open source Argo continuous delivery (CD) platform.
There are now five plugins available via a Spotify Plugins for Backstage subscription service, and Spotify said more are planned.
Meg Watson, a group product manager at Spotify, said Backstage has been gaining traction in cloud-native application environments that are especially challenging to develop using multiple tools. Because of Backstage, developers are not only more productive but turnover has also been reduced because developers are provided with a better experience, she noted.
There’s a lot more focus than ever on developer productivity, which Backstage addresses by creating a catalog of blueprints that developers can readily consume rather than requiring them to build these capabilities themselves multiple times over. The goal is to create scaffolds that developers can consistently reuse across multiple application development projects.
Backstage was donated to the Cloud Native Computing Foundation (CNCF) and is now being advanced by contributions from multiple vendors. Much of that focus is on lowering the barrier of adoption for the platform, said Watson. Specifically, Spotify is working on a QuickStart for Backstage edition of the platform that is simpler to install, she said.
In general, Spotify is trying to strike a balance between centralizing the management of DevOps workflows and the need to enable developers to define workflows that are natural to them versus ones that have been imposed on them, added Watson.
It’s not clear to what degree Backstage will help fuel the adoption of a shift toward platform engineering to centralize the management of DevOps tools and platforms. The concept of a portal through which developers can self-service their own needs may not be new, but as an open source project, Backstage has made it simpler for many organizations to achieve that goal.
Just about every DevOps team is now being asked to find ways to help improve developer productivity at a time when many organizations are trying to do more with fewer resources. Regardless of the motivation, however, it’s apparent that DevOps workflows are continuing to evolve and mature.
In the meantime, DevOps teams would be well-advised to eliminate as many bottlenecks as possible before they negatively impact developer productivity.
At the Open Source Summit North America conference today, Amazon Web Services (AWS) announced it is making Cedar, a language for defining permissions as policies that includes automated reasoning to mathematically prove an IT environment is secure, available as an open source project.
In addition, AWS launched SnapChange, an open source fuzz testing tool that cybersecurity researchers can employ to discover vulnerabilities by replaying physical memory snapshots in a KVM virtual machine. Fuzz testing uncovers software security issues by monitoring how the system behaves while processing random data.
David Nalley, director of open source marketing for AWS, said both projects are part of an ongoing AWS effort to contribute intellectual property to the open source community that plays a pivotal role in the applications that organizations deploy on the AWS cloud.
AWS has been employing Cedar to provide IT organizations with the ability to write authorization policies as code that can be deployed anywhere. It is most widely used within the managed AWS Verified Permissions service that manages those processes on behalf of organizations that deploy workloads on its cloud.
Cedar differs from other policy-as-code approaches because it employs automated reasoning, a computer science discipline that is considered a subset of artificial intelligence (AI), to enable machines to prove theorems and checks. That latter capability is especially critical to enable organizations to mathematically prove that compliance mandates have been continuously met, noted Nalley.
Otherwise, IT organizations become wholly dependent on an assessment of intermittent audits conducted by humans that are likely to arrive at varying subjective conclusions, he noted.
The latest open source contributions from AWS come at a time when there is more scrutiny being applied to how open source software is developed and employed. The scrutiny increased in the wake of a series of vulnerability disclosures involving open source software, such as the Log4j monitoring software project.
The maintainers of many open source software projects often don’t have the resources or capabilities required to immediately patch a vulnerability every time it is discovered. That’s become especially problematic, because more research into open source software vulnerabilities is being conducted; it’s only a matter of time before a vulnerability is detected in software that was largely created by developers that have limited cybersecurity expertise. Enterprises that consume open source software need to pay careful attention to what dependencies they are creating when that software gets incorporated into an application environment. A patch for a zero-day vulnerability that is discovered might not be readily available.
Hopefully, consumers of open source software will become more involved in its development by contributing code or making available the financial resources that maintainers need to devote time and effort to creating a timely patch to a vulnerability. In the meantime, however, they should also be reviewing what open source software is included in applications running in production environments. Cybercriminals are increasingly looking to discover and exploit vulnerabilities in not just existing applications but not also in components they know are likely to be employed across a wide range of downstream applications.
As such, finding ways to apply more cybersecurity expertise to application development by embracing DevSecOps practices has become an imperative rather than a recommendation.