Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Home working is ripping the heart out of cities, the DoD’s bizarre cloud strategy, and Apple adds a $10,000 app price option. (more…)
Chainguard today added a private preview of a Chainguard Enforce Signing service, enabled by the open source Sigstore project, that allows developers to generate digital signatures for software artifacts using identities and one-time-use keys they create themselves.
Kim Lewandowski, head of product for Chainguard, said Chainguard Enforce Signing provides an alternative to relying on a public service to generate those digital signatures. It also allows organizations to bring their own key and certificate so that key usage can be monitored and audited per compliance and privacy requirements.
In addition, Chainguard added a library of policies that administrators can deploy directly from the Chainguard Enforce web console to their environments.
The Chainguard Enforce Signing feature is based on Sigstore, an open source project that provides a protocol to simplify cryptographic certificate generation by removing the need to manage keys. At its core, Sigstore creates a set of short-lived ephemeral keys with a trust root based on an auditable public log. Because it creates an immutable log, everyone participating in the software development process, including auditors, can now transparently track the chain of custody for any software artifact.
Key management, key compromise/revocation and the distribution of public keys and artifact digests are challenging to manage. Developers are also required to determine which keys to trust in addition to learning how to sign them when different platforms are employed.
Chainguard’s alternative is enabling organizations to host their own immutable log rather than relying on a public service to eliminate the need to manage keys, noted Lewandowski.
Historically, code signing platforms have been problematic because they were susceptible to tampering. Cybercriminals can, for example, swap out digests to gain access to compromised credentials. Digests and public keys are also often distributed across vulnerable websites susceptible to hacks or inadvertently stored on a public Git repository.
Code signing—along with software bills of materials (SBOMs)—is one of two technological advances that will improve the security of software supply chains. SBOMs, of course, are going to be considered more reliable when the software components they describe are signed by the developers that created them.
Less clear is whether the signing of code will improve its overall quality. Most developers take pride in their code, but the overall quality of the software deployed in production environments could always be improved. Developers may decide to review the code they contribute to a build more intently if they know they will be required to sign it.
In the meantime, it’s only a matter of time before organizations require that developers sign code before it is deployed in a production environment. In addition to increasing confidence in the quality of the code employed, those signatures make it simpler to track down the development team that created code whenever there is an issue that needs to be resolved. After all, the most challenging issue organizations encounter when trying to update software is not so much the creation of a patch as much as it is identifying who should create that patch based on their knowledge of the software components used.
A report published this week by Synopsys in collaboration with the Consortium for Information & Software Quality (CISQ) estimated that software quality issues might adversely impact the U.S. economy to the tune of $2.41 trillion in 2022.
Cybercrime losses due to a rising number of software vulnerabilities alone are on track to increase 42% in 2022 after growing 64% in 2021, according to the report.
More troubling still, the number of failures due to weaknesses in open source software components accelerated by an alarming 650% from 2020 to 2021.
Finally, the report estimated that overall technical debt would increase to $1.52 trillion in 2022.
Dr. Anita D’Amico, vice president of cross-portfolio solutions and strategy for Synopsys Software Integrity Group, said that despite these issues, there is cause for optimism. The silver lining is that more organizations are aware of software supply chain issues in the wake of a series of high-profile breaches.
As more responsibility for application security also is shifted further left toward developers, the number of vulnerabilities being introduced into applications should also begin to decline steadily, she added. In addition, more organizations will soon be following the lead of the federal government by requiring software bills of materials (SBOMs) that will make it simpler to remediate vulnerabilities once they are discovered, noted D’Amico.
Of course, there are still many cultural and technical issues that need to be resolved before DevSecOps best practices are consistently employed across most organizations. However, the expectation is that cybersecurity teams will be able to concentrate more on crafting cybersecurity policies as application development and IT operations teams assume more responsibility for implementing those policies, said D’Amico.
But while giving individual developers the tools they need to write more secure code is a major step in the right direction, there is also a need to make sure the builds to which they contribute code also remain secure. As such, DevOps teams need to implement guardrails across every stage of a DevOps workflow to make sure vulnerabilities are not created as code is merged. In addition, DevOps teams now need to be aware of any malware added to an open source software component downloaded from a compromised repository. Even once patched, there’s a good chance another developer will have downloaded the same vulnerable component to construct a different application.
No one knows for certain how many vulnerabilities may have already been introduced into production environments, but it’s unlikely all those issues will ever be completely addressed. Organizations with limited technical resources will need to prioritize the patching of the most severe vulnerabilities in their application environment based on how accessible code may be to cybercriminals.
Cybercriminals certainly are getting more adept at exploiting vulnerable code. Developers may soon find themselves being required to update software components deployed by someone else many years earlier. In the meantime, more accountability for application security should lead to better quality code that, over time, will reduce the current unsustainable level of technical debt.
Developer experience is where it’s at! Developer experience (DX) means taking a user experience approach to the developer journey. DX (also abbreviated as DevEx) is commonly applied to areas like documentation, sandbox environments, SDK ergonomics and more. By improving a developer’s experience with a tool, software providers can increase stickiness and retain more users.
Nowadays, having a sleek developer experience is a competitive advantage—especially since developers often have the discretion to purchase new tools. And amid an ongoing skills gap, maintaining quality DX for internal software ecosystems is also important to retaining happy engineers and avoiding burnout. All these factors mean developer experience is becoming a C-suite objective. So, where is DX heading? How will the developer workflow change in the near future?
I recently met with Ash Arnwine, director of developer relations, Nylas, to ponder what’s on the near horizon regarding developer experience. According to Arnwine, DX is still a very new concept. Yet, it has become an integral aspect of positioning modern software, especially for public-facing SaaS, and will undoubtedly drive purchasing decisions in the year ahead. Below, we’ll share some predictions of how DX will evolve within four general categories: onboarding, building, deploying, and maintaining.
First off, what is the state of developer experience within the tech world at large? Well, Arnwine is quick to note that the concept is still relatively nascent. “Developer experience was something that we didn’t talk about that long ago.” Yet, the idea is growing in mindshare as developers exercise greater control over the software life cycle.
In fact, the 2022 StackOverflow survey found that 66% of professional developers have at least some influence over their organization’s purchases of new technologies, up from 56% in 2020. Furthermore, the way developers learn is changing. The new generation of programmers is primarily self-taught, relying on coding bootcamps or other online resources. An impressive 70% of developers learned how to code from online resources.
What we end up with, said Arnwine, is a growing developer population with a much wider range of skills than in the past. And since no one is there to hand-hold them, the technology itself must be usable and intuitive—the onus is on the software provider to craft frictionless self-service workflows. And with developers’ growing influence in purchasing, DX becomes a competitive advantage and an executive talking point, said Arnwine.
Developer experience is a broad topic, but it’s most often associated with the time it takes to get started with a tool. This is often referred to as ‘Time to First Hello World.’ But although it’s an important step, the metric might doesn’t always carry much significance. Time to value, on the other hand, said Arnwine, takes this a step further and is a better gauge of success.
For example, onboarding to a web API would likely involve the signup process, creating an account, getting API keys and perusing the documentation—all before a first call is sent. According to Arnwine, there are techniques to apply at this stage to help developers move faster throughout the onboarding process to reach value sooner.
Here are some ways in which developer onboarding is becoming more streamlined:
What can SaaS companies do to help improve DX while building? Well, one way is to enable auto-complete within IDEs. When developers onboard with a new tool, they are often handed an SDK and reference documentation. But, it doesn’t add value to send programmers back and forth between a browser to look up method names. Instead, Arnwine believes developer tools should be “baked into the IDE in a way a modern developer would expect.”
One example of auto-complete on steroids is Fig, a next-gen command line that supports auto-complete for over 500+ popular CLI tools. Taking this a step further is GitHub Copilot, an AI assistant that can take natural language prompts and generate pretty complex code suggestions. The industry appears to be heading toward more AI-fueled low-code software development—a trend that could significantly improve a developer’s experience during the construction process.
However, one potential downside of increased software development automation is ensuring the AI suggests correct code snippets. Thus, Arnwine said he anticipated possible troubleshooting and debugging problems. Furthermore, SaaS providers will have to vet systems that generate glue code to ensure it’s accurate and up-to-date with the latest versions. His advice to SaaS providers is to meet developers where they are, but do their best to maintain a quality experience that they can control.
For a long period of software development history, IT administrators were the only ones able to deploy software. The old-school method required getting a hosting provider, installing Ubuntu and plenty of other legwork. Nowadays, however, software deployment has become far more drag-and-drop. DevOps has become more democratized in recent years, and we will likely see greater usability emerge in the years ahead.
One way to increase developer experience for deployments is through more custom automated routines via GitHub Actions or similar event-based systems. And plug-and-play pre-built actions could reduce the barrier to using such systems. Such connectivity is necessary to enable a docs-as-code approach, said Arnwine.
Another area that could use a serious developer experience upgrade is dependency management. Software dependencies go through a life cycle. They are versioned, updated, retired and sometimes replaced outright. As such, keeping track of an evolving surface area of third-party dependencies can be a headache. To make matters worse, software supply chain threats have complicated the integrity of many open source dependencies consumed by most applications.
Although it’s imperative for organizations to maintain parity with dependencies, a single person is rarely tasked with continuously checking dependencies and monitoring for security threats. Therefore, more automation for dependency management and security monitoring is necessary. For example, Arnwine uses Snyk, which sends a weekly report on the state of his dependencies.
Assuming you have the killer functionality to back it up, an investment into developer experience can help a developer tool stick out in a crowded marketplace. But of course, improving the DX means many different things depending on your context. It could be a subtle user interface tweak, adding relevant code samples or making your documentation more intuitive.
On the other hand, a negative developer experience can turn away potential users and cause frustration throughout the application support life cycle. This can instill a poor reputation among programmers.
To review, in the near future, we’ll likely witness the developer experience evolve in the following areas:
While developer experience is a growing concern, it shouldn’t be the only measurement of a SaaS — the ongoing value it brings developer consumers and end users is just as (if not more) important to consider.
A global survey of 606 IT, security, application development and DevOps decision-makers found that the biggest barrier to adoption of DevSecOps best practices is cultural rather than technical. However, the survey, which polled organizations with more than 500 employees and was conducted by Progress, also found only 16% of respondents planned to prioritize addressing those cultural issues in the next 12 to 18 months.
A full 71% of respondents cited culture as the biggest barrier to DevSecOps progress, with only 30% expressing confidence in the current level of collaboration between security and application development teams currently being achieved. In contrast, 46% of respondents were not particularly confident while nearly a quarter (24%) were not at all confident.
Nevertheless, well over half of respondents (57%) said security threats are the number-one technology factor driving the evolution of DevSecOps, even though just over half (51%) admitted they were only somewhat familiar with how security fit into a DevSecOps workflow.
Overall, 86%of respondents said they are experiencing security challenges that include prioritizing externally facing applications over internal applications (47%), securing different types of workloads that were using different development and delivery methodologies (46%), meeting delivery deadlines (45%) and meeting audit requirements specified by the security team (39%). Just over three quarters (76%) recognized they could be more strategic in managing DevSecOps.
The top three initiatives needed to support a shift to a more strategic DevSecOps approach identified are more investment in continuous learning for developers and engineers (61%), upskilling of developers and engineers to move into site reliability engineering (SRE) roles (60%) and improved communication between developers, security and operations (60%).
Survey respondents also identified a need to define a clear set of policies and procedures (66%), define the role and responsibilities of staff across teams (62%), create a continuous feedback loop (49%) and automate recurring security tasks (41%).
Despite that level of awareness, however, only 40% said they believed implementing security training and upskilling efforts across multiple stakeholders was very important when implementing DevSecOps.
Prashanth Nanjundappa, vice president of product management for Progress, said given all the cultural and technical challenges organizations face, it’s apparent that both a top-down and bottom-up approach to implementing DevSecOps best practices is required. Attempting to shift responsibility left toward application development teams is not going to suffice on its own, he noted.
The survey also suggested that many organizations are trying to address application security issues as they modernize their application portfolio. A total of 59% said they struggled to attain buy-in/funding for refactoring efforts that didn’t provide new user capabilities. However, only 39% said they had adopted a comprehensive modernization approach based on cloud-native architecture principles, while another 22% felt they lacked one entirely. Just under a quarter (24%) said their approach to modernization is to largely rip-and-replace applications.
The top business factor driving the adoption of DevSecOps was a focus on business agility enabled by fast and frequent delivery of application capabilities (59%). Just under half (45%) said it had taken six to 12 months to achieve a return on DevSecOps investments, while 31% said it had taken longer than a year. However, only half (50%) of respondents were familiar and interested in both infrastructure and policy-as-code.
There is no doubt that the DevSecOps journey ahead will be long and arduous. However, there are also a wide range of small process changes that could have a massive impact in terms of improving application security in the short term. The challenge is to quickly identify those opportunities while at the same time employing a more systematic approach that ultimately makes it a lot simpler to implement a set of DevSecOps best practices consistently.
Amazon Web Services (AWS) is making available in preview a C7gn instance on the Amazon Elastic Compute Cloud (Amazon EC2) based on AWS Graviton processors. The C7gn instances provide up to 200 Gbps network bandwidth and as much as 50% higher packet-processing performance compared to previous-generation C6gn instances.
Announced at the AWS re:Invent 2022 conference, these instances are based on Arm processor architecture that AWS provides as an alternative to x86 processors.
Rahul Kulkarni, director of product management for Amazon EC2, said interest in more efficient AWS Graviton processors has spiked sharply as the overall economic outlook softened. More IT teams than ever are being tasked with finding ways to reduce costs as the overall number of applications being deployed in the cloud continues to grow, he added.
While it’s not clear how many applications are being developed to run natively on Arm processors, Kulkarni noted that the process of refactoring existing applications to run on Arm processors is not as intensive for most applications as it was when previously moving applications from one class of processors to another.
Overall, AWS now offers more than 600 types virtual machine instances on its cloud platform, including a set of Inf2 instances, available in preview, based on custom AWS processors that are optimized for processing workloads that include machine learning algorithms that drive artificial intelligence (AI) models.
Every AWS instance takes advantage of a set of AWS Nitro Cards that offload the overhead virtualization creates so IT organizations can take full advantage of compute resources without having to allocate any resources to run virtualization software, noted Kulkarni. That virtualization tax is instead absorbed by AWS to reduce total cloud costs for the customer, he added.
In addition, more customers are taking advantage of tools such as AWS Cost Optimizer and AWS Karpenter for Kubernetes clusters that employ machine learning algorithms to identify opportunities to reduce the cost of cloud infrastructure by, for example, shifting workloads to different classes of services.
There are, of course, multiple strategies for containing cloud costs that range from committing to consuming a specified amount of compute resources annually at discounted rates to relying more on spot instances that are available for limited amounts of time. Less effective is moving workloads from one cloud provider to another simply because the total cost migrating workloads can be substantial depending on the number of proprietary application programming interfaces (APIs) that have been invoked.
Regardless of approach, the days when developers were allowed to invoke cloud resources at will appear to have come to an end. Enterprise IT organizations are a lot more conscious of the total cost of IT in the cloud era. As a result, many of them are adopting financial operations (FinOps) best practices to maximize utilization of cloud infrastructure resources.
It’s not clear how readily IT organizations will embrace AWS Graviton alongside other classes of Arm processors to achieve that goal, but as the economic outlook continues to remain uncertain, there is no doubt all options are now on the table.
DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to serve our audience better.
Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers. Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role, it creates an opportunity for others.
The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale offered.
We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:
Kforce
McLean, VA
Senior DevOps Capability Engineer
$135,000 to $145,000
Dice.com
Emids
Remote
Technical Product Owner – DevOps
$120,000 to $140,000
ZipRecruiter
Progressive Insurance
Carmel, IN.
Azure IT DevOps Engineer Lead
$120,000 to $135,000
Indeed.com
Amazee.io
Remote
DevOps Engineer
$106,000 to $134,000
SimplyHired
Starbucks
Remote
DevOps Engineer II
$94,000 to $166,000
Amazon Web Services (AWS) this week launched a preview release of the Amazon CodeCatalyst framework that leverages blueprints to automatically provision and configure developer environments in its cloud.
Announced at the AWS re:Invent 2022 conference, Amazon CodeCatalyst takes advantage of a Project Blueprints tool to launch a source code repository and a project management tracker that are then used to create build and release pipelines. DevOps teams can use either the tools provided by AWS or substitute an instance of, for example, GitHub or the Jira project management software.
Amazon CTO Werner Vogels told conference attendees the goal is to reduce the friction that developers currently encounter whenever they need to launch a new development project. As part of that effort, Amazon CodeCatalyst promises to make it simpler to consistently replicate development environments using scaffolds in addition to providing dashboards and search capabilities for tracking changes across commits, pull requests and deployments.
Amazon CodeCatalyst uses a devfile to define the configuration of an environment that comes in four resizable instance options made up of two, four, eight or 16 virtual CPUs. The devfile defines and configures all the resources needed to code, test and debug a project. Devfiles, which are added to the source code repository by the selected blueprint, can also be modified.
Collectively, those capabilities make it possible to deploy sample TypeScript, Python, Java, .NET, JavaScript, React, Angular and Vue code running in a few minutes. Developers can work with the AWS Cloud9 integrated development environment (IDE) or use tools such as JetBrains IntelliJ IDEA Ultimate, PyCharm Pro, GoLand and Visual Studio Code on their local machine. Developers, as a result, can simply switch between multiple codebases or collaborate with other developers as needed.
The pipelines can be run on pre-provisioned builds in the cloud or organizations can opt to deploy their own container environments. Developers can incorporate build actions that are built in or provided by AWS partners in addition to incorporating GitHub Actions to compose fully automated pipelines that can be configured using either a visual editor or YAML files. Build and release pipelines enable deployment on Amazon Elastic Container Service (Amazon ECS), AWS Lambda and Amazon Elastic Compute Cloud (Amazon EC2) services.
In general, Vogels advocated that IT teams should build event-driven applications that are loosely coupled to make it simpler to add new capabilities to existing applications running at scale without impacting the entire codebase. To make it simpler to build those types of applications, AWS added an Amazon EventBridge Pipes tool that creates point-to-point integrations between event producers and consumers in a way that eliminates the need to write that code. AWS also made available in preview a graphical AWS Application Composer tool that makes it simpler to design, configure and build serverless applications using infrastructure-as-code (IaC) definitions.
It’s not clear to what degree developers will be embracing event-driven applications. As a programming model, event-driven architectures have been around for decades. The issue has been those applications also have tended to be among the most challenging to build and maintain. AWS is now clearly signaling its intent to make building those applications a lot easier.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week, four things that caught my eye from Amazon Web Services’ re:Invent 2022 conference: SnapStart turbocharges Lambda, Graviton3E ARM HPC SoC, AWS continues hiring, and AWS’s origin story. (more…)
Nobl9 this week announced it is making available a perpetual free tier of its software-as-a-service (SaaS) platform for managing service level objectives (SLOs).
The free tier was announced at the AWS re:Invent 2022 conference and the company also revamped its pricing. Nobl9 Teams Edition (formerly Nobl9 Hydrogen) now includes 50 SLOs, up from 25, for $850/month. Nobl9 Enterprise Edition also has doubled the included SLOs from 75 to 150.
Finally, Nobl9 also launched SLOcademy to help educate IT teams on how best to create and manage SLOs.
The Nobl9 platform simplifies the creation and tracking of SLOs using metrics and observability data collected from more than 24 widely used DevOps tools including Prometheus as well as services provided by Datadog, New Relic, Dynatrace, Google, Amazon Web Services (AWS) and Splunk.
Kit Merker, chief growth officer for Nobl9, said the company’s platform collects signals from logs, metrics and traces to define what normal should look like for any given service. The free tier of the platform has some limitations, but it does provide access to the full-featured platform to enable IT teams to get started managing SLOs.
SLOs, of course, are not a new idea. They have been used as a metric to track the performance of IT services for decades. It’s not clear whether reliance on SLOs has waned over the years or if application environments simply became too complex to track meaningful metrics. However, as applications are increasingly viewed as services, it’s now only a matter of time before SLOs become more widely adopted across a modern application environment. The challenge, of course, is that defining an SLO is a lot easier than maintaining it—especially within today’s highly dynamic cloud-native application environments where services with lots of dependencies tend to come, change and go unexpectedly.
Nobl9 is making a case for an SLO-as-code approach that makes it simpler to track service levels. A recent survey of more than 300 IT managers and executives conducted by Dimensional Research on behalf of Nobl9 found only 29% of respondents had no plans to implement SLOs. A full 94% of respondents that have or plan to implement SLOs intend to map them directly to business operations, with 91% reporting that they expect that effort to improve decision-making. More than 80% also said their organizations are planning to increase the use of SLOs, with 87% indicating SLOs should improve overall microservices performance.
The challenge is that most organizations have limited visibility into their IT environments. The survey, for example, found less than half of respondents (46%) had visibility into all their IT environments. Only 45% and 35% claim to have visibility into containers and microservices, respectively. More than three-quarters (78%) said hybrid clouds make observability more difficult. Ironically, 45% of companies reported they already employed 11 or more observability and monitoring tools. On the plus side, 31% have hired site reliability engineers (SREs) while nearly half (46%) planned to create that role.
It may be a while before most IT teams are organized around the concept of applications as services but as the management of IT continues to evolve it’s now just a matter of time before that approach becomes more the rule than the exception.