Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Twitter is in a “non-redundant state” thanks to a hot summer, AI is likely to eliminate us, and Patreon fires 80 staff amid nasty allegations (TW: Child safety). (more…)
Developers spend much of their day digging into their systems’ guts, allocating storage to an application, connecting the security system to the user interface and tracking which version of the application is where in the development cycle, among other tasks.
That’s why a new category of software has emerged, infrastructure-as-code (IaC), that promises to automate system infrastructure provisioning and integration. It frees up time so developers can focus more on frontend solutions rather than backend configuration management. First, though, let’s find out what IaC is and how it helps your organization.
IaC reduces system infrastructure creation and configuration time through the use of sets of tools, protocols, languages and processes. In conventional data center system management, each configuration entry and change has historically required manual action. Now, technology teams can manage everything in an intelligent, automated manner.
With IaC, infrastructure configuration management data is stored within standardized files. The tools monitor and maintain the state of the infrastructure and ensure that the right resources are available whenever an application runs. This saves DevOps teams from excessive manual effort. In essence, large-scale configuration and management become dramatically simplified.
Currently, development is a cumbersome process because friction arises in many places. A great deal of progress has been made in speeding up the front end of application development—building the feature set. Still, the process slows down on the back end when the software components are tied to the computer infrastructure.
Historically, software was deployed as one large, monolithic block of code. Updates were tedious to make because all of the application’s code had to run, even if it wasn’t involved in the update or bug fix. The advent of containers and microservices enabled developers to design applications that are modular in nature and therefore easier to update, add to or debug.
However, the application still has to be tied into the existing compute infrastructure. Cloud solutions—like AWS, Google Cloud Platform or Microsoft Azure, for instance—provide companies with quick access to large volumes of computing processing power. But that application software must be told how to access the network, present security information, allocate storage and run code on central servers.
Because of this, developers spend a lot of time building and maintaining infrastructure-as-code. To do so, they use languages like Python and solutions such as Chef and GitHub. They want to use automation to simplify and accelerate the process, but here, again, they encounter some roadblocks.
As applications become larger and components become smaller, developers find themselves mixing and matching software from a wide variety of places. Tracking what’s where in the process is a frequent challenge. Current workflows are often non-intuitive and rely heavily on pull requests. Some solutions try to address the problem with multiple workspaces, but the end result is fragile and nondeterministic code.
Without mapping of projects to branches or tags, someone who comments on an approved pull request can push unapproved code to production. In today’s fast-paced DevOps work environment, an approval meant for a short-lived test run becomes the current release, creating a ripple effect of unintended consequences that create confusion and defective code, slowing development—often substantially.
Most tools do not offer access control models. Developers can track a single repository and a single Terraform project, for example, but as the number of repositories and projects grows, ensuring that proper access control features are in place becomes more daunting.
Legacy waterfall and static development projects allowed IT teams to set their system infrastructure up once and forget about it. But in a dynamic environment, microservices come and go in the blink of an eye. New environments proliferate and product teams must continually configure their own workspaces, which eats up time and resources.
Generic IaC platforms don’t provide mechanisms to detect if the infrastructure is drifting. Because so much configuration is done at runtime, differences arise between the desired and the actual state of the infrastructure.
Drift has many causes, such as developer or machine script contingencies or the lack of system resources. Too much drift results in unstable applications. Catching such problems early in the development cycle is more efficient than trying to fix a production system. But needed visibility is often lacking, and shortcomings become clear only when the system is fully in use.
Given today’s dynamic environment, developers need clear visibility. They must be able to closely examine what happens as their job runs. Though they offer many advantages, CI/CD platforms deliver little insight into resource usage from a real-time or historical perspective.
So, whenever performance problems arise, these platforms offer little insight into what the source of the problem is or how to fix it. Consequently, developers’ attention shifts from application enhancements to troubleshooting.
Legacy infrastructure failed to let teams reuse tried-and-true system images across multiple environments. To address this problem, teams often create a new image for each version of each application, slowing development time.
Setting up each IT environment manually requires time and manpower, which raises costs. Companies must have dedicated teams of engineers responsible for creating the infrastructure. But they also need supervisors and data center resources, which ultimately boost technology expenses. IaC tools centrally manage the process and create the right environment. That way, organizations pay only for system resources that they truly need.
Legacy IT practices are expensive, slow and inconsistent because they rely heavily on manual input. Infrastructure-as-code automates much of the infrastructure management process, which delivers a range of benefits.
With IaC, provisioning and management are lightyears faster and more efficient than with a manual system. The environment’s infrastructure is defined in code. That code-based infrastructure significantly accelerates each phase of the software development and delivery cycle. Consequently, DevOps teams deliver applications faster and at scale while improving software quality.
IaC deploys consistent configurations constantly and with no discrepancies. Because of this, it enables fast delivery of consistent infrastructure and environments. This reduces errors and overhead while boosting productivity.
Infrastructure automation helps to reduce the number of people with hands on the routing parts of the software development process. That’s because human resources aren’t needed in the management of networking and storage. Companies free up personnel to work on more strategic initiatives.
IaC helps to reduce risks and avoid human errors. It also protects against runtime issues and security weaknesses caused by missing dependencies or configuration drift.
Developers version IaC configuration files the same way they do any other source code file. This gives them complete traceability of any changes. In this way, IaC ensures efficient, exceptionally transparent workflows and speeds up the development process.
Maintaining legacy infrastructure was a time-consuming, complex process. IaC is emerging as a solution that alleviates much of the stress and pain that developer teams feel daily. Now, they can speed up the deployment and connection of system resources to applications. Developers can become more efficient and enable the company to respond to business demands in a faster, more reliable and more cost-effective manner. The development process no longer slows down the business.
There was a time when pundits said that DevOps was the symptom and CI/CD the cure. While that may or may not be true, there is no denying that CI/CD is the very heart of DevOps. While that might still be true, DevOps today is so much more than CI/CD; it has evolved, far beyond just its core of CI/CD.
In my preparations for the upcoming DevOps World 2022, held in Orlando, Florida, September 27 to 29, I’ve been briefed on upcoming product announcements by the CloudBees product team. I find it fascinating to see where the CloudBees product line is going. In many ways, CloudBees product evolution mirrors the evolution of DevOps itself.
Today, DevOps includes areas such as observability, security and compliance, GitOps, cloud-native and more. Developing, integrating, testing and deploying software is only one aspect of today’s DevOps. Without divulging any specific product announcements that CloudBees will be making at DevOps World, I can tell you that CloudBees is on a journey that closely mirrors that of DevOps itself.
In my opinion, this is as it should be. CloudBees was one of the pioneers of the DevOps movement. By harnessing Jenkins—the original DevOps killer app—CloudBees helped move DevOps into the mainstream. Of course, Kohsuke Kawaguchi (KK), the creator of Jenkins, has moved on from CloudBees and the Jenkins community was always independent of CloudBees. Nevertheless, many of us in the field see CloudBees and Jenkins as inextricably linked.
While Jenkins is still widely deployed and used, it is no longer the epicenter of the DevOps universe. Over the last couple years, CloudBees has transitioned to become much more than just Jenkins or Enterprise Jenkins and remains at the center of DevOps innovation.
This year‘s DevOps world will be a great opportunity for CloudBees to showcase just how far they’ve come from being solely the enterprise Jenkins company while remaining a preeminent end-to-end DevOps tools provider. If you want to know just how far, well, stay tuned for our coverage of DevOps World where we will be covering all of the product announcements as well as speaking to CloudBees executives and customers about the new products and product lines and capabilities that will be announced.
I have written before that the theme for DevOps World this year is DevOps Remix. Our own DevOps Experience virtual event, coming in November, has a theme of DevOps Everywhere. I think both of these themes speak to the same general idea: DevOps is going where it’s never gone before. It has evolved beyond CI/CD to a new way of developing, delivering and managing software that includes securing it and testing it. Over the next few years, I think we’ll see we’re entering a new golden age of DevOps.
If you can, join us in Orlando for DevOps World. Techstrong.TV will be broadcasting live from the event; if you can’t make it, we’ll keep you updated on all of the developments announced there. Also, join us in November for our own DevOps Experience virtual event, which will highlight DevOps Everywhere: The Edge and Beyond. Hope to see you soon!
DevOps creates value through constant deployment changes. When done right, more frequent deployments allow teams to continuously improve their products or services while making it easier to manage the effects of the change. One way to do this is with continuous integration and continuous deployment (CI/CD). CI/CD helps to reduce the risks to each build and allows teams to release valuable features quickly and safely. But CI/CD alone is not sufficient for understanding change and service impact in production. The recent DEJ State of IT Performance Report found that change is the largest source of production issues. A full 76% of all performance problems can eventually be traced back to changes in the environment but 67% of organizations don’t have the ability to identify changes in their environment that caused performance issues.
In a modern IT environment, different teams are constantly integrating new features and new technologies and renewing their stack. With many moving parts, it’s impossible to mimic the production environment when testing new features in pre-production. By the time you’ve implemented and tested new code, your branch will be different from what is in production.
Here’s the reality we’re working in:
● Integrating testing in CI/CD pipelines will not keep bugs out of production. Some organizations don’t do true CI/CD, meaning some processes are not even tested and automated in the pipeline.
● User configuration errors are common. Here’s a consolidated list of configuration errors that brought down production.
● Change anxiety is real: Ops folks get nervous when hitting the deploy button, even for small changes.
Changes are inevitable in DevOps, but organizations can protect themselves from a performance incident or an outage by integrating change in production safely with a few simple steps.
Today, data is kept in silos, which is why it’s time-consuming and challenging to make associations and leverage them to find the root causes of performance issues. DevOps teams need to connect their data in meaningful ways to troubleshoot complex issues in distributed systems. Building the “connections” foundation is critical to understanding the state of a system and how the pieces interconnect and interact.
How well an application performs in production is related to a combination of code, infrastructure and operations. For instance, when an incident occurs, teams should quickly know if that issue is related to code, infrastructure or operational activity. Understanding these connections and correlating the data cannot be performed manually. We have hit the wall in linking together data silos; we just don’t scale as software does. Teams need to consider a monitoring and analytics approach that connects operational data together from the start and models the causal links between the data for real-time root cause diagnosis.
How well can your DevOps team identify the key changes made to your production environment? If your answer is “not very well,” then you’re missing critical data that helps to speed up troubleshooting. The lack of visibility into changes and how they impact production makes it impossible for operations staff to identify the cause—the very thing that SREs care most about.
Changes are made to the code or the cloud infrastructure, either directly or through infrastructure-as-code (IaC) templates, for a variety of reasons. The risk of misconfiguration and performance issues increases when these operational activities are not continuously monitored and correlated across the stack.
Metrics don’t tell the whole story. Logs don’t pinpoint where to act. Simply collecting these data types leaves many companies struggling with knowing what is important and what’s not when it comes to real-time troubleshooting.
Having a shared “system of record” for changes for all the different streams of activities increases visibility and control of all changes in real-time. Teams can see pre-incidents and be aware of increased risks when changes are made, especially during high-velocity change periods. By seeing how a code change can impact things down the line, developers can proactively troubleshoot and debug code.
In the same DEJ report, organizations shared that 66% of MTTR is spent on the ‘diagnose’ stage. This correlates to the 67% of teams that are unable to identify the changes in their environment that caused performance issues.
With a shared “system of record” for changes, DevOps teams can troubleshoot incidents in real-time, dramatically cutting the time spent on the “diagnose” stage of MTTR. This is especially effective when teams can see all the changes and other events that are happening to the system in a single, searchable/filterable real-time feed; this timeline would provide the potential causes of issues.
The best tool would allow you to start from an effect and work backward, chaining multiple events overlaid with observability data to determine the causes quickly. Building the causal chain from the end helps teams hone in on the connected change and answer each “why” in a straightforward way. This capability makes it easier and faster to prioritize relevant changes and resolve performance degradations or outages with actionable context.
During diagnosis and resolution, it’s important to see what’s being done without having to ask on Slack, wait for each other or unknowingly interfere with remediation actions. The best option would be leveraging the same timeline to capture all the context of an incident at the moment it occurs, and at any moment in time during diagnosis and resolution, to avoid reverting each other’s changes.
While an incident postmortem brings teams together to gather data and discuss the details and learnings from it, this process is often inefficient and time-consuming. Each team member involved in the incident response effort would have to recollect what they observed, what actions they took and what they communicated to other team members. What if a system could capture all the actions and communications during an incident response effort? Wouldn’t it be nice to just summarize your thoughts in one sentence and be done with the reporting?
When preparing for planned and unplanned changes that can impact the bottom line, DevOps teams need to consider these steps to gain true observability of their operations. Focus on finding the connections in the data to explain cause and effect. Understand change impact —as those changes are being made —to avoid outages. Capture and communicate activities as seamlessly as possible and identify and capitalize on opportunities to automate the workflow.
Establishing clear communications and expectations across software organizations is essential to the success of any project. A company is less likely to experience unexpected delays or other unpleasant surprises if everyone is on the same page. To ensure all team members are synchronized and to eliminate unnecessary ambiguity, engineering teams should create a “Definition of Done” to guide projects.
In Agile software development, the definition of done is a working agreement between teams that defines a set of standards to identify something as “done.” Because Agile is an iterative approach to software development, sometimes it is unclear when a task is complete. Product managers often report that something is ready for delivery when it’s “Dev done” and not “QA done.” In other words, the code has been written, but no documentation or tests have been created or executed. For other parts of the software organization, code without QA or documentation is not “done.” Intuitively, the definition of done exists to eliminate confusion about what “done” means across an organization.
For some software organizations, the definition of done is a static set of guidelines and benchmarks established at the start of a project. For other organizations, including here at Allstacks, engineers have the latitude to amend the definition of done as they go. That approach empowers an engineering team to make decisions that will produce the best possible results.
When establishing a definition of done within an engineering organization, consider these tips:
● Communicate criteria in a way that the whole software organization can universally understand.
● Ensure that the definition of done addresses both tactical and technical requirements.
● Create a living document for alignment that can be amended because of unforeseen circumstances, like losing a team member or a significant market shift.
● Consider that the robustness of the document will vary by team maturity, and specific teams might need an addendum for special requirements. For example, bigger enterprise IT teams frequently need more detailed documentation compared to two-person shops.
● Establish an understanding among all teams that, until all criteria are met, the user story, task or project is not “done.”
A definition of done acknowledges the shared responsibility between the frontend, backend and QA. Beyond those roles, other stakeholders might contribute to defining “done,” like Scrum masters, board members or business analysts, depending on the organization and its structure. Further, the product owner may also define “acceptance criteria” with goals targeted toward the end-user experience that the engineering team must also address as part of their work.
The definition of done helps a software organization maintain alignment on projects and keeps everyone on the same page. Misalignment can have heavy consequences. For example, it can initiate premature conversations with customers before a product is really ready, create friction between internal stakeholders and/or risk other go-to-market initiatives. When the definition of done encompasses defined steps for a project down to the task level, Agile teams can create more cohesion across an organization and eliminate unforeseen delays.
A definition of done outlines each task in a software engineering project. Noting the completion of each task can be an indicator of whether a project is on track to meet established deadlines. By noting the time it takes to complete each task, an engineering team can measure the velocity of a project, and that can help protect due dates or raise a warning flag about the need to shift timelines.
Objectives and key results (OKRs) are a goal-setting framework to help senior leadership focus on what matters to the organization. By defining software engineering OKRs, senior developer leaders can keep their teams focused on aligning engineering goals and activities to business outcomes while also giving them room to experiment.
Creating engineering OKRs involves taking the organization’s objectives and crafting actionable, generalized engineering objectives while identifying the metrics and milestones that drive progress. Translating the organization’s objectives for the engineering team helps them to understand exactly how their work impacts the company.
One of the key benefits of the OKR framework is giving the engineering team a shared language and vision for success. However, without an established definition of done, OKRs will have a lot of unintentional gaps (and teams will have a lot of misunderstandings). Start with a definition of done to ensure that each team knows what their part is and when it’s officially ready for handoff to another team. Teams will pass the baton smoothly across the organization and there will be fewer obstacles in the race toward release in the market.
In a small startup with a handful of software employees, establishing a shared understanding of timelines and goals is easy. As software organizations grow, casual chats at the water cooler aren’t enough to keep an organization in alignment. All team members must understand and have input on goals and objectives and those goals must be in writing and accessible to everyone. The benefits of establishing a definition of done include creating a shared understanding and unified language for software delivery, ensuring that new employees have access to tribal knowledge and process expectations, and detecting and removing false agreements between teams to create real consensus. By creating a definition of done early in an organization’s history, company leaders can ensure alignment even as more people join.
The Rust Foundation announced today it is working with the Open Source Security Foundation (OpenSSF) and JFrog to help maintainers secure open source software created using the Rust programming language.
Rebecca (Bec) Rumbul, executive director at the Rust Foundation, said even though Rust is a memory-safe language, there are still security issues that maintainers need to be able to address. The goal is to provide the resources those maintainers need to better secure their software supply chains.
As part of that effort, the Rust Foundation is adding a dedicated security team to help maintainers of open source projects built using Rust to better secure their software supply chains. That team is being underwritten by JFrog and the Alpha-Omega Initiative funded by Google and Microsoft which was recently launched by OpenSSF. JFrog last week announced it is joining the Rust Foundation at the Platinum level alongside Amazon Web Services (AWS), Google, Huawei, Meta, Microsoft and Mozilla.
The first initiative for the security team will be to conduct a security audit and develop threat modeling tools that will provide the foundation for a set of best security practices for maintainers, said Rumbul. The team will also help advocate for those security practices across the Rust landscape, including the Cargo package manager and Crates.io directory projects.
The OpenSSF has previously called for replacing programming languages like Java that are not memory-safe as part of a larger plan to secure open source software supply chains.
In general, the Rust Foundation is committed to enabling Rust developers to achieve their scale, security and sustainability goals, said Rumbul. Given the early stages of the Rust programming language, there is no better time to start addressing security issues that application developers can’t ignore, she added.
In the longer term, The Rust Foundation is also committed to providing developers with Rust security education and is also working to come up with a fair approach to certifying which projects have achieved an appropriate level of cybersecurity maturity, said Rumbul.
It’s not clear whether developers are abandoning older programming languages in favor of a memory-safe language such as Rust. Rust employs a compiler to track the ownership of values that can be used once and a borrow checker that manages how data is used without having to rely on traditional garbage collection techniques.
Of course, replacing trillions of lines of code that have already been constructed using a variety of legacy languages is a gargantuan task that might never be fully completed. However, as new applications are built using a memory-safe programming language, the overall security posture of an organization will steadily improve. The challenge, of course, is that the number of developers that know how to build applications using those languages is still relatively small.
Nevertheless, there may come a day when the way applications are coded is a lot more secure by default than it is today. In the meantime, however, the number of vulnerabilities that exist with application environments is still too numerous to count.
I have a fondness for philosophy. I’m about three classes short of a degree, and every few years I tell myself one day I’ll finish it. Thus, I am very familiar with what is known in statistics—and logic—as a post hoc fallacy, from which we get the saying “correlation is not causation.” This is the logical error of assuming that if event Y followed event X, event Y must have been caused by event X. The most famous callout of this fallacy came from Bobby Henderson, who illustrated the absurdity of assuming causation from correlation with his chart demonstrating that global warming was caused by the declining number of pirates in the world. Yeah, that doesn’t make sense, but neither do a lot of charts from which people draw causality. Just because two data points are mapped against each other does not mean one caused the other. In many cases, it doesn’t even make logical sense to correlate the two. After all, pirates and global warming? No one actually takes that seriously. But it’s an important point to make as we dive into the question of the relationship between site reliability engineering (SRE) operations and cloud repatriation.
To be clear, I’m not suggesting that adopting SRE practices causes cloud repatriation. But I am suggesting that there is a close and meaningful relationship between the two. The fact that Google—a cloud provider—created SRE as a practice is not a mistake. The model, mindset and skillsets associated with SRE are integral to successfully operating cloud infrastructure and services.
Public cloud repatriation itself is a somewhat taboo topic in certain circles. Consider the controversy raised by Andreessen Horowitz when it published “The Cost of Cloud, a Trillion Dollar Paradox” and suggested companies were repatriating from the cloud and realizing significant cost savings as a result. Some would have you believe it’s not happening, but there’s enough data and anecdotal evidence to indicate that yes, it is.
F5’s 2021 state of application strategy report asked the market about public cloud repatriation. A mere 13% had repatriated apps and another 14% were planning to. One year later, that combined total rose 40 percentage points to 37% and 30%, respectively. This is not an anomaly, as there are multiple credible analyst firms reporting similar results. Interestingly, the rate of repatriation is not globally universal. APCJ and LATAM are both far less likely to repatriate than EMEA and NA.
I maintain that companies are repatriating apps from the public cloud—and the question isn’t ‘Are they?’ but rather ‘How many workloads are they pulling out—and where are they going?’ That’s a question for the State of Application Strategy 2023 research.
For now, let’s consider a possible enabler of repatriation—SRE operations. Because even if the increasing cost of cloud is a driver of the desire to repatriate if you don’t have the skills to operate as efficiently elsewhere—and thus benefit from lower cost—then why would you repatriate?
And we posit that it is SRE operational practices and skills that enable companies to repatriate and maintain the efficiency and cost savings needed to justify the decision—whether they’re moving those workloads to another public cloud, on-premises or to the edge.
On the surface, there is a strong correlation between the adoption and application of SRE practices with cloud repatriation that seems to indicate that organizations with the ability to operate in a cloud-like manner, i.e., they’ve adopted SRE practices, effectively pick up their toys (apps) and go home (on-premises or elsewhere) because they can.
Cut another way, only 4% of organizations that have not adopted SRE practices have repatriated apps from the public cloud. A whopping 73% of those who have adopted SRE practices have also repatriated apps.
Of course, adopting practices does not necessarily mean applying practices. So, we looked at how organizations are actually operating applications, systems, and infrastructure. Specifically, we looked at the percentage of their operations that use SRE practices. Perhaps unsurprisingly, that generated similar results.
Of those who operate 0% of their apps, systems, and infrastructure using SRE practices, 81% are not repatriating. Conversely, of those who use SRE practices for 76%–99% of apps, systems and infrastructure operations, 54% have repatriated. The point at which repatriation appears to begin picking up steam is when organizations surpass using SRE practices to operate more than one-quarter (25%) of their apps, systems and infrastructure.
Remember I noted that APCJ and LATAM were far less likely to repatriate? They’re also far less likely to be leveraging SRE practices to operate their apps, systems, and infrastructure. In fact, over one-quarter (26%) of respondents in LATAM and APCJ (29%) were operating ZERO percent of apps, systems and infrastructure using SRE practices. In EMEA? That’s only 5%. And in NA, even lower at 2%.
There appears to be an inarguable correlation between organizations embracing SRE as an operational practice and public cloud repatriation rates. But is it a meaningful relationship or merely a curious coincidence?
I’m going to argue that it’s a meaningful relationship.
The practices and skillsets associated with SRE are wholly suited to operating a cloudy environment—at scale. As I said before, it’s no mistake that it was Google who created SRE and has literally written the book on it. The value of cloud is in its operational model, which can dramatically lower the cost per transaction, whether measured by HTTP exchanges or customer sessions. That enables a cost-efficient scale of applications and digital services.
The use of automation and practices that tend to focus on meaningful incidents rather than non-disruptive occurrences provides the cost-efficient scale of the people (and thus their expertise) who are tasked with maintaining a high level of availability and performance.
The adoption and use of SRE practices enable organizations to efficiently scale operations whether in the public cloud or on-premises or at the edge. And what the data tells us is that organizations appear to be using the capability to do just that.
As supply chain attacks become increasingly prevalent, visibility is emerging as a necessity in cybersecurity. One way to improve visibility and secure your software supply chain is with a software bill of materials (SBOM).
An SBOM is a list of all the components within a codebase. By providing insight into open source and third-party components, SBOMs improve overall visibility and are integral to the identification of security or license threats. To properly secure the software supply chain, organizations must have an understanding of all the components in their applications; it is especially important to do so with containerized applications and those built using cloud-native, microservices-based architectures. That’s why SBOMs are becoming a cornerstone of a comprehensive application security strategy.
Gartner’s recent report Innovation Insight for SBOMs emphasizes the need for constant visibility into the software development life cycle and how SBOMs can help organizations and developers achieve that goal. While Gartner recognizes that open source software is a foundational component of modern software development, it also noted that it can be a major contributing factor in software vulnerabilities.
“While reusable components and open source software have simplified software development, this simplicity has exposed a critical visibility gap: Organizations are unable to accurately record and summarize the massive volume of software they produce, consume and operate,” the report said. “Without this visibility, software supply chains are vulnerable to the security and licensing compliance risks associated with software components.”
To mitigate the associated risks, Gartner’s report not only emphasizes the importance of SBOMs but the need to automate the SBOM creation and validation process. To achieve supply chain security at scale, organizations should automatically generate and verify SBOMs and use that data to constantly evaluate oncoming risks.
This ties directly into the continuous integration and continuous deployment (CI/CD) process, which uses SBOM data and assessments to ensure threats don’t slip under the radar and wreak havoc. You can more easily identify security vulnerabilities at any stage in the pipeline and remove them—because an SBOM gives you more granular visibility.
The importance of SBOMs has even been noted by the U.S. federal government. As stated in president Biden’s Cybersecurity Executive Order 14028, any company that sells software to the federal government will be mandated to provide a complete SBOM.
The initiative has also spread to other branches of the government, with the Cybersecurity and Infrastructure Security Agency also emphasizing the significance of SBOMs.
“CISA will advance the SBOM work by facilitating community engagement, development, and progress, with a focus on scaling and operationalization, as well as tools, new technologies, and new use cases,” said CISA in a statement on its website.
The U.S. Department of Commerce and the National Telecommunications and Information Administration (NTIA) have outlined the minimum elements required. For one, data fields with baseline information about each component are required. Also, automation support is needed, as SBOMs necessitate machine readability for support at scale. Finally, the need for practices and processes was outlined, calling for frequent SBOM generation, including transitive dependencies and widespread availability of this information to those who need it.
On many fronts, SBOMs have become non-negotiable throughout the software industry. In order to effectively detect and respond to threats, organizations must take preemptive steps to ensure visibility is maximized, and SBOMs accomplish just that.
Want to learn more about this and other related topics? Join CloudBees for DevOps World 2022, held in Orlando, Florida, at the World Marriott Center. Use discount code DW22 when registering and get access to all the fantastic offerings, including keynotes, sessions, training and other interactive activities. We can’t wait to see you there for the DevOps Remix!
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Facebook engineers admit they’ve no idea what Meta stores (or where), California requires job ads quote salaries, and RISC-V will power future NASA spacecraft. (more…)
Software development strategies continue to evolve to meet changing business needs. There has been a lot of discussion of digital transformation recently, and custom software development is driving much of that transformation. Agile software development and DevOps have become the norm as companies strive to develop better, faster, more flexible applications. Microservices offer the next step in the evolution of enterprise application development.
DevOps separates developers (Dev) and operations (Ops) into teams collaborating on new enterprise solutions. Similarly, microservices expand on the idea of specialization, creating specialized app building blocks to build new applications. Microservices enable an even shorter development time and greater agility. They also present a new set of challenges for enterprise architects who must oversee these microservices and govern the development process.
If you are unfamiliar with the term, microservices are a series of building blocks assembled to make new applications. Just as agile software development is a philosophy as well as an approach to development, microservices are an architectural and organizational approach to enterprise software development.
The value of microservices is in the building block approach. With old-school monolithic services, you must design and maintain a single application and its various components to act as a unit. Change or remove one element, and the entire application needs to be rewritten and tested, which is costly and time-consuming.
With microservices, the individual services function as self-contained functional units that can be mixed and matched as needed to create different custom applications. Using microservices is faster, more versatile, and more reliable, simplifying the delivery of large, complex applications.
Microservices also are built around business capabilities so that business application components can be created in tandem with DevOps agile enterprise development. And with different teams working to create smaller components, there is less chance of having a single point of failure.
No-code/low-code development solutions are placing the power of microservices in the hands of business users. Domain experts in product development, finance, logistics, and other departments can use a drag-and-drop interface to develop microservices that can then be incorporated into other applications. In short, microservices gives subject matter experts the tools to create custom apps that can be added to the enterprise architecture.
Microservices offer businesses and enterprise architects several benefits. Deployment of new applications is faster, and you have more flexibility to adapt to changing enterprise and business needs. They also scale quickly and make it simple to add new features to existing applications. And microservice elements are reusable.
Of course, there are challenges as well, including governance. As the architecture evolves, there are more components to be managed. In addition to different departments building microservices, the DevOps team is still responsible for overseeing application development and solving everyday problems. Having a mature engineering culture helps automate app development and track microservices, providing the necessary guidance to ensure app development supports larger business objectives.
Microservices also create architectural overhead. Applications must be documented to ensure ongoing business value. Trying to document everything, however, creates delays, so the documentation is never current with the stage of development. The challenge is providing enough documentation to be useful without delaying ongoing software changes.
This requires a new enterprise application mindset. DevOps needs to closely watch autonomous teams to control development and ensure they support larger corporate goals. The use of no-code/low-code tools also could result in shadow IT not being sanctioned by the network architects or DevOps.
One of the market forces driving microservices adoption is the need for “outside-in” application development. All microservices should be developed at the request of a customer, whether that customer is inside or outside the organization. Customer requirements continually change, and microservices can accommodate those changes faster and more efficiently.
Microservice development works best in a domain-driven architecture, which models the applications based on the organization’s real-world challenges. A domain-driven architecture assesses the enterprise infrastructure in light of business requirements and how to fulfill them. Most organizations already have a domain-driven design strategy in place that maps the architecture to business capabilities.
Bounded Context is a strategy that is part of domain-driven design. Autonomous teams responsible for microservices are formed around areas of responsibility such as inventory management, product discovery, order management, and online transactions, i.e., bounded context. The domain expertise resides within the team, so the enterprise architect’s responsibility is to guide development to align with strategic goals, balancing immediate needs and future business objectives.
When governing microservices as part of the enterprise, applying the C4 model for software architecture—context, containers, components and code—makes sense. The context defines the relationship between the customer or end user and the application. The container represents the microservice itself and how it is packaged to serve as an application building block.
The secret to governing microservices in any enterprise architecture is to map business capabilities to the digital infrastructure, then promote the vision behind the enterprise architecture as well as the guidelines to achieve that vision. Focus on organization-wide objectives for both business transformation and IT strategy. Educate the stakeholders on when to involve the enterprise architect. Also, try to anticipate topics beyond the perspective of the DevOps team and focus on the big picture—what is the long-term business strategy and how do microservices contribute to that strategy?