Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: People won’t shut up about Rustlang, and Musk mandates Twitter teams return to the office. (more…)
Category: DevOps Toolbox
-
Rust Momentum Intensifies | Elon Says No WFH
-
CloudBees Demos Orchestration Capabilities of ReleaseIQ Platform
At the online DevOps World 2022 event today, CloudBees demonstrated for the first time the capabilities of the application release management platform it gained last month through the acquisition of ReleaseIQ.
Shawn Ahmed, chief marketing officer and interim chief product officer at CloudBees, told conference attendees that the ReleaseIQ platform will provide application development teams with visibility into precisely how their code is making it into production environments.
Armed with those insights, it then becomes possible to understand exactly how the way commits are made during a build process are, for example, creating a workflow bottleneck, he added.
The ReleaseIQ platform is designed to enable DevOps teams to both create and maintain pipelines using a no-code tool. That enables greater transparency into workflows via dashboards that also provide optimization recommendations.
CloudBees already provides a Flow orchestration engine within its continuous delivery platform. However, ReleaseIQ provides a similar capability across multiple continuous integration/continuous delivery (CI/CD) platforms using a SaaS platform that is more accessible. ReleaseIQ can also be integrated with multiple platforms including CloudBees CI, Jenkins, CircleCI, GitLab, Bamboo, ArgoCD and others.
Naturally, the current level of DevOps maturity varies widely by organization. However, as more organizations realize how much they depend on software to drive digital business transformation processes, the need to further automate application development and delivery becomes more apparent. While CI is widely used today, the number of organizations that have consistently implemented CD to automate application release management is still fairly limited. The underlying infrastructure that software is deployed on typically has a unique set of application programming interfaces (APIs) that each CD platform needs to support. Platforms such as ReleaseIQ provide a higher level of abstraction that simplifies managing the pipelines that deliver software.
CloudBees is essentially positioning ReleaseIQ as the orchestration engine for both unifying pipeline management and, ultimately, enabling value stream management across increasingly complex application development environments. It may take a while to fully realize that ambition, but with the acquisition of ReleaseIQ, the company is now pursuing a more cloud-centric approach toward enabling DevOps teams to, for example, more easily reuse pipelines.
DevOps teams, of course, have created a lot of scripts over the years to provide orchestration capabilities. The issue is that those scripts tend to be brittle in the sense they don’t scale especially well over time. They also tend to be poorly documented and challenging to maintain if the script writer leaves the organization. It’s not uncommon for new members of a DevOps team to replace a script with one they wrote simply because they’re more familiar with how it’s constructed.
More than a decade after the launch of the Jenkins CI/CD platform, there are still plenty of manual DevOps processes that should be more automated. A no-code orchestration platform may prove to be a significant step toward automating a wide range of tasks that consume more time than they should.
-
OpenSSL Fiasco: What can DevOps Learn? | Elon Fires ‘50%’ of Twitter
Welcome to
The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: The OpenSSL project has egg on its face, and half of Twitter’s staff are for the chop tomorrow. (more…)
-
Gitpod Looks to Drive Application Development Into the Cloud
Gitpod today revealed it will add an application programming interface (API) to make cloud development environments (CDEs) more extensible. Gitpod also said Johannes Landgraf assumed the role of single CEO for Gitpod, as the previous co-CEO, Sven Efftinge, becomes the company’s first technical fellow.
Fresh from raising an additional $25 million in funding, Gitpod’s Landgraf said that API will make it easier to build plug-ins for CDEs. In time, Landgraf said, CDEs will become as accessible as any development environment running on a local desktop.
Software engineers typically waste five hours a week maintaining application environments, noted Landgraf. Via an open source development platform, Gitpod automates the provisioning of secure workspaces in the cloud that provide the added benefit of making it easier for multiple developers to collaborate, he added.
Capabilities added to the GitPod product roadmap include dramatically faster startup times, improved resiliency and a Gitpod Cloud offering for larger companies as well as a preview environment and workflows that foster collaboration. Gitpod also announced it passed a SOC2 Type II security audit.
Today, the Gitpod platform is being used by more than 750,000 developers at organizations such as Shopify, Stripe and Slack, noted Landgraf.
One of the longstanding issues that has always been problematic when building software is that code that runs on a local desktop doesn’t always work when deployed in a production environment. The Gitpod API will make it simpler to spin up various platforms and tools whenever a developer accesses a CDE. The goal is to make it possible to more closely replicate the production environments on which developers will ultimately deploy their applications, noted Landgraf.
In addition, a CDE should reduce the amount of data that developers routinely move between laptops and cloud computing environments as they build more complex applications, he added.
In the wake of the economic downturn, there is a lot more focus on developer productivity these days, especially as organizations look to accelerate their of digital business transformations. DevOps teams are also keen to find ways to centrally manage developer experience at a time when many of those developers continue to work from home, said Landgraf.
It’s not clear to what degree CDE will replace integrated development environments (IDEs) running on a local machine, but all the major cloud service providers are encouraging developers to build applications in the cloud where their applications will run. The challenge is that many of those offerings tend to lock developers into a specific cloud platform, noted Landgraf. Most developers today are building applications that need to be deployed across multiple clouds, he added.
One way or another, software development is increasingly shifting to the cloud. Most organizations will likely try to find a balance between the amount of code developed on a local machine that developers can just fire up when the mood strikes and the amount of code developed in the cloud. The most immediate challenge is to make sure the CDE experience is every bit as good as the existing local IDE.
-
Pulumi Adds Deployment Capability to IaC Platform
At the Pulumi Cloud Engineering Days event, Pulumi announced today it has added a code deployment capability to its Pulumi Cloud platform for managing infrastructure-as-code (IaC).
Pulumi CEO Joe Duffy said Pulumi Deployments will make it possible for DevOps teams to provision infrastructure and deploy applications via a single Git commit that can be invoked via an application programming interface (API) or graphical tool.
A Git Push to Deploy capability allows DevOps teams to connect to any Git repository to drive deployments for a given project path, push to any given branch and set up GitOps workflows such as pull request reviews. Deployment actions include both updates and destroys in addition to providing drift detection and remediation capabilities.
The Deployments and Automation API Pulumi developed also enables ephemeral code review environments, automatic cleanup and deployment orchestrations across multiple cloud platforms using a library of templates Pulumi provides. In effect, Pulumi is now providing a GitOps workflow out of the box, said Duffy.
Pulumi has already integrated these capabilities with GitHub and plans to add support for GitLab, Atlassian Bitbucket and other DevOps platforms. The goal is to make it possible for DevOps teams to effectively manage ten times the amount of cloud infrastructure resources per engineer using GitOps workflows than is currently possible relying on IaC tools such as Terraform, said Duffy.
As an alternative to Terraform, the Pulumi platform has been gaining traction because it makes it possible to more easily centralize the management and provisioning of infrastructure using a set of embedded guardrails. One of the major issues enterprise IT organizations are encountering these days is the number of misconfigurations created when developers use tools like Terraform to provision infrastructure on their own. Those misconfigurations often lead to, for example, ports being left open and cybercriminals exfiltrating data. In the wake of a series of high-profile breaches, more organizations are now reviewing the security of their software supply chains including how infrastructure is provisioned by embedding policy-as-code into DevOps workflows.
At the same, organizations are also keenly interested in increasing the productivity of DevOps teams during an economic downturn that limits the number of IT professionals an organization can afford to hire, added Duffy.
Pulumi claims there are now more than 5 million users of its platform spanning more than 1,000 organizations. Less clear is the degree to which organizations are embracing GitOps workflows to unify deployments on IaC and application code. As platforms make it easier to make that transition, the number of organizations using a more opinionated approach to DevOps should steadily increase.
In the meantime, DevOps teams are going to be required to find ways to build and deploy safer applications faster. Much of the focus in recent years has been on accelerating application deployments with little regard for cybersecurity. Those days are clearly now coming to an end.
-
Know What You Want and Need
Over the years, what we purchase from vendors has changed significantly. It used to be that IT would define a need, go out and find a bit of software to solve it, install and configure the software and, in most cases, spend the majority of time on the project getting it to work correctly in their environment. Those days are largely gone. First, we started buying appliances for some functionality, then we started using MSPs for some or all functionality, then some vendors wanted to deliver in a VM where they could make configuration much easier by controlling the environment. That was the first step to virtualizing purchases. Then we started looking at infrastructure-as-a-service (IaaS), then SaaS, then some vendors offered PaaS … Currently, all of the cool vendors are offering containers—which are an improvement over VMs, and offer a virtual appliance that is quick to get running and tested to do what it was purchased for. Or at least tested to do what it was sold for. We’ve all got stories about what we did with these purchases that were “off label”—things that a product really wasn’t designed for. Along the way, we also introduced open source vendors; a bit of an oxymoron that really described service orgs that base their product on an open source project. Some far more successfully than others.
And all of that, dear reader, is why we are here. Before purchasing, there is a lot more to find out than there used to be. What, actually, is the product this software vendor is selling? In the end, IT is looking to solve a problem. Most of the time, whether that problem is solved by SaaS, local install or cloud instance is less important than how well the problem is solved. But implementation—and licensing—are important to an organization if not immediately, then over the long term.
When purchasing an open source “product”, you’re … actually not. You are getting the product and purchasing support. Since open source can be hacked to run just about anywhere given enough time, more than one OSS vendor has pretended that makes them superior to commercial products targeted at a specific platform. That is blatantly untrue—unless the OSS vendor is supporting the product anywhere and everywhere which, of course, they are not. I can port the OSS project MyFunApp to any platform I want without paying for support that won’t do a thing for me when I get it up and running on OS/2 (as an extreme non-existent example).
SaaS has more issues when it comes to TCO. While pay-as-you-grow is cool and all, locking in a monthly payment forever is a big step and should be considered. And at an enterprise, “forever” needs to be considered realistically. Implement a particular SaaS CRM and you are unlikely to move to another, even if you claim that you will in a heartbeat. It’s work. A lot of work. It is definitely possible, but for most organizations, unlikely. So know what you are getting into. And while you are evaluating, don’t forget to consider variable charges, which most SaaS products have. Make certain you understand what the actual monthly costs are likely to be. IaaS suffers this same variability problem, as many of us have learned.
Containers are a very sweet solution to appliance-like functionality—drop a container into your Kubernetes flavor of choice and move along. The problem here is a combination of proliferation—how many are required? One per pod can get expensive over time, for example. Those resources aren’t free, no matter the platform. The other issue is updates. Automatic updates are the norm for container instance-delivered products. From a security perspective, you should have questions. Updating things in your network should probably require your approval.
I’m not suggesting that one delivery method is better than others here; I’m simply suggesting that we all need to be more deliberate about those choices than we’ve needed to be in the past. Knowing not just SLAs but the cost and support implications of every choice is important both today and down the road.
Keep kicking it, and keep picking the solutions that best solve the organization’s problems. Just be aware that sometimes, the issues a delivery method introduces undermine the effectiveness of the solution.
-
InfluxData Delivers Revamped Database for Real-Time Processing
InfluxData has made good on a promise to deliver a next-generation time-series database that enables observability and monitoring tools used by DevOps teams to ingest high volumes of data more cost-efficiently.
In addition, for the first time, the latest iteration of the InfluxDB database supports the de facto standard SQL query language. Developers can run unlimited time-series workloads and contextualize data by any dimension without restrictions using SQL or existing tools such as the InfluxDB application programming interface (API), Flux scripting language or the InfluxQL query language
Brian Gilmore, director of internet-of-things (IoT) and emerging technologies for InfluxData, said InfluxDB is now based on a columnar engine, dubbed IOx, that leverages the open source Apache Arrow memory format and the Rust programming language accessed via a multi-tenant InfluxDB Cloud service that company manages on behalf of application developers. Use cases that require processing large volumes of data such as metrics, events, traces and other high cardinality data can now be queried in milliseconds, he noted.
The goal is to make this latest incarnation of InfluxDB available via a managed cloud service before providing IT teams with a version they can install themselves.
The arrival of a revamped InfluxDB architecture comes as the amount of data being generated across application environments continues to rapidly expand. Many DevOps teams are especially struggling with ways to process and store all the data being generated by cloud-native applications made up of microservices generating a steady stream of logs, metrics and traces.
InfluxData, of course, has ambitions for its database that go beyond application development and deployment environments. Edge computing platforms, for example, will increasingly process and analyze data in real-time at the point where it is generated and consumed. Similarly, applications infused with machine and deep learning algorithms will require databases that can process data at scale. Regardless of the use case, however, it’s apparent more IT teams will be embracing DataOps best practices alongside DevOps to manage data as applications are deployed and continuously updated.
It remains to be seen how many observability and monitoring tools might embrace this latest iteration of the InfluxDB database, but it’s clear DevOps teams should expect to see major advances as the underlying databases upon which these platforms depend continue to evolve. The goal is to ultimately eliminate the tradeoff that now often occurs between how much data is collected and analyzed versus the cost of processing and storing it.
In the meantime, DevOps teams will need to decide whether they want to manage databases themselves as more managed services become easier to invoke in the age of the cloud. Most database selections these days are made by development teams that aren’t especially eager to take on the tasks normally performed by a database administrator. A managed service essentially outsources responsibility for managing a database to a services provider instead of requiring IT teams to hire a DBA themselves.
One way or another, however, the speed at which data can now be processed in real-time should sharply reduce the need to rely on existing and cumbersome batch-oriented approaches to processing data.
-
Nobl9 Simplifies SLO Identification
Nobl9 today added a Replay capability to its service level objectives (SLO) management platform that makes it simpler to collect the historical data a DevOps team needs to get started.
Nobl9 COO Kit Merker said Nobl9 Replay makes service level indicator (SLI) data instantly available in minutes using 30 days of historical data collected from observability and application performance monitoring (APM) platforms. Previously, DevOps teams would have had to wait as long as a month before the Nobl9 platform could collect enough spreadsheet data to create a graph that identified the relationship between various services.
Nobl9 now enables historical data to be merged with data collected in real-time to allow DevOps teams to view SLO charts and generate reports shortly after creating an SLO within a Git-based workflow.
Nobl9 is trying to spur greater adoption of SLOs by making available an open source SLO specification that defines a common interface for constructing them that has garnered support from Dynatrace, Sumo Logic and others. SLOs, of course, are not a new idea; they have been used as a metric to track the performance of IT services for decades. However, as more microservices-based applications are built and deployed, it’s becoming more challenging to maintain SLOs across applications that have many more dependencies than legacy monolithic applications.
The Nobl9 platform is designed to enable DevOps teams to employ an SLO-as-code approach that makes it simpler to gather the metrics that confirm whether SLOs are being achieved.
It’s not clear whether reliance on SLOs has waned over the years or if application environments simply became too complex to track meaningful metrics. A recent survey conducted by Dimensional Research on behalf of Nobl9, however, found a full 94% of respondents that have or plan to implement SLOs intend to map them directly to business operations, with 91% reporting they expect that effort to improve decision-making. More than 80% also said their organizations are planning to increase the use of SLOs, with 87% indicating SLOs should improve overall microservices performance.
The issue is that most organizations have limited visibility into their IT environments. The survey found less than half of respondents (46%) had visibility into all their IT environments. Only 45% and 35% claim to have visibility into containers and microservices, respectively. More than three-quarters (78%) said hybrid clouds make observability more difficult, even though 45% said they already employed 11 or more observability and monitoring tools. Trying to track SLOs using tools such as Grafana dashboards that DevOps teams need to set up and maintain is simply too difficult, noted Merker.
As applications are increasingly constructed using services that drive digital business transformation initiatives, it’s now only a matter of time before SLOs are more widely tracked. That will enable DevOps teams to not only ascertain their current level of performance but also more easily work toward achieving their actual performance goals, he added.
The challenge is that defining an SLO in a highly dynamic application environment where services are added and replaced with much greater frequency is often a lot easier than maintaining it.
-
The Dog Ate My DevOps Dozen
As the new Techstrong Group team member, I wasn’t familiar with the DevOps Dozen awards. The basics: It’s a cool program that recognizes excellence across a number of categories in the DevOps world. The winners will be announced during our Predict 23 conference on January 12. I have to tell you that the submission process was interesting.
It’s not surprising that many of the submissions came in on the last day. Some within the last hour of the last day. But what was surprising were some of the excuses we got when companies inevitably asked for extensions.
No one actually used the “dog ate our submission” excuse—mostly because, well, it’s an online submission process so, unless the dog swallowed a computer, that’s not really a thing anymore. It seemed that honesty prevailed this year. We got a bunch of folks that said they were working on a major launch; others said they’d been consumed with getting ready for KubeCon and asked for a little more time.
We were somewhat sympathetic since we also were getting ready for KubeCon and totally understand the all-hands-on-deck push required to get ready for a major event. We also encountered some issues with our payment processor which prevented the completion of some submissions. We’ve fixed the issue and we’ve learned from that experience, so it won’t be a problem next year.
In terms of the categories, unsurprisingly the top three that attracted the most submissions were “Best DevSecOps Solution,” “Best End-to-End DevOps Tool and Service” and “Best New DevOps Tool/Service Provider”. Given the move to integrate security into the DevOps environment and the move to integrate the variety of DevOps tools onto a platform, the popularity of those two categories was expected.
The next wave of categories included “Best Observability Solution,” “Best Cloud-Native Security Solution/Service” and “Best CI/CD Tool” to round out the top six. Again, less strategic and more tactical in this second group.
If you are looking to go where the crowds aren’t, the least-crowded three categories in terms of submissions were “Best DevOps-Related Video Series (Video),” “Best DevOps Transformation (non-vendor)” and “Best DevOps Presentation of the Year.” Maybe those aren’t the sexiest, most buzzworthy categories, but the competition is minimal.
So what did we learn from this year’s submission process? Try not to wait until the last minute and that the most competitive categories are those that deal with hot issues. Keep an eye out for the beginning of public voting, and we’ll see you at the awards ceremony during Predict 23!
-
Meta Income Down by Half | Will Apple Make it Worse? | Linux Secure Boot Fix
Welcome to
The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Meta’s latest results are very bad, Apple wants its cut of Facebook ads, and Lennart Poettering proposes a new Secure Boot for Linux. (more…)