devopsy.kinsta.cloud

Category: DevOps Toolbox

New Relic Adds Support for W3C Trace Context Spec

New Relic has begun adding support for the W3C Trace Context standard within the agent software it uses to collect metrics for its application performance management (APM) platform.

W3C Trace Context is a specification that defines the format for propagating distributed tracing context between services. Distributed tracing makes it simpler for DevOps teams to track events across multiple services. Each step of a trace is correlated through an ID that is passed between services, with the specification defining a standard for the context propagation headers.

The W3C Distributed Tracing working group moved the Trace Context specification to the point where it could be consumed by platform vendors late last year. TraceContext has since been adopted by the OpenTelemetry project, an open source observability framework being developed under the auspices of the Cloud Native Computing Foundation (CNCF), along with providers of APM platforms such as New Relic, LightStep and Dynatrace. Vic Soares, director of product management for New Relic, said many of the individuals working on the TraceContext and OpenTelemetry projects come from the same organizations.

The first phase of the Trace Context project has focused on HTTP. The members of the project are also working toward defining trace context propagation for other formats, starting with AMQP and MQTT for IoT. The project team is also working on providing context propagation from clients and web browsers.

That approach solves an issue that arises when using proprietary formats: As trace context headers move across boundaries such as proxies, service meshes and messaging systems, proprietary headers tend to be dropped, which often winds up breaking traces, said Soares.

Soares said DevOps teams will need to upgrade the agent software they have in place to take advantage of Trace Context to provide more context across distributed services. Distributed tracing, however, will not replace sampling. Rather, distributed tracing provides a mechanism for identifying the root cause of issues that may first be uncovered using sampling techniques.

New Relic APM agents for Java 5.1.0 and higher, Python 5.5 and higher, Go 3.1.0 and higher and the New Relic open source Elixir agent now support the standard. Support for Trace Context support in other New Relic agents, as well as the New Relic Browser agent, are expected to be available shortly. New Relic APM agents that support W3C Trace Context can accept and emit both the W3C Trace Context header format and the New Relic header format. The new agents are also backward-compatible to enable trace context and will be propagated between services with older and newer releases of New Relic agents. IT organizations can also opt to disable the New Relic proprietary trace format once they’ve upgraded all agents to a W3C-compliant version.

Given the complexity of microservices-based applications, many organizations that deploy these applications will be among the first DevOps teams to take advantage of distributed tracing to not just troubleshoot applications but also proactively manage their application environments. Going forward, a lot more of the DevOps focus needs to be on identifying potential issues long before they can become a bigger problem.

— Mike Vizard

February 7, 2020
How to Achieve a Converged Toolchain

In recent years, there’s been an explosion of tools for developing and delivering software. In a single pipeline, a company will have separate tools to ideate, plan, code, verify, configure, release and monitor new software, and every pipeline can have different tools for each individual stage (i.e. ideation). That’s a lot of tools, and providing visibility into the process of application development and delivery is difficult and can create many challenges.

The key to eliminating these challenges is to develop a converged toolchain.

What Is a Toolchain?

A toolchain–in the context of software delivery–is a collection of tools managing the planning, execution, quality and delivery of the software being developed, starting from idea generation all the way to production. Each phase of development can have numerous tools, and it’s not uncommon for an organization to use multiple test tools, multiple tools for backlog/user story planning or multiple defect tracker tools.

By default, each tool operates in a silo–not really interacting with other tools and not visible to other groups. Without a converged toolchain, companies are left with fragmented visibility along the application development and delivery pipeline. This means the development team has no idea what the planning team is doing, the operations team can’t see what’s happening with the development team, and so on. Of course, someone from the development team could walk over to the planning team and ask, “what’s going on?” But that halts momentum and slows down the entire process.

The Kanban framework is a great example of a siloed tool commonly used in planning. It presents a list of all the features that need to be built, but it can’t track features in production unless someone manually changes a status. Thus, it’s only as reliable as each person making the updates, and it results in the status of many projects being out of date and stale.

The service desk is another good example. Here you’ll find which projects are in or about to be in production, but it leaves you with very limited visibility into what’s coming down the pipeline. It also doesn’t provide you with context. For example, the operations team may know what app they’ll be working on, but they don’t have an idea of the risk profile of what’s coming.

Benefits of a Converged Toolchain

With a converged toolchain, disparate and siloed tools can work in one connected ecosystem. Help desk tickets can match a bug to the specific developer who worked on the feature, and initiatives can be matched to applications, environments and/or production change requests. The convergence of tooling data enables visibility between groups–allowing each group to use their own tools but at the same time work with data that is augmented from the rest of the toolchain–and this visibility enables collaboration between the groups. Connected toolchains also enable each team to continue to work on their part of the process using their own tools, and teams can evaluate the entire process of software delivery and monitor for hand-offs, waste and inefficiencies. This holistic visibility enables efficiencies not previously possible.

Without a converged toolchain, teams are likely to run into problems around timing and planning. For example, when the testing team is ready to tackle their part of a project, their first questions are typically, “what environment is it on?” and “what features does it include?” Unfortunately, these can be some of the hardest questions to answer. But a converged toolchain allows the testing team to easily see where exactly the unverified feature is, and quickly move it along the pipeline.

Achieving Convergence

By converging the toolchain, you’re integrating and interconnecting the tools in the toolchain. To achieve this, you’ll need to complete three steps. First, you need to correlate and integrate data from the tools into a common data model. This will help create a standard way of looking at features, tests, environments, release processes and phases of delivery. All of these tools need to feed into their part of application delivery so you can correlate all the data together.

Once you have that, the second step is to synchronize data between the tools and bring all the data into your common data model. This will give you the visibility your teams need so you can connect particular features to the environments they are on or know what features still need to be built.

The third step is to create workflows. This allows you to not only bring data in but also take action on the data and react to things based upon events. For example, when the operations team is transitioning a new feature to production, they can mark it as ready for production, and it will be labeled that way across all tools and all teams. This ensures every team is on the same page and no piece of work can become stale.

This workflow can also help when it comes to issues that are reported to an organization’s help desk or technical support of internal applications. Without a converged toolchain, that help desk worker has to create a ticket, call up to the developer team and try to track down the person who can fix the issue. However, with a converged toolchain, that help desk worker can automatically create an alert for the developer of that system. Then the developer is able to quickly see the bug, begin working on a hot fix and send a note back to the end user with an estimated time of completion. All of this is completed within the workflow of the converged toolchain and without anyone having to physically chase down another person for answers.

By restricting themselves to only living in silos, most organizations have created a problem when it comes to communication and workflow across teams. Instead of working on items that deliver value to their customers, teams are spending a considerable amount of time on hand-offs. While it is important to know where a project stands, teams should be focused on delivering value to the customer, not gathering status updates.

The days of living with this kind of fragmented visibility into the application development and delivery process is not acceptable anymore. Every organization’s toolchain should be focused on removing execution barriers and automating the development and continuous delivery process. In order to achieve this kind of success, teams need to be able to work with full transparency in their tool of choice while also being able to see the broader context of how their work fits in to the rest of the toolchain. With a well-designed, integrated and automated DevOps toolchain, development and operations team members are able to work together with common objectives and metrics to ensure quality, on-time application delivery to the customer.

— Jeffrey Keyes

February 3, 2020
Latest Chef Habitat Release Embraces On-Premises IT

Chef today announced that the Chef Habitat automation framework for building and deploying applications can now be used in on-premises IT environments.

Mike Krasnow, a senior product manager for Chef, said version 1.5 of Habitat makes it possible to now store applications packages and their dependencies locally in addition to the cloud. In addition, pre-built application packages and dependencies from the Builder service can be deployed across multiple on-premises instances.

An Applications tab in Chef Automate now shows the service groups and instances that are deployed to make it easier to understand what’s deployed where and the health of each service. Both the Habitat Builder installer and the application dashboard are integrated with Chef Automate, which uses the same installer as Chef Habitat.

Finally, the Habitat Supervisor module is now more stable, easier to configure and manage, and provides improved support for Windows applications.

Krasnow said Chef has seen a 300% increase in the use of the software-as-a-service (SaaS) instance of Chef Habitat in the last year. Organizations that have embraced Chef Habitat include Alaska Airlines, Rakuten, and Walmart. That rate of adoption should increase now that an on-premises edition is available, he said, including among IT organizations in highly regulated industries that are not allowed to employ cloud services. In fact, at least 70% of all application workloads are still deployed in on-premises IT environments.

Chef Habitat appeals most to organizations that have embraced best DevOps practices to programmatically deploy applications, said Krasnow. There are rival automation platforms that make use of declarative approaches to automation that appeal more to IT generalists. However, he noted, organizations that want to empower DevOps teams tend to prefer a more programmatic approach.

For more than a decade now Chef has been driving adoption of IT automation. However, it’s only relatively recently that organizations have been employing IT automation at scale as part of their efforts to embrace best DevOps practices. As interest in IT automation frameworks has risen, so too has the number of options. Some organizations are even employing multiple IT automation frameworks within different business units. They may one day standardize on a single IT automation framework, but many DevOps teams will simply opt to employ the IT automation framework the members of their team already know. If the members of that team in charge of IT operations don’t have a lot of programming expertise, chances are high they will opt for a more declarative approach. DevOps teams that have developers who deploy applications on their own are likely to prefer a more programmatic approach.

Regardless of the platform chosen, it’s clear the days of manually deploying applications using custom scripts are coming to an end. IT organizations now require an industrial approach to managing IT as a factory. The biggest challenge achieving that goal is no longer the technology but rather the culture of the IT organization.

— Mike Vizard

January 29, 2020
F5 Networks Launches Next-Gen ADC Based on NGINX

F5 Networks this week launched version 3.0 of its NGINX application delivery controller (ADC) that adds application programming interface (API) management tools, analytics and monitoring tools, a service mesh, a certificate manager, a developer portal and integrations with multiple continuous integration/continuous deployment (CI/CD) platforms.

Ken Bocchino, director of product management for NGINX at F5 Networks, said as the first major update to the NGINX Controller since F5 Networks acquired NGINX last year, the 3.0 release represents a concerted effort to reduce the total cost of DevOps by bundling many capabilities that previously would have needed to be acquired separately.

In addition, NGINX Controller 3.0 reduces the complexity associated with building DevOps workflows because all the components that make up NGINX Controller have been pre-integrated, said Bocchino.

Based on the open source NGINX proxy server, NGINX Controller is the mechanism through which commercial capabilities are provided and supported as part of the NGINX Plus subscription service offered to enterprise IT customers. Previous iterations of ADCs were much more infrastructure-centric than application-centric, noted Bocchino.

While F5 Networks naturally would prefer IT organizations to consume all those services, Bocchino said the ADC has been designed in a modular fashion to enable organizations to consume services exposed as REST APIs as they see fit. A future version of the NGNIX proxy server will be able to support multiple types of service meshes, including the lighter-weight offering created by the NGINX team that will become available later this year as part of NGINX Controller 3.0, he said.

Bocchino noted F5 Networks envisions the NGINX Controller playing a major role in driving further convergence across network operations, security and DevOps teams. NGINX Controller 3.0 facilitates that goal by providing a self-service portal along with role-based access controls and modular workflows that eliminate the need for tickets to be issued to achieve every IT task.

It’s not clear at what rate that convergence is occurring. However, F5 Networks last year published a survey of DevOps and NetOps teams that suggests there is still a wide gap between NetOps and DevOps teams. The survey found only a little more than a third (38%) of DevOps teams cited the “integration of toolsets across vendors/devices” as a challenge to network automation. In contrast, almost half of NetOps respondents (47%) noted a lack of integration as a problem, second only to a lack of automation expertise (49%).

F5 Networks by acquiring NGINX is betting than most organizations that have already embraced NGINX proxy server software would rather extend those existing capabilities versus replacing their ADC with a new stack of software based on, for example, the rival open source Envoy proxy server being developed under the auspices of the Cloud Native Computing Foundation (CNCF).

How the proxy server war and associated ASC platform wars will play out is still anybody’s guess. What is clear is that DevOps teams will need to access a wide range of programmable networking and security services easily to take DevOps to the next level.

— Mike Vizard

January 28, 2020
The Battle for End-to-End DevOps Champion Is On
This past week’s CollabNet/Xebia Labs merger brings another challenger to the “end-to-end” DevOps market. There is a huge prize waiting for the winner. But which of the challengers has the right stuff to win this battle? Will any of the current pack even be around at the end of the day? Who will be the winners and losers? What will it take to win?

The eventual dominant player in the end-t0-end DevOps provider market will be rewarded with what should be a great pot of gold. It is this promise of hegemony over DevOps and the rewards it commands that have so many chasing the prize. CloudBees, JFrog and GitLab are some of the DevOps darlings in this market. Chef has a similar game plan, and now the new entity stemming from the CollabNet/Xebia Labs deal will also be a player. I believe GitHub/Microsoft will make a play in this market, too.

At this point in time, who knows which company will be triumphant and which will be regulated to the trash heap of also-rans? One thing I learned from my friend Brad Feld is that if you are not in the top three in your category, get out. In the end-to-end DevOps or complete DevOps market, there will be some winners and there will be some losers.

Sanjeev Sharma, principal analyst for ASG, had this to say about the market:

As engineers strive to streamline their application delivery pipelines to be able to achieve true continuous delivery, a massive stumbling block becomes an end-to-end delivery pipeline with tools that are seamlessly integrated. At one end of this spectrum we have organizations that acquire best-of-breed tools and glue them together themselves, owning the resulting plumbing. At the other end there are organizations that want tools from a single vendor that ‘just work,’ integrated out of the box. The pendulum today has swung toward the single vendor, integrated out-of-the-box side. The growth of platform vendors like GitLab and the consolidation of CI/CD tool vendors with larger ones is the result of this swing.

Right now, while several contenders have a great vision, none of them really have a bonafide end-to-end platform or suite with functionality all the way across. There are also other contenders that I have not mentioned, but here is my take on what is out there that I see:
1. 1. GitLab – The company recently won our DevOps Dozen DevOps solution provider of the year and it was well-deserved. Sid Sijbrandij and team have done a great job laying out a bold, ambitious vision of GitLab’s eventual full functionality. On top of this, they add almost 200 new features a month, every month. The company is growing by leaps and bounds, with almost 1,200 employees, and has raised hundreds of millions of dollars with a valuation of $2.75 billion. GitLab has the resources to go deep in this market and has built some tremendous momentum. On the other hand, the fact is today GitLab customers primarily use the Git repo functionality and the CI module. The other modules are still being built out to be market-competitive, including some good work in security. I suspect it may take a few acquisitions to give the company market-competitive solutions in some of the other areas it is building out. At the end of the day, you have to make GitLab a clear favorite to be one of the survivors/winners, especially with such a high valuation.
    
    CloudBees – The Jenkins train has pulled CloudBees to the top of the mountain, but to succeed in this race, the company is going to need more than that to stay there. Jenkins is a bit long in the tooth and Kubernetes has been a game-changer. Jenkins itself has been turned over to the Continuous Delivery Foundation (Linux Foundation) as has JenkinsX, which works very well in K8s environments. But CloudBees has shown it is more than Jenkins and made some key acquisitions including Electric Cloud, which have given the company great enterprise customers and added to its talent pool. Other, smaller acquisitions have given CloudBees some fresh new functionality such as feature flags, for instance, which are upgrading and refreshing the mix. At the end of the day, though, CloudBees’ potential in this market is based on its ability to execute on its vision of a dual-product lineup that features both software delivery management (SDM), which they have already announced, and upgrading their existing software, which helps people with their CI/CD pipelines. Having both working together could give CloudBees an unbeatable place in the market. This vision may not be as ambitious or broad as GitLab’s but could be more attainable, more quickly.
    
    JFrog – Shlomi Ben Haim and his “Frogs” have built a DevOps powerhouse by understanding developers’ needs and wants and using open source as a market-penetration weapon as well as or better than anyone else. The company’s community/customers are fanatical in their support and the “change the world with liquid software” culture it has built is contagious.
    When you consider JFrog started with a repository for artifacts and what it built on top of it is pretty amazing. The company continues to add new repositories for many of the popular languages in use as well. The downside for JFrog is that it just added its JFrog pipelines via acquisition and is still working to make it a best-in-class solution. XRay is JFrog’s security product for testing artifacts and code prior to deployment, and I think it will probably make another acquisition to bolster its security chops. The Frogs have the energy and will to win this thing, but they have a lot of work ahead of them to compete with an end-to-end or complete DevOps vision. Rumor is a new version of the platform is on the way; we will have to see what this brings. The company may wind up being bought for billions by a larger player (GCP, Microsoft), which could make JFrog a winner in any event.
    
    GitHub – GitHub is just a repository, you say? Nonsense. Make no mistake about it, GitHub is going to be a player in this race. With the deep pockets and ambition to continue being a developer powerhouse at Microsoft, the company is determined to make sure GitHub is a complete DevOps solution. GitHub is adding functionality every day and has an amazing array of tools that plug into and extend GitHub. As the de facto standard—with something like 30 million developer accounts already and Microsoft’s Azure and war chest—don’t bet against GitHub being the last man or company standing here.
    
    CollabNet VersionOne/Xebia Labs entity. So, this is the new kid on the block—so new that we don’t even know its name yet. This merger was orchestrated by the private equity group TPG, which bought CollabNet last year and pledged $500 million to use to acquire and grow the CollabNet business into a complete DevOps platform. That $500 million probably represented a lot more than the firm paid for CollabNet, which was a 15 years-or-so-old company that had some good tools in the agile market. Since its merger with VersionOne, it has released a value stream management solution (it may not have been called that, but that’s what it is) and tried to market the whole thing as a suite. Obviously there was not a smooth path between the companies and some parts were missing; the street had the TPG transaction at less than $100 million. Some of its holes are getting filled with the Xebia Labs merger/acquisition. Rumor on the street was that TPG paid something like $226 million for Xebia—not a petty amount, but remember, Xebia raised about $100 million a while back. Again, the rumor is that a good chunk of that money was used to “buy” Xebia out of its ownership by the Dutch Xebia group. After paying that off, the current Xebia ownership used the remaining funds to try and keep Xebia competitive. You can do the math from there. In recent times we haven’t heard a lot from Xebia. It was assumed the company would either raise more money or look for a merger to gain enough girth to compete in this newly energized complete DevOps market. This deal seems to fit the bill.A real plus for this new entity is the addition of Ashok Reddy as CEO. Reddy is a seasoned player with large public company executive experience in the DevOps space, as well as GM of enterprise divisions. However, it is going to take more money and more functionality to create a market competitive entity in this space. There has to be a security module, for instance. The issue is with the inflated prices that companies in this space are garnering—what can be bought with the dollars left or what can be built holistically? Will TPG perhaps put more money on the table? It’s too early to count this new player out, but as the new kid on the block, it has to play catchup against some of the other players.
These are just five of the players. As I mentioned, Chef, one of the original DevOps darlings, also wants to play in the complete DevOps space. There are others lined up behind Chef, but we have to end this article somewhere. It is going to be an interesting few months as we see the winners in this space emerge.

In the meantime, if you like this kind of analysis—no-holds-barred, tell it like it is, open and free—check out Accelerated Strategies Group at https://accelst.com. ASG is the new analyst firm MediaOps is working with, and it is doing to analysis what open source is doing to software. Knowledge wants to be free.

— Alan Shimel
January 27, 2020
The Risks and Potential Impacts Associated with Open Source

Open source software (OSS) is built by communities of developers who contribute their knowledge and time to OSS projects they find appealing. That code can then be used by individuals, communities and organizations in their software products—the only obligation they have is to play under the rules of the license with which the OSS project was published.

This type of knowledge sharing brings many benefits to OSS users as it speeds up software development time and can help companies become more competitive in the market. Unfortunately, there is also a catch. Those benefits also come with certain risks which every OSS user needs to be aware of and take necessary actions to mitigate.

The OSS License

One specific risk to consider involves the OSS license. Not knowing what your obligations are under the license (or not abiding by those obligations) can cause an OSS user to, for example, lose intellectual property or experience a monetary loss.

Operational Impact on the Product

Another risk is the operational impact that the OSS in use will have on the product. It’s important to understand whether the community is consistently managing the OSS project and its issues in a timely manner, and that your organization has implemented the latest version and deployed patches have been applied. Using out-of-date, unpatched or unmaintained OSS could impact your release capabilities, product quality and customer trust.

Security Vulnerabilities

The third risk to consider when using OSS is that of security risk. As open source is software—and in many cases very complex software—it brings with it the possibility of introducing software security vulnerabilities into your source code. Some vulnerabilities can be introduced into your projects through dependencies that the OSS your organization is using has on other OSS projects. Additionally, vulnerabilities could be unintentionally introduced by developers as they code a functionality using open source. The reason for this is that software developers, whether professionals or hobbyists, oftentimes do not consider security while coding. This has a substantial impact on their work and their contributions to open source projects.

It’s not necessarily that developers intentionally skip the implementation of security procedures. Secure coding is a specific discipline in software development that guards against the accidental introduction of vulnerabilities—a discipline that must be learned.

Understand that many developers are self-taught or have been trained in schools and universities (where we are only recently seeing courses in software and application security emerging). Many use their software development skills as a hobby, to contribute to OSS projects or to pursue their own personal projects. Unfortunately, training programs, programming courses in schools and online tutorials often neglect to convey this knowledge or to stress the importance of building secure software.

It’s also important to note that many organizations find it difficult to introduce a secure coding discipline into their software development process. In other words, this struggle persists beyond open source projects. If you’re with an organization experiencing this pain point and unsure where to begin, you’ve come to the right place. There are several strategies that can be used to begin your secure coding journey.

Training

By giving developers the education required for them to become security aware, you’re setting them up for consistent, secure development success. There are a variety of in-person and computer-based training options on the market.

Software Composition Analysis (SCA) Tooling

Implementing an SCA tool will uncover incorrectly coded strings or vulnerabilities that have been introduced in code or OSS components. Not only can SCA tools assist developers in identifying potential software risks, some SCA solutions can help to train developers by offering remediation guidance or short training videos on how to avoid such vulnerabilities in the future.

It’s essential for software issues to be discovered early because the deeper into the process they’re identified, the more time-consuming and costly they are to remediate. As such, in recent years there has been a push to shift left, introducing security into the product development life cycle from the very beginning.

Companies and individuals need to also understand that this security mindset doesn’t start and end in the coding phase of the development process. A security mindset needs to be a part of every phase: from design to deployment to maintenance, all the way through to the product’s end of life.

Emphasizing a security mindset also has a long-term, broad reaching positive effects on the whole software development ecosystem, including the OSS world. As developers are trained to code securely (through university courses and other training institutions, coding portals or professional development through their employer) they pass this knowledge on to the projects with which they’re involved. This then leads to the production of higher quality proprietary and open source software, which in turn minimizes risk across the board.

— Boris Cipot

January 27, 2020
GitLab Updates Core CI/CD Platform

GitLab this week delivered an update to its namesake continuous integration/continuous delivery platform that promises to make pipelines more efficient and easier to manage, in addition to making it easier to set up pipelines in Windows environments.

Version 12.7 of the core GitLab platform adds support for Parent-Child Pipelines and Pipeline Resource Groups. Parent-Child Pipelines make it possible to run separate child pipelines concurrently, while Pipeline Resource Groups enable DevOps teams to limit pipeline concurrency to better manage jobs and resources.

A Code Review Analytics module is designed to make it easier to find Merge Requests in review that require some level of intervention, while enhanced Merge Request widgets show when changes have made it to a specific environment.

Other new capabilities include being able to automatically stage all changes with the GitLab integrated development environment (IDE) and the ability to share group access with another group.

Finally, GitLab has added a beta release of Windows Shared Runner, which brings Windows environments to the GitLab platform via a managed service. Darren Eastman, senior product manager for Runner at GitLab, said instead of having to separately download agent software through which jobs on a GitLab pipeline are executed, Windows environments are becoming full-fledged citizens of the GitLab platform.

Eastman said IT organizations running Windows compared to their Linux brethren are still playing a little catch-up in terms of DevOps maturity. However, with the arrival of microservices based on Docker containers on Windows, the need to embrace best DevOps practices to manage application development and deployment is becoming more apparent, he said.

Of course, Microsoft is pushing its own rival CI/CD platform in the form of Azure DevOps, formerly known as Visual Team Studios. GitLab is betting that most organizations will prefer to consume a fully managed CI/CD service that spans both Windows and Linux platforms, especially as hybrid cloud computing gains traction in the months and years ahead.

Regardless of the CI/CD platform employed, the overall size of the DevOps community should expand considerably in 2020 as more organizations that rely on Windows to build and deploy applications embrace DevOps’ core principles. Naturally, competition between Microsoft and providers of rival CI/CD platforms will be fierce. Microsoft in many ways is a behemoth that was constructed mainly by a loyal base of developers that the company can be expected to guard jealously. However, as Microsoft has embraced Linux alongside Windows in the cloud, the number of Windows developers who have been exposed to alternative platforms has increased.

In the meantime, organizations that have teams that build and deploy applications on both Windows and Linux will have some cultural challenges to overcome should they decide to standardize on a common set of DevOps processes. Rather than managing each platform in isolation, there’s a clear opportunity to reduce costs and increase application development efficiency. The issue now is determining which DevOps platform might best enable them to begin a journey that is likely to take a few years to fully complete.

— Mike Vizard

January 24, 2020
Launchable Aims To Improve Delivery Velocity, With Kohsuke Kawaguchi and Harpreet Singh

A new company, Launchable, aims to improve your delivery velocity by helping you be much smarter about which tests you run and when. By applying machine learning (ML) and better use of the data output by your software development work, the two co-founders think they can make a meaningful impact on development efforts.

Sure, you say, lots of developers say they can change the world. But in this case, the co-founders have some very impressive résumés to give you confidence they just may do it. The dynamic duo is none other than Harpreet Singh, former GM of Atlassian BitBucket and VP of Product and Design at CloudBees, and the one and only Kohsuke Kawaguchi, founder of Jenkins and CloudBees Chief Scientist, most recently.

Wait a second, you may be asking, you’re saying KK from Jenkins/CloudBees has started a new company? Yes, that’s what I am saying. In a podcast that will be available very soon, KK and Harpreet talk with us about Launchable’s why—and why now. Stay tuned for that.

It’s worth noting that KK has been at CloudBees for 10 years and working on Jenkins for 15 years. In the tech business, that is nearly a lifetime. During our podcast, KK said he felt that with Jenkins being passed to the Continuous Delivery Foundation (CDF), coupled with a healthy group of leaders within the Jenkins community, now was a good time to move on to the next adventure.

Beyond Jenkins, CloudBees has grown into its own next phase, with lots of exciting developments announced and more coming. The scientist in KK has led him to explore solutions to problems he sees in the way so many companies are doing development. His longtime friendship with Harpreet, dating back to their days at Sun, has energized them both to once again work together and tackle the real problems that development teams are facing.

What will come out of Launchable? I don’t know. But the company has already attracted some blue-chip VC investment, as well as an impressive list of angels including Sacha Labourey, CEO and founder of CloudBees.

They (like everyone else, it seems) are hiring development engineers and data scientists. So if working with KK is something that appeals to you, head over to the Launchable site and check out the Careers section.

I am sure we will be hearing more about Launchable in the days, weeks and months ahead. In the meantime, best of luck to Harpreet and KK, and many thanks for a job well done with Jenkins.

— Alan Shimel

January 23, 2020
CircleCI Extends Orbs Ecosystem for CI/CD

CircleCI announced today it has added integrations in collaboration with 20 partners to the family of connectors, known as Orbs, that it makes available for its continuous integration/continuous delivery (CI/CD) platform.

Thomas Trahan, vice president of business development for CircleCI, said integrations are now available with Heroku, AWS, Cloud Foundry, Google Cloud, Red Hat, Helm, Kublr, Alcide, Azure, DeployHub, Fairwinds, VMware, Kubernetes, Rafay Systems, Pulumi, Convox, Spinnaker, realMethods, Quali, Pantheon, LaunchDarkly and Salesforce.

Since launching Orbs a year ago, CircleCI claims there are now more than 13,000 organizations using Orbs across nearly 40,000 repositories and 9 million CI/CD pipelines. There are also more than 1,200 Orbs provided by more than 116 partners listed in the CircleCI registry. In the future, Trahan said the company plans to begin surfacing recommendations concerning which Orbs to employ based on the nature of the workflow being constructed.

Trahan said Orbs are gaining traction because they reduce the amount of custom code that needs to be written to set up workflows across a CI/CD pipeline. Without the existence of Orbs, DevOps teams are forced to rely on application programming interfaces (APIs) that are more liable to break, he said.

Competition between CI/CD platform providers is expected to become fiercer as adoption of CI/CD platforms continues to accelerate. A report published by Data Bridge Market Research this week forecasts demand for continuous integration (CI) tools alone will grow to $18.9 billion by 2025, representing a 19% compound annual growth rate. While the current CI/CD market is dominated by open source platforms managed by internal DevOps teams, CircleCI is betting interest in accessing a managed CI/CD service delivered via the cloud will rise as organizations opt to devote more resources to building applications versus managing the platforms they are built on.

A recent CircleCI study of more than 30 million workflows spanning 1.6 million jobs runs per day on its platform found 80% of all workflows finish in less than 10 minutes. Those jobs spanned 150,000 projects launched by more than 40,000 organizations.

The report noted 50% of those organizations start six workflows per day across their projects and that 50% of CircleCI projects never had a failure during pipeline changes made over the 90-day period when the study was conducted. When a failure did occur, 50% recovered in less than an hour, with 25% of organizations recovering in 15 minutes or less, according to the report.
As organizations embrace best DevOps practices more broadly, CircleCI is betting these kinds of metrics will become more important as a platform differentiator.

Naturally, it’s too early to say how the CI/CD platform wars will play out in the months and years ahead. But the amount of code organizations will need to deliver at increasingly faster rates is only increasing. The challenge organizations are facing now is finding a way to deliver all that code at an unprecedented level of scale.

— Mike Vizard

January 22, 2020
Red Hat Extends Runtimes Middleware Portfolio

Red Hat has made available the latest instance of Red Hat Runtimes, a suite of lightweight open source components and frameworks that makes it easier to discover the middleware most appropriate for building a specific type of application.

James Falkner, product marketing director for Runtimes at Red Hat, said as organizations embrace cloud-native application architectures based on microservices it’s become increasingly challenging to determine what middleware to deploy optimally and where. Red Hat Runtimes not only makes it easier to navigate all those options, Falkner said, but all the components and frameworks are certified to be pre-integrated.

The latest version of Red Hat Runtimes adds support for Open Liberty, a lightweight runtime for building Java applications developed by IBM. Red Hat was acquired last year by IBM.

The latest installment of Red Hat Runtimes also provides access to what has become a standard distribution of Node.js based on Red Hat Software Collections that will be implemented across all Red Hat offerings.

Finally, Red Hat has also added support for Red Hat Single Sign-On, an access control framework based on the Keycloak project.

Falkner said Red Hat Runtimes also serves to reduce the total cost by providing a single subscription through which organizations can access everything from container runtimes to the 3Scale application programming interface (API) management middleware, which Red Hat acquired in 2016.

Very few IT organizations these days are standardized on a single class of middleware, especially as they embrace microservices-based applications built using containers. Most IT teams for the foreseeable future will need a flexible way to access multiple runtimes in support of everything from monolithic applications based on Java to applications built using Node.js code deployed using Docker containers.

Rather than a piecemeal approach to the middleware infrastructure required to build, deploy and integrate all those applications, Falkner said Red Hat Runtimes provides a more holistic approach.

It’s not clear how much or even whether IT organizations are centrally managing middleware in the age of DevOps and the cloud. Historically, many IT organizations preferred to standardize on a single application server platform to deploy various classes of middleware. However, many developers today are deciding what middleware to include within their application stacks without much input from a central IT function.

Via Red Hat Runtimes, Red Hat is trying to make a case for providing developers with the flexibility they cherish in the form of a subscription service that provides IT teams with a modicum of control over what middleware gets employed.

Tensions between DevOps teams and central IT organizations have arguably never been higher. In fact, many DevOps teams are operating within lines of business that are outside the direct control of centralized IT. As such, the level of standardization that can be imposed by central IT organizations has become more constrained. Many centralized IT organizations are now trying to influence middleware platform decisions rather than outright make decrees that might get ignored anyway.

Regardless of the middleware platform choice that ultimately gets made, however, the one thing that most centralized IT teams will have to come to terms with is finding a way to support it.

— Mike Vizard

January 21, 2020