devopsy.kinsta.cloud

Author: Mike Vizard

With YAML Support, Puppet Dives Deeper Into DevOps

Puppet this week announced an update to Puppet Enterprise that adds additional DevOps capabilities, including integration with Bolt, an open source agentless task runner that Puppet employs to make IT automation more accessible to IT administrators.

Matt Waxman, vice president of products at Puppet, said adding integration with Bolt will make it easier to integrate DevOps processes that rely on YAML to deploy applications.

Available this month via a Puppet Enterprise 2019.1 update, Bolt provides an agentless approach to automation that enables IT teams to execute existing commands and scripts written in any language within the context of an IT automation framework that is designed to be accessible to the average IT administrator, Waxman said.

At the same time, Puppet is providing the ability to set up pipelines for deploying automation code directly from within Puppet Enterprise. Rather than having to set up a separate continuous integration/continuous deployment (CI/CD) platform, Puppet is embedding the capability it acquired from its purchase of Distelli in 2017. In doing so, Puppet is making it possible to accelerate deployment of automation software in a way that doesn’t require every change to be approved by a master Puppet administrator. Default pipelines guide organizations new to continuous delivery practices through setting up their first pipeline in less than 30 minutes.

Puppet is also including an Impact Analysis capability that enables teams to assess the impact of a proposed Puppet code change before it is deployed.

One of the issues that has held back adoption of DevOps practices has been the simple fact that most IT administrators have limited programming skills. By making available an IT automation framework that provides the ability to manipulate YAML files at a higher level of abstraction, it should become easier of organizations to incorporate IT administrators into any set of DevOps process they care to define, Waxman said.

Puppet via this upgrade is also extending the reach of the agentless Bolt framework to include support for network devices running Cisco NX-OS and Cisco IOS, as well as firewalls from Palo Alto Networks. As network operations continues to be more integrated with DevOps processes, IT organizations increasingly need to be able to apply workflows across compute, storage and networking platforms, Waxman said.

The challenge many IT organizations now face in terms of achieving that goal is each provider of a platform creates their own automation framework, which winds up creating islands of automation across the enterprise. Longer term, Waxman said that, in addition to invoking the application programming interfaces (APIs) that many platforms already expose, Puppet will be working toward integrating domain-specific automation frameworks with its cross-platform framework.

Overall, Waxman said Puppet remains committed to providing a mix of agentless and agent-based approaches to IT automation based on the nature of the IT tasks that need to be automated. It will be up to each IT organization to determine when to employ one of those options. But regardless of the path chosen, Puppet is clearly trying to make its core automation platform a lot more accessible to everyone in IT.

— Mike Vizard

May 3, 2019
Wavefront Provides Map for Microservices

VMware announced it is adding a service mapping capability to the Wavefront 3D Observability platform it provides with the Wavefront by VMware monitoring tools.

Stela Udovicic, senior director of product marketing for Wavefront, said Wavefront Service Maps will make it easier for IT organizations to visualize what will soon be thousands of microservices running highly distributed computing environments, including those based on containers running on Docker Engine, connected via service meshes such as Istio or spanning proxy servers such as Envoy.

Capabilities provided by Wavefront Service Maps include packaged visualization of request flows with a map of service dependencies, built-in tracking of health metrics and view into individual traces to pinpoint performance bottlenecks.

Accessed as a set of software-as-a-service (Saas) applications, Wavefront by VMware relies on an agentless approach to monitoring that supports all the major public clouds along with providing integrations with platforms such as Kubernetes, GitHub, Slack, PagerDuty, Kafka and Pivotal Application Service.

Udovicic said by making it easier to visually identify and troubleshoot microservices, IT organizations should be able to accelerate their transition to modern cloud-native applications. Building and deploying a single application based on microservices is complicated enough. Trying to support hundreds of those applications is simply too daunting without the ability to visualize all the interdependencies between those services. Troubleshooting any issue then becomes easier because Wavefront Service Maps is designed to enable DevOps teams to drill down to the traces being employed to monitor the application, she said.

Earlier this year, Wavefront added support for distributed tracing and machine learning algorithms that also make it easier to identify anomalies within a microservices-based application.

In terms of adoption of cloud-native applications, Udovicic said there are basically two classes. The first are early adopters who are already highly committed to developing cloud-native applications. The bulk of the enterprise IT community, however, is still in the early stages of building and deploying these types of applications, said Udovicic.

There’s naturally a high correlation between organizations that have adopted cloud-native application development using microservices aggressively and those that have implemented best DevOps practices. In fact, one of the benefits of a visualization tool is that it makes it easier to bring new members on to those teams. Rather than having to piece together what is occurring in the environment, a new member of a DevOps team can visually inspect each microservice they might be responsible for managing.

It may be a while before DevOps processes become the dominant default method for managing IT. But as DevOps tools become easier to use, the number of DevOps practitioners should increase steadily. Arguably, one of the reasons DevOps adoption is not as far along as it might be is that the learning curve is still too steep. But as visualization capabilities gets added to the analytics frameworks baked into those tools, it’s now only a matter of time before the number of IT professionals that might be intimidated by DevOps processes starts to decline steadily.

— Mike Vizard

May 1, 2019
Dell Technologies Drives Automation Deeper into Platforms

Dell Technologies this week launched a series of initiatives intended to provide IT organizations with multiple deployment options for Dell EMC appliances and servers, including forthcoming managed service offering through which Dell will manage private cloud platforms based on VMware software on behalf of customers.

Announced at the Dell Technologies World 2019 conference, Dell EMC also unveiled a new appliance series and revealed an alliance with Microsoft under which its VMware arm will make available cloud software on the Microsoft Azure cloud, in addition to existing support for Amazon Web Services (AWS), IBM Cloud and servers deployed on-premises.

Jon Siegal, vice president for product marketing for converged platforms and solutions at Dell EMC, said Dell EMC plans to make it easier for IT organizations to consume IT infrastructure in an on-premises IT environment as a service or continue to use their capital budgets to acquire servers and appliances.

To advance that latter approach, Dell EMC is launching VxFlex, a set of appliances that include an operating system that enable compute, storage and networking separately. In contrast, the existing VxRail series of appliances are optimized for VMware and assume compute, storage and networking will be scaled in tandem, said Siegal. Dell EMC is also now unifying a set of hyperconverged appliances and converged infrastructure servers under a common VxFlex brand, and Siegal said in the future those offerings will be extended to add support for Kubernetes alongside platforms from vendors other than VMware.

At the same time, Dell EMC is previewing a VxRail Analytical Consulting Engine (ACE) that will be embedded in VxRail HCI system software that Dell EMC provides in its appliances optimized for VMware. That analytics engine employs machine learning algorithms to maximize application performance, while also providing self-optimization recommendations and capacity forecasts.

The Dell Technologies Data Center-as-a-Service offering, meanwhile, is scheduled to be generally available in the second half. It marks the first time Dell has launched a managed service under its own brand. VMware, however, has been providing managed services for the past year. The Dell Technologies Data Center-as-a-Service offering extends that concept to include Dell EMC VxRail appliances.

Automation is clearly at the heart of the evolving Dell Technologies strategy. The challenge organizations that have embraced DevOps will face is aligning their processes with the automation frameworks Dell Technologies is embedding in both VMware and the underlying infrastructure on which VMware gets deployed. To address that issue, Dell Technologies over time will bundle frameworks such as the open source distribution of the Pivotal platform-as-a-service (PaaS) environment from Pivotal Software, an arm of Dell Technologies, on its platforms, Siegal said.

These moves signal Dell Technologies is trying to fulfill a promise to make it as easy to manage IT infrastructure deployed in an on-premises IT environment as it is to manage on a public cloud. There’s obviously still a gap. But as Dell Technologies starts to make IT infrastructure available as a service, it may one day become challenging for DevOps teams to tell where any one cloud happens to be running at any given moment.

— Mike Vizard

April 30, 2019
VMware Drops Managed Cloud Onto Dell EMC Appliances

At the Dell Technologies World conference, VMware today announced it will make available an instance of its managed cloud platform on appliances from Dell EMC in the second half of this year.

VMware Cloud for Dell EMC is a turnkey platform that combines VMware compute, storage and networking software in a single offering managed and updated by VMware on behalf of its customers. It is similar to VMware Cloud for AWS, a managed service that VMware provides for Amazon Web Services, said Wei Wang, director of product marketing for VMware.

VMware Cloud for Dell EMC represents the first commercial instance of Project Dimension, an initiative launched last year that promised to make available a software-defined data center (SDDC) as a managed service for on-premises IT environments in an effort to “ruthlessly automate IT.” In addition to supporting VxRail appliances from Dell EMC, VMware plans to make available instances of Project Dimension on other platforms, Wang said.

Pricing for the managed service initially will be based on a flat monthly subscription, but Wang said VMware plans to provide a metered option that will enable customers to pay for VMware Cloud for Dell EMC on a pure consumption basis.

In the meantime, VMware Cloud for Dell EMC, coupled with VMware Cloud for AWS, will provide IT organizations with a consistent control plane across a hybrid cloud computing environment based on vSphere, vSAN and NSX software that virtualizes compute, storage and networking, noted Wang. VMware is already managing infrastructure running more than 25 million workloads on AWS, a number that is expected to increase substantially as it rolls out instances on Project Dimension on additional public clouds including Microsoft Azure and on-premises IT environments.

By opting to rely on VMware to manage IT infrastructure, organizations should be able to shift more of their own internal IT resources toward developing and deploying more applications. The VMware approach also potentially reduces the scope of the operational challenge many organizations now face when trying to manage virtual compute, storage and networking software in tandem.

The challenge those organizations will face is implementing a set of best DevOps practices that match the cadence of the updates being made to the IT environment by VMware. It’s not clear to what degree organizations will want to relinquish operational control over IT infrastructure to VMware. Historically, roughly a quarter of all IT is consumed as a managed service usually provided by an IT services firm, but as vendors such as VMware begin to offer those managed services, the number of organizations that might switch to the model may increase.

Many organizations already rely on cloud service providers such as AWS to manage infrastructure on their behalf. This latest offering essentially extends that same concept to on-premises IT environments.

The degree to which organizations will save money by relying on a managed service is going to be difficult to quantify. In some cases, relying on a managed service might prove more expensive, while in other cases the savings on IT administrators alone might pay for the service. Whatever the path chosen, the number of workloads that will running on IT infrastructure managed by vendors such as VMware will increase sharply in the months and years ahead.

— Mike Vizard

April 29, 2019
Mirantis Adds Modeling Tool to Configure Private Cloud

At the Open Infrastructure Summit conference, Mirantis today unfurled a software-as-a-service (SaaS) application through which IT administrators can employ modeling tools to configure private clouds based on either OpenStack or Kubernetes.

Boris Renski, chief marketing officer for Mirantis, said rather than having to manually configure a private cloud for an on-premises IT environment using YAML files, the Model Designer for Mirantis Private Cloud enables IT administrators to configure a private cloud using modeling tools they access via a graphical interface (GUI). The Model Designer is scheduled for general availabilility in May.

That approach then makes it feasible for IT administrators with little to no programming expertise to quickly spin up private clouds within the context of a DevOps environment using a set of declarative tools, added Renksi.

Those tools also enable IT administrators to specify the degree of configurability they require. A basic configurability level automatically populates most of the cluster settings to a default set of pretested values. The most advanced “ultraviolence” configurability setting enables IT administrators to adjust virtually every aspect of their on-premises cloud. The resulting configuration models are then fed into DriveTrain, which is the automation framework that Mirantis provides within Mirantis Private Cloud.

Renski said requiring IT administrators to master all the nuances associated with managing YAML files to stand up a private cloud is one of things holding back adoption of both on-premises private clouds and the best DevOps practices that should be relied to mange them. Most organizations that want to make the transition to DevOps clearly want to be able to manage infrastructure as code in much the same way public cloud service providers do today. The challenge is, the learning curve associated with achieving that goal is simply too high for the average IT administrator, he said, adding a modeling tool made available as SaaS application will make it possible for even mere IT mortals to easily spin up a private cloud within an on-premises IT environment.

Both OpenStack and Kubernetes suffer to varying degrees from often being too complex for the average IT organization to set up. Because of those complexity issues, adoption of both platforms often is limited to IT organizations that have advanced IT skills. But as the deployment processes surrounding both platforms become more automated, it’s probable the number of organizations adopting either platform will increase. At the same, IT organizations that do embrace either platform more likely will spin up more instances once they no longer have to rely on a cumbersome manual set of processes.

Of course, setting up a private cloud based on either OpenStack or Kubernetes is only the beginning of the IT challenge. Once all those private clouds are deployed, IT organizations need to acquire the tools and expertise required to manage them on an ongoing basis. In fact, one of the things IT organizations should consider in the age of the private cloud is the level of integration of the tools from vendors they intend to rely on to set up and then manage a private cloud.

— Mike Vizard

April 29, 2019
Sauce Labs Acquires Screener Visual Testing Tool

At the SauceCon 2019 conference, Sauce Labs today revealed it has acquired Screener to add a visual testing tool to its portfolio of automated testing tools.

Lubos Parobek, vice president of product for Sauce Labs, said the addition of Screener will make it possible for DevOps teams to not only test functions and performance using existing Sauce Labs tools, but also now confirm things such as the colors of a visual element of an application are being rendered correctly, for example.

In addition to building their own visual tests, DevOps teams can employ Screener to automate the running on those tests and store baselines in the cloud to create a visual testing life cycle.

At the same time, Sauce Labs announced the general availability of Sauce Headless, which leverages container technologies to make it possible to leverage a cloud service through which developers can now run tests via a Chrome or Firefox browser every time they commit code. Parobek said the goal is to make it easier for organizations to embrace continuous testing as an extension of the pipelines constructed using a continuous integration/continuous deployment (CI/CD) platform.

The acquisition of Screener, he said, is part of an ongoing Sauce Labs effort to extend the reach of its testing portfolio beyond functional testing. Sauce Labs earlier this month launched Sauce Performance, its first set of application performance testing tools.

The rise of DevOps is challenging traditional approaches to application testing as organizations move to combine various types of testing into a single continuous process. To meet that requirement, Sauce Labs is expanding its portfolio of automated testing tools at a time when the emergence of microservices-based applications is significantly increasing the level of dependency between all the components that make up an application. That increased level of dependency will require DevOps teams to rely more on automated testing tools. In fact, Parobek said the day is not far off when DevOps teams will include digital assistants that leverage machine learning algorithms to enable DevOps teams to continuously run those tests at the right moment in an extended DevOps pipeline. Those digital assistants will not replace the need for human testers anytime soon, but they will augment their capabilities to make is feasible to implement continuous testing at scale, he noted.

In the meantime, as application testing continues to shift further left in the DevOps process, more developers will be building and running their own tests during earlier stages of the application development process. At some point, either another developer or dedicated tester still will need to evaluate that code to provide an unbiased opinion. But the number of common mistakes making it into the latter stages of the application development process should be substantially reduced. Ultimately, that shift to the left should result not only in more development projects meeting deadlines, but just as significantly higher levels of overall application quality. The challenge now, of course, is figuring out how to implement a best DevOps practice that pushes application testing as far to the left as possible.

— Mike Vizard

April 24, 2019
Harness Invests in Automating Continuous Delivery

Fresh off raising an additional $60 million in funding, Harness plans to make more investments in machine learning algorithms to advance continuous delivery through automation.

Harness CEO Jyoti Bansal said the startup, which he co-founded with Rishi Singh, a former DevOps platform architect at Apple, is now in a position to invest more aggressively in additional engineering and sales resources needed to expand adoption of the cloud-based service, which provides a level of abstraction through which much of the continuous delivery process can be automated using drag-and-drop graphical tools. Bansal and Singh co-founded Harness after AppDynamics, provider of an application performance management (APM) platform Bansal also co-founded and served as chairman of the board, was sold to Cisco Systems.

Bansal said Harness takes advantage of YAML files and machine leaning algorithms to present DevOps teams a set of visual tools accessed via a cloud service that enables them to build sophisticated pipelines as quickly as 10 minutes. That approach promises to significantly advance DevOps adoption by making it easier for IT operations teams to set up those pipelines as part of continuous delivery process without needing deep programming expertise.

While there’s been a lot of progress made in terms of driving adoption of best continuous integration practices among developers, the continuous delivery side of the DevOps equation has proved much more difficult to achieve. Most IT organizations rely on multiple platforms that have their own unique attributes. As such, IT operations teams typically have created their own scripts or rolled out automation frameworks such as Ansible to allow them to automate some aspects of the application deployment process.

However, Bansal noted, custom scripts typically don’t scale very well and usually have not been well-documented. What’s more, frameworks such as Ansible are still too low-level to enable DevOps teams to build pipelines in minutes. The approach Harness has designed enables DevOps teams to describe their intent using a set of visual tools, which then employs machine learning algorithms to automate the deployment process regardless of the underlying platform, he said.

That approach is further augmented by a set of rules that can be added to make sure security and compliance requirements are met, he noted, adding Harness also provides views into how processes are being automated to provide DevOps teams with the confidence required to trust a highly automated platform.

Bansal said the goal is to not just make it simpler to build and deploy pipelines, but also maintain them in what have become highly dynamic IT environments where there is simply too much data that needs to be tracked for human administrators to effective manage relying on low-level tools and frameworks. Rather than having to build and deploy the framework needed to achieve that goal, he said, Harness is making it possible to automate the continuous delivery process using a cloud service on demand.

It may take a while for IT organizations get comfortable with the level of automation being enabled by machine learning algorithms. But like it or not, it’s already becoming apparent the ability to dynamically manage IT at scale is clearly on the cusp of making another significant leap forward.

— Mike Vizard

April 23, 2019
CloudBees Acquires Electric Cloud to Advance Continuous Delivery

CloudBees announced it has acquired Electric Cloud as part of an effort to extend the reach of its DevOps portfolio further into the realm of continuous delivery and application release automation (ARA).

Announced at the CloudBees Days event in San Francisco, the deal follows CloudBees’ acquisition of Codeship, a provider of a cloud-based continuous integration/continuous development (CI/CD) platform, last year.

CloudBees CEO Sacha Labourey said both deals are part of an effort to start rolling up many of the smaller companies providing DevOps tools that have not been able to achieve the critical mass required to operate at scale. As organizations of all sizes realize how much they depend on software to differentiate their company from competitors, he said, they are looking for a single vendor that can address all their requirements end to end. At the time of the acquisition, which closed earlier this month, Electric Cloud had 110 employees.

Labourey added the acquisition of Electric Cloud is also notable because it extends the reach of the CloudBees Jenkins and Jenkins X CI/CD platforms to the realm of application delivery for both legacy applications and emerging modern applications based on cloud-native microservices architectures.

While application development teams have largely embraced continuous integration, getting IT organizations to successfully implement continuous delivery has proven more challenging. The next phase of DevOps, Labourey said, will need to focus on applying machine learning algorithms to automatically discover the environment in which an application is to be deployed. Armed with those insights, it should be much easier for organizations to continuously deploy and update applications across diverse IT environments, he noted.

As DevOps continues to evolve, it’s apparent that legacy categories such as application release management are going to be subsumed into larger software development lifecycle frameworks based on best DevOps practices. The debate many organizations are currently engaged in is determining how much they need to change their IT culture before acquiring the tools and platforms required to successfully adopt DevOps. However, Labourery said this “chicken and egg” debate overlooks the importance of getting the tools to the teams that need to make the transition to DevOps occur. Otherwise, IT leaders will find themselves advocating for a theoretical cultural change without creating the opportunity for anyone to gain hands-on DevOps experience, he noted. In short, there’s no substitute for on-the-job DevOps training.

A consolidation of DevOps vendors is underway. In some cases, those acquisitions will be driven by leaders in a category, such as CloudBees. In other cases, providers of software development tools commonly used in legacy application environments will need to extend their reach further into the realm of DevOps. IT leaders, going forward, will need to consider carefully just how and where they want to place their bets, at a time when DevOps teams tend to add and discard DevOps tools, often with little to no regard for any larger IT strategy.

For more insight on the CloudBees/Electric Cloud acquisition, check out Alan Shimel’s video conversation with Sacha Labourey.

— Mike Vizard

April 18, 2019
XebiaLabs Injects Predictive Analytics Into DevOps Platform

XebiaLabs is moving to take the guesswork out of DevOps following the launch of a set of tools that leverage machine learning algorithms to predict when and how the release of an application is most likely to go awry.

The Risk Prediction Module being added to the XebiaLabs DevOps platform comes on the heels of XebiaLabs’ move to make its XL JetPack software for automating application deployments available on the Amazon Web Services (AWS) Marketplace. XL JetPack is an automation framework based on YAML files and a declarative programming framework that enable DevOps teams to push applications into production on a cloud platform in as little as 15 minutes.

XebiaLabs CEO Derek Langone said that while DevOps advancements have been made in terms of achieving continuous integration, the goal of achieving continuous delivery has proven to be more elusive. As automation frameworks such as XL JetPack become more accessible, however, IT operations teams will be better able to keep pace with the rate at which organizations want to be to deploy and update applications, he said.

It’s becoming increasingly apparent that a combination of automation frameworks and machine learning algorithms will enable DevOps teams to make a significant leap forward in terms of application release lifecycle management. Because of all the nuances of the different platforms that IT organizations are trying to leverage, it’s become a major challenge to roll out an application manually. As the number of applications that need to be released at roughly the same time increases, managing the release cycle process is moving beyond the capabilities of an IT team made up of mere mortals.

The Risk Prediction Model from XebiaLabs promises to increase the odds of success by applying predictive analytics infused with machine learning and other proprietary XebiaLabs algorithms to identify DevOps bottlenecks before they are encountered. Capabilities provided by that module include alerts that warn the team when a release is likely to be delayed or to fail before the release pipeline starts running; a Risk Forecast view that summarizes predicted delays and failures for every task in the release process; an overview of statistics for similar releases to provide historical context; and forensics tools that uncover flaky automated tests, problematic build setups, long-running deployments and time-consuming manual tasks. That Risk Prediction Module is meant to complement the risk scoring capability that XebiaLabs already includes in its platform.

Arguably, the biggest barrier to DevOps adoption today continues to be the level of expertise required to succeed. But as the combination of predictive analytics infused with machine learning algorithms and algorithms that can discern instantly how changes to the IT environment will impact applications become more common, the level of expertise required to master DevOps processes should decline. As that transition occurs, the number of organizations embracing DevOps more broadly should increase.

In the meantime, DevOps teams would be well-advised to consider to what degree they might next want to rely on predictive analytics to prescriptively automate processes with eye toward eliminating the need for as much human intervention as possible.

— Mike Vizard

April 17, 2019
CloudBees Allies with Atos to Accelerate DevOps Adoption

In a move that should accelerate adoption of best DevOps practices in the enterprise, CloudBees and Atos announced this week that the two are collaborating to advance adoption of continuous integration/continuous development (CI/CD) platforms on the Google public cloud.

Announced at the Google Cloud Next 2019 conference, the alliance will result in the IT services giant delivering a managed instance of a CloudBees CI/CD platform running on the Google cloud, in addition to providing traditional consulting services.

Francois Dechery, chief strategy officer for CloudBees, said the alliance with Atos will accelerate adoption of DevOps processes in traditional enterprise IT organizations as Atos consultants engage customers looking to modernize their application development and deployment processes. In fact, many of the principles first pioneered by proponents of agile development methodologies are now being applied to business process management, he noted.

Atos plans to add a DevOps practice as a complement to its existing processes. Michael Kollar, senior vice president for cloud engineering, said the company expects that, for the foreseeable future, most of their clients will be trying to meld some degree of DevOps and existing ITIL-based approaches to IT services management (ITSM). The alliance with Google will go a long way to accelerating that transition because it will allow Atos to deliver access to a CI/CD platform based on a consumption model that reflects the way public cloud infrastructure is consumed, he said.

Kollar noted enterprise IT organizations are under more pressure than ever to deliver applications faster as more organizations recognize they are a software company that happens to make something. Most organizations today need to create a superior customer experience through software to differentiate themselves from their competition, he said.

Many of those companies will need help setting up the processes that will enable them to function as a software company, he added. The biggest challenge, however, is finding a way to meld new ways of building and deploying applications with investments in legacy systems on which the business still depends, he said.

Naturally, Atos won’t be the only IT services firm to launch a managed DevOps process. But as more IT services firms invest in developing managed services, there will be need for a balance between the managed services provided by IT services providers and cloud service providers such as Google and the IT services that will continue to be managed and delivered by internal IT organizations. The degree to which organizations will rely on external IT services firms for DevOps will be dictated by how mature they are today. But it does not follow that an organization which sees a need to rely on external expertise today will be that position tomorrow. As more organizations develop internal expertise, a managed service provider will need to earn its seat at the table every time its contract comes up for renewal.

— Mike Vizard

April 12, 2019