devopsy.kinsta.cloud

Author: Mike Vizard

Survey Finds Mixed Progress on DevSecOps
A survey of 5,558 IT professionals published today by Sonatype in collaboration with CloudBees, Carnegie Mellon’s Software Engineering Institute, Signal Sciences, 9th Bit and Twistlock finds 27 percent of organizations have mature DevOps practices in place, while another 48 percent are still working on improving them.

Despite the DevOps works that remains to be accomplished, however, the survey also noted that 47 percent of respondents still manage to deploy code multiple times a week.

The survey, announced at the RSA Conference, also finds that 25 percent of respondents now consider security to be a fundamental element of quality assurance process. But responses among the 23 percent of respondents who identified themselves as developers suggest implementing best DevSecOps practices remains a significant challenge. For the third year in a row, about half of developers reported they don’t have enough time to address cybersecurity issue.

The survey, however, also noted that organizations that do implement best DevSecOps practices see substantial benefits:
- Elite DevSecOps practices are 700 percent more likely to have fully integrated and automated security practices across the DevOps pipeline.
- Almost two-thirds (62 percent) of respondents with elite programs have an open source governance policy in place that is improved via automation.
- Organizations with elite DevSecOps practices are three times more likely to provide application security training to developers.
- A full 81 percent of those with elite practices have a cybersecurity response plan in place.
- More than half (51 percent) of respondents with elite practices leverage automated security products to identify vulnerabilities in containers.
Brian Dawson, DevOps evangelist at CloudBees, said the survey results show that despite the gains being made by some organizations, there is much work to done in terms of overcoming technical hurdles and cultural tribalism. For example, 35 percent of all respondents said their No. 1 challenge in application security is that they find out about the problem too late in the process. Nearly half (46 percent) of organizations without a DevOps practice do not have application-level credentials encrypted. Half of respondents (50 percent) with a cloud infrastructure said they rely on their cloud service provider to secure their IT environment, and 28 percent of respondents admitted they do not protect secrets such as passwords, API keys and certificates.

The core issue most organizations need to focus on first is to firmly establish a DevSecOps mindset, said Dawson. Much progress has been made in driving adoption of DevOps practices. Now those principles need to be extended to include automating the implementation of security controls as applications are being built and deployed. The survey noted that 63 percent of organizations that have elite DevOps practices are now informing developers of cybersecurity issues directly within the tools provided to the DevOps team. Armed with that information, it becomes much easier to incorporate security fixes within the overall application lifecycle management process.

It may be a while before best DevSecOps process are widely implemented. But as cybersecurity issues continue to be incorporated within the quality assurance process, it’s now only a matter of time before applications become much more secure.

— Mike Vizard
March 4, 2019
Akamai Extends Scope of DevOps Reach

Akamai today moved to further extend the number of capabilities it exposes via its content delivery network (CDN) that are designed to appeal to DevOps teams.

Those enhancements include HTTP/2 performance optimizations and the ability to automatically load font libraries, as well as providing access to “middle-mile” logs that will enable DevOps teams to more precisely investigate performance issues and test logic in a local development environment before moving those changes into production.

Akamai has also re-engineered Control Center to provide more detailed reporting across security, acceleration and delivery functions, and made TLS 1.3 the standard mechanism for establishing connections.

Finally, Akamai is also looking to advance DevSecOps processes. Its CDN now can optimize or block access to content within a fully integrated virtual private network and domain name system proxy detection service. Akamai’s Token Authentication capabilities have been extended to include coverage for browsers and devices that do not support cookies.

Ari Weil, vice president of product marketing for Akamai, said these latest extensions are part of an ongoing effort to make the company’s CDN a natural extension of any DevOps process. Developers today often wind up wasting a lot of time and effort writing code to load fonts that now can be automated using via the Akamai CDN, for example. The enhancements should free them to write code that adds more value to the business, he said.

Akamai is also promising DevOps teams they will see much faster activation times across Akamai CDN, as well as have deeper visibility into application performance issues beyond what most application performance management (APM) tools offer, added Weil. Akamai is especially focused on any application development project where service level agreements need to be both achieved and maintained, he said.

Thus far, Weil said Akamai is finding a higher level of appreciation for the role CDNs can play in DevOps. CTOs and enterprise architects who are typically held responsible for achieving application delivery goals in particular are now taking notice of CDNs’ capabilities, as most individual developers aren’t usually focused enough on finding ways to automate the routine aspects of building a modern web application, he said.

Akamai will continue to focus on making its CDN a natural extension of the continuous integration/continuous deployment (CI/CD) platforms on which DevOps processes are implemented, with an eye toward making its CDN a preferred deployment platform for web and mobile applications. It may take while longer before DevOps teams view CDNs as a deployment platform for their applications. Much of the decision regarding what CDN to employ historically has been been left to application owners. But Akamai is clearly betting that if more applications take advantage of CDN functionality as they are being built, there’s a much greater likelihood those applications will be distributed on the Akamai technology.

— Mike Vizard

March 4, 2019
Atlassian Adds Pre-Configured Pipelines to Bitbucket CI/CD

One of the primary reasons DevOps processes have not been widely adopted across the enterprise is that building pipelines across all the tools required is a painstaking effort. As a result, most pipelines today are constructed using manual pipelines. However, Atlassian has announced it is now making available pre-configured Bitbucket Pipelines that DevOps teams can access via a simple drag-and-drop interface.

Harpreet Singh, head of product for Bitbucket Cloud at Atlassian, said DevOps teams that rely on the Atlassian Bitbucket continuous integration/continuous deployment (CI/CD) platform will be able to employ a series of pre-configured Bitbucket Pipelines built by Atlassian or one of its third-party partners, rather than having to manually build every pipeline.

The goal is to make it much easier for DevOps teams to incorporate, for example, a new cloud platform into an existing set of DevOps processes without waiting for someone on the DevOps team to write, test and validate a script, said Singh. IT vendors that have worked with Atlassian to create Bitbucket Pipelines include Google Cloud, Microsoft, Slack and Amazon Web Services (AWS).

One of the ironies of DevOps is that while CI/CD platforms make it easier to build and deploy applications, constructing the CI/CD platform can be exceedingly complex. A recent survey conducted by Atlassian found DevOps teams rely on 4.3 tools to move code from development into production environments. A full 62 percent of respondents reported they resort to performing manual testing due to insufficient automated test coverage, additional manual processes or a lack of build/deployment pipeline automation.

While some organizations might have the engineering resources required to build pipelines, he noted, the average IT organization can’t afford to hire operations specialists that have the requisite programming skills required to build pipelines manually.

Best of all, noted Singh, the pre-configured Bitbucket Pipelines that Atlassian makes available will be maintained and updated by their original authors.

It’s too early to say to what impact pre-configured pipelines is likely to have on DevOps adoption in the enterprise. But most enterprise IT organizations won’t significantly alter their process until it becomes relatively easy to make that transition. Enterprise IT organizations have relied on administrators to manage IT for decades, and most of those administrators wouldn’t know where to begin if asked to manually construct a pipeline. The few enterprise developers who might be able to build those pipelines are simply to busy writing application code. The result is a paralysis that prevents organizations from modernizing their application development and deployment processes.

Of course, any meaningful change in the enterprise also involves a chicken-and-egg debate. Lacking skills and tools, enterprise IT organizations are unable to overcome internal inertia, which in turn results in conservative IT organizations preferring to rely on known processes no matter how inefficient they might be. The real challenge facing IT leaders trying to advance DevOps adoption is to remove as many excuses for resisting change whenever and wherever possible, starting with how pipelines are constructed and maintained.

— Mike Vizard

February 28, 2019
VMware Brings NSX Network Virtualization into DevOps Realm

At the Mobile World Congress (MWC) 2019 conference, VMware moved to bring network virtualization into the realm of DevOps by adding support for a declarative programming model and JSON data formats to version 2.4 of NSX-T.

In addition, VMware is making it possible to use the open source Ansible automation framework to deploy NSX-T.

Finally, VMware is adding support for IPv6 as well as the ability to now extend microsegmentation all the way to Layer 7 of the application stack.

Jonathan Morin, product marketing manager for NSX at VMware, said this release of NSX-T not only will become the dominant version of NSX for VMware in the enterprise, it also represents a concerted effort to make network virtualization directly accessible to both developers and IT operations teams alike. JSON support is critical from that perspective because it makes it easier to programmatically replicate the networking requirements and security controls attached to any class of applications.

Longer-term, JSON support will also make it easier for a wide variety of management tools to consume metrics generated by NSX-T.

IT organizations must determine to what degree they want to fold network operations into DevOps. Some organizations will simply prefer to provide developers with access to self-service virtual networks, while others will move to expose the declarative application programming interfaces (APIs) in NSX-T 2.4 directly to developers, said Morin.

VMware expects that, going forward, a hybrid method for deploying and managing NSX will emerge. Developers will want to programmatically invoke network resources as they build and deploy applications. But network administrators will need to be able to access tools to manage those virtual networks over time. To facilitate that process, the latest version of NSX-T has been updated to add support for an HTML5-based user interface. The goal is to not just distribute NSX-T, but also make it accessible to everyone, said Morin.

Rather than requiring organizations to upgrade switches and routers to enable programmability, VMware has been making a case for employing network virtualization to create a programmable overlay. That approach serves to make the underlying physical network agile enough to be employed with a DevOps framework without requiring organizations to upgrade the entire physical network.

In many ways, networking represents the last major frontier to which DevOps processes need to expand. It’s possible for developers to spin up a virtual machine in the cloud or in an on-premises environment in seconds. But it can still take weeks to manually provision the networking resources in an on-premises environment that a virtual machine requires. VMware has been advocating for the deployment of instances of NSX in both on-premises IT environments and in public clouds to provide a consistent network overlay that can be programmatically invoked and automated.

It remains to be seen to what degree network operations will be pulled into DevOps processes going forward. But the one thing that is clear is that IT organizations have the option to decide to what level they might want to achieve that goal.

— Mike Vizard

February 27, 2019
FinOps Foundation to Help Rein in Cloud Costs

A new consortium has been formed to promote best practices for controlling cloud computing costs.

Announced at the Cloud Economic Summit, the non-profit FinOps Foundation counts among its founding members Atlassian, Autodesk, Cloudability, HERE Technologies, Just Eat and Nationwide.

Cloudability co-founder J.R. Storment said the goal is to bring financial accountability to variable cloud spending that can easily spiral out of control if not monitored carefully. The adoption of best DevOps practices makes IT organizations more agile, but not every organization has the financial controls in place to ensure spending on cloud services is aligned with business outcomes, said Storment.

These days engineers can use their corporate credit card to spin up virtual machines in the cloud. The FinOps Foundation is making a case for more accountability over the spending.

Gartner estimates worldwide cloud spending will reach $360 billion by 2022, which is five times the compound annual growth rate for overall IT spending. The unfortunate truth, however, is that most finance organizations don’t have a lot of visibility into how those dollars are being allocated.

To coincide with the launch of FinOps, 451 Research also published a survey of 300 IT and finance executives in the United States and the U.K. that finds more than half (57 percent) of large enterprises worry about cloud costs daily, with 80 percent admitting poor cloud financial management has had a negative impact on their business.

In fact, the 451 Research report makes it clear most finance departments have little visibility into cloud spending. Almost three-quarters (72 percent) of finance executives acknowledged they have no formal reporting capacity between departments. More than half (51 percent) of respondents admitted to occasionally overspending and 68 percent said they are only alerted to overspend too late. A full 82 percent of finance respondents rely on cloud vendors, spreadsheets and manual processes to track cloud costs.

Despite much of the variable cost hype surround cloud computing, 73 percent of the finance respondents in the United States and 81 percent in the U.K. still treat cloud as a fixed capital expense rather than an operating expense.

To help finance teams better understand how cloud spending is being allocated, the FinOps Foundation will provide access cloud economic research and related articles, FinOps training and meetup groups. Early access to the forthcoming “Cloud Financial Management Strategies, Creating a Culture of FinOps” book published by O’Reilly Media will also be provided.

As IT organizations increasingly employ multiple cloud platforms to run different classes of workloads, understanding the true cost of choosing to use one cloud over another will become more important. At the same time, IT organizations have a greater appreciation for the need to move workloads between platforms as costs rise. Just because an application workload was created in the cloud doesn’t mean that workload should be deployed there. Many long-running applications are less expensive to deploy in an on-premises IT environment. The challenge IT organizations arguably face going forward will be integrating streams of real-time financial data into DevOps processes to make more economically informed decisions about how to allocate their cloud resources.

— Mike Vizard

February 26, 2019
Kemp Adds Predictive Analytics to ADC to Advance DevOps

Kemp this week enhanced the automation and predictive analytics capabilities it makes available within its application developer controller (ADC) as part of an effort to ease deployment of applications across multi-cloud computing environments.

Company CEO Ray Downes said the goal is to make it easier for DevOps teams to avoid overprovisioning IT infrastructure, at a time when applications are more distributed than ever.

The Kemp ADC offerings are comprised of a mix of physical and virtual appliances that have now been deployed more than 60,000 times. The volume of data generated by those deployments now makes it possible for Kemp to apply predictive analytics via a 360 Central monitoring tool that, for example, can now make suggestions concerning where to deploy a workload based on current utilization rates, Downes said.

Kemp also now can provide comparative analytics, he noted. For example, the company revealed that its analytics show 61 percent of application experience issues are related to events caused by capacity issues such as failing or degrading application servers. Kemp also revealed that 48 percent of organizations see as many as nine application incidents per week.

Now that Kemp has begun to apply predictive analytics to its ADC platforms, Downes said the next logical step will be to apply machine learning algorithms to help further automate application deployments.

While ADCs provide critical load balancing capabilities, they are often overlooked as a source of actionable intelligence for DevOps teams. That data is more critical than ever because as IT organizations modernize their application portfolios by embracing more complex microservices, many of them are discovering every application will soon require its own load balancing capability. Previously, the primary role of an ADC had been to enable as many applications as possible to share the same IT infrastructure efficiently. Now the goal is becoming to orchestrate and optimize the allocation of IT infrastructure resources across a range of loosely coupled microservices. The Kemp ADC is designed to enable DevOps teams to define a desired state of all the load balancers in the distributed IT environment, including F5 Big-IP, AWS Elastic Load Balancer, Nginx, and HAproxy load balancers. That capability then makes it easier to roll out distributed application services.

Over time, Downes said Kemp will extend that capability to open source load balancers such as Envoy, which are starting to be deployed on top of Kubernetes clusters. Both Kubernetes and Envoy are being developed under the auspices of the Cloud Native Computing Foundation (CNCF). As critical as load balancing will be within a Kubernetes cluster, however, Downes noted distributed applications will span cloud-native and legacy IT infrastructure environments as well as multiple Kubernetes clusters.

It may take a while for DevOps teams to fully sort out what form of ADC platform and load balancing capability to employ when and where. But as IT becomes more distributed, there’s rising interest in making sure IT infrastructure utilization rates for those applications remain as high as possible regardless of how they have been developed.

— Mike Vizard

February 21, 2019
Survey: DevOps Progress Slow, but Steady

Atlassian has published the results of a survey of 500 IT and software professionals that suggest now is the best and worst of times when it comes to DevOps.

On the plus side, the survey found that more than half of respondents (57 percent) are encountering fewer bugs and outages after deploying a continuous integration/continuous deployment (CI/CD) platform. Nearly half (47 percent) of those who have adopted a CI/CD platform also report they make changes and receive customer feedback faster. A full 71 percent who have embraced microservices have also discovered it is much easier to test and deploy new software features, according to the survey results.

However, the survey found three-quarters (75 percent) of development teams face issues with bugs, defects or delays when it’s time to release. A total of 73 percent of software development teams also reported spending 10 percent to 50 percent of their time on updates and upgrades to self-hosted software. On average, according to the survey results, teams rely on 4.3 tools to move code from development into production environments.

More frustrating, 62 percent of respondents reported they resort to performing manual testing due to insufficient automated test coverage, additional manual processes or a lack of build/deployment pipeline automation. However, among those organizations that have adopted feature flagging to ease management of the application development process, 63 percent report better testing a feature and higher overall rates of software quality.

The metric IT organizations are employing to measure DevOps success appears a little soft. A full 93 percent of respondents consider customer satisfaction to be the most important DevOps metric, but 60 percent admit it’s difficult to measure customer satisfaction for the features that have been developed.

Sean Regan, head of growth for software teams at Atlassian, said that while there is clearly much work still to be done, the democratization of DevOps continues unabated. Rather than being limited to a few high IT practitioners, Regan said DevOps processes increasingly are being pushed into the realm of everyday enterprise IT operations. At the same time, many line of business executives are being included within those teams to ensure the software being developed achieves the desired business outcome, noted Regan.

IT organizations are also starting to apply more engineering principles to the management of IT infrastructure, added Regan. In many cases, IT organizations are discovering a single engineer can be more efficient than 10 systems administrators. However, finding and retaining those engineers is a major challenge. The goal needs to be to create a set of DevOps processes defined by an engineer that are simple enough to be implemented by IT administrators, said Regan.

Of course, many of those IT administrators today are schooled in less agile ITIL processes. But as ITIL continues to absorb some of the concepts and best practices pioneered by the DevOps community, the divide between engineers and traditional IT administrators should become that much more narrow to bridge.

— Mike Vizard

February 21, 2019
New Relic Accelerates AIOps with SignifAI Acquisition

New Relic, following its acquisition of SignifAI, is in the process of accelerating its investments in machine learning algorithms and other forms of artificial intelligence (AI) as part of an effort to make it easier to separate the signal from the noise created by all the alerts in an IT environment.

Aaron Johnson, senior vice president of product management at New Relic, said SignifAI represents an opportunity for New Relic to create a superset platform that can apply advanced analytics to a much wider range of data sources. SignifAI has already created more than 60 integrations spanning open source and commercial monitoring tools that are widely used across DevOps toolchains, he noted, adding that as New Relic moves to add other data sources, there’s an opportunity for the company to distinguish itself beyond providing software-as-a-service (SaaS) tools for monitoring applications and infrastructure.

It’s obviously still early days for AI in the realm of DevOps. But it’s also clear that with the rise of microservices and containers, soon it will be impracticable for IT organizations to manually manage their IT environments. Various forms of automation based on AIOps platforms infused with machine and deep learning algorithms will be required. As a provider of SaaS-based tools for monitoring applications and infrastructure, New Relic already has access to vast amounts of data required to train AI models on optimizing a specific IT environment.

Of course, there’s lots of competition across the IT monitoring category, so it remains to be seen which vendors will have the resources required to drive continuing investments in AIOps.

In the meantime, DevOps teams will need to adjust their processes to take advantage of AIOps. It’s unlikely AIOps platforms will eliminate the need for DevOps teams, but the number of applications and platforms any DevOps team must manage effectively will increase substantially. Rather than replacing humans, AIOps platforms will reduce the pressure to hire additional IT staff when professionals with DevOps skills are already in short supply. The AIOps platform, for all intents and purposes, acts like a co-pilot.

The biggest issue IT organizations are likely to encounter along that journey is establishing trust with any AIOps platform they implement. It takes a while for most AI models to learn the IT environment before that platform makes any reliable recommendations. In most cases, DevOps teams will have to dedicate staff to train and update AI models continuously as the IT environment changes. It may take years, however, before DevOps teams are comfortable enough to move beyond trusting a recommendation and enable an AIOps platform to automate processes on its own. After all, the only thing worse about being wrong about any IT decision is being wrong about it at scale.

AI at this point is all but inevitable in DevOps environments. But it doesn’t necessarily follow that just because something is based on AI it always will be smarter than experienced DevOps professionals.

— Mike Vizard

February 20, 2019
IBM Adds Automated Testing Tool for Mainframe to DevOps Portfolio

IBM has added an automated testing tool to the portfolio of DevOps tools it has developed for mainframes.

Sanjay Chandru, director of Z DevOps for IBM, said IBM Z Open Unit Test adds a continuous testing capability to the continuous integration/continuous development (CI/CD) platform based on an instance of an open source Git repository developed by Rocket Software that IBM launched last year. The automated unit testing tool automatically records and plays back test data for both batch-oriented applications written in COBOL or PL/I and CICS transaction processing applications.

Chandru said IBM Z Open Unit Test is unique in that it eliminates the need to employ traditional debugging tools. Developers can generate test cases for COBOL CICS programs, as well as “stub out” CICS to achieving environment independence from CICS runtime during test case execution. A stub is an object that holds predetermined data, which is provided in response to calls made during tests. That approach eliminates the need to have objects that answer with real data. That means Z Open Unit Test operates without adding the overhead of a debugging tool, said Chandru.

Any test case and the resulting program created then can be stored in any source code management (SCM) system.

IBM is making a concerted effort to make mainframe a natural deployment target for any DevOps team, said Chandru. It increasingly has become apparent that millennials in particular aren’t biased toward one platform or another, as long as each platform is equally accessible, he noted, adding that as organizations embrace best DevOps practices to modernize their application portfolio, IBM wants to make sure the mainframe remains relevant.

IBM launched the z14 mainframe in the fourth quarter of 2017. According to the company, 92 of the world’s 100 biggest banks use its mainframes, which handle most global credit card transactions alongside billions of ATM transactions and annual passenger flight bookings. While use of mainframes overall isn’t growing, the platform isn’t going away anytime soon.

IBM has also enhanced the cryptographic capabilities of the z14 to make it one of the most secure platforms available and continues to invest in an instance of Linux for the mainframe. That latter capability makes the mainframe a more natural target for applications being developed and deployed using DevOps processes, said Chandru.

Of course, the biggest challenge when it comes to DevOps and mainframes is not necessarily the technology. Most mainframe applications have been developed using waterfall processes for decades. Getting those teams to embrace DevOps processes represents a significant cultural hurdle for any IT organization.

At the same time, however, DevOps teams who ignore mainframes are not taking advantage of an existing platform investment that has performance and security capabilities that are still unmatched after more than five decades. The real challenge is finding a way for old mainframe dogs to learn some new DevOps tricks while exposing proverbial whippersnappers to the capabilities of the most powerful mainstream platform ever built.

— Mike Vizard

February 19, 2019
Survey Finds Wide Gap Between DevOps Adoption and Success

A recent HBR survey shows there is work to be done to advance DevOps in organizations

A survey of 654 subscribers to the Harvard Business Review found that while 86 percent of respondents said it is important for their organizations to be able to build and deploy software quickly, only 10 percent said they are successful at it.

Conducted by Harvard Business Review Analytic Services on behalf of Google, the survey found nearly half (48 percent) of respondents said their organization always relies on DevOps methodologies when deploying software, while another 21 percent rely on DevOps selectively. An additional 8 percent said their organization plans to adopt DevOps practices.

Among those respondents employing DevOps practices, the top benefits cited were increased speed to market (70 percent), productivity (67 percent), customer relevance (67 percent), innovation (66 percent) and product/service quality (64 percent). Despite those benefits, however, most organizations still have a significant amount of work to do in terms of implementing best DevOps practices—the three biggest hurdles organizations face are organizational silos (50 percent), legacy technologies (49 percent) and resistance to change (46 percent).

Melody Meckfessel, vice president of engineering for Google Cloud Platform (GCP), said these issues are precisely why Google is sharing how it operates in the form of site reliability engineering (SRE) principles, which are a prescriptive set of guidelines for implementing best DevOps practices. Not every organization needs to implement those principles to the same level, but just about every organization can benefit from adopting a set of well-defined engineering principles, said Meckfessel.

The challenge many organizations face is the fact that most of the IT staff are not engineers by trade. Most IT staffs are made up of administrators who tend to have limited programming skills. But Meckfessel said even IT organizations that don’t necessarily have a lot of engineering expertise still can benefit from automating operations based on the SRE principles.

Many of those organizations will need to slow down the rate at which they are building applications today to be able to speed up tomorrow. It takes time for organizations to master the cultural change required to implement DevOps practices. But once they begin to master DevOps, the benefits to the organization compound quickly. Meckfessel said two of the most important attributes of DevOps is adopting an immersive approach to training and establishing a no-blame culture within the IT organization when it comes time to conduct post-mortems. That latter goal is especially difficult to achieve sometimes, because most interactions between developers and IT operations teams historically have been less than positive.

Meckfessel also noted it’s critical now for organizations to incorporate cybersecurity into its DevOps processes as well, which requires not only additional training for developers but also closer alignment with cybersecurity teams.

It’s now only a matter of time before the complexity associated with emerging technologies such as microservices based on containers force many organizations to dive deeper in DevOps. The real challenge now is charting a path of least resistance that also generates the maximum level of benefit for all concerned.

— Mike Vizard

February 15, 2019