With the beta release of the next version of Red Hat Enterprise Linux (RHEL), Red Hat is setting the stage for making it easier for DevOps teams to consume emerging operating system services without having to upgrade their entire operating system.
The beta release of RHEL 8 adds an Application Streams capability designed to make it possible to leverage userspace more flexibly, as modules running in userspace now can be updated more quickly than the core operating system.
Matt Micene, the technical product marketing manager for RHEL 8, said it’s also now possible for multiple versions of the same package—for example, an interpreted language or a database—to be made available via an application stream. That means different DevOps teams will be able to more easily work with different versions of various classes of technologies without having to wait for the next upgrade of the operating system, he said.
As part of that focus on flexibility, Red Hat is also moving to make it easier for organizations that have embraced microservices based on containers to build applications. Open source tools such as Buildah for container building, Podman for running containers and Skopeo for sharing and finding containers are now fully supported in the operating system. Red Hat is also extending support for container networking using IPVLAN, which enables containers nested in virtual machines (VMs) to access networking hosts. There’s also a new TCP/IP stack with Bottleneck Bandwidth and Round-trip propagation time (BBR) congestion control. As organizations increasingly adopt containers, many of them are starting to make operating systems decisions based on which platform has the most containerized optimized services built in.
Other capabilities being added include support for OpenSSL 1.1.1 and TLS 1.3 along with Systemwide Cryptographic Policies; a Stratis file management system for dealing with lots of volumes; a Composer tool for building custom images that can deployed on hybrid clouds; support for the Yum 4 packager; and a new web console for managing the RHEL environment that presents administrators the same user interface experience regardless of their experience.
Micene said Red Hat is trying to strike a balance between stability and IT agility. There are plenty of organizations that still prefer to upgrade their operating systems at very well-defined intervals. But there are also an increasing number of organizations that want to be able to take advantage of the latest IT innovations without having to wait for an operating system upgrade. Many of those organizations increasingly view IT as being core to their ability to compete, he said.
Over time most organizations will develop a more schizophrenic approach to IT. There will be some application workloads where stability trumps all other requirements. At the other end of the scale, innovation will be the most prized attribute. To one degree or another, every workload is going to fall along a spectrum between those two extremes. The challenge is defining the right mix of DevOps processes to serve the needs of every and any workloads regardless of its attributes.
When software development became a cumbersome task of repetitive coding, testing and glitches, organizations had to come up with a solution.
The solution? An organized approach that would not only eliminate the endless cycle of repetitive tasks but also would establish proper coordination and execution in the software development life cycle (SDLC).
That very systematic approach with fewer glitches, more production and exemplary coordination today is called DevOps.
But where did this wonderful process start? How did it make its pathway into our the cycle of IT industry?
The Beginning
It all began one day when Andrew Clay Shafer scheduled a session, “Agile Infrastructure,” at the Agile Conference in Toronto.
However, the DevOps model back then didn’t seem to make much of a difference to those associated with software development. As a result, Shafer closed the session. But something struck developer Patrick Debois that day, who was presenting a paper on “Agile Infrastructure and Operations” in the same vicinity. Shafer’s theme seemed to draw his attention and he went straight to meet Shafer.
A long talk in the hallway that day ignited the fire of DevOps, which started with Shafer and Patrick three months after their first meeting.
Together, they started the Agile Systems Group, with an aim to foster greater collaboration between developers and operators.
Today, the DevOps process continues to survive on the principles of increased collaboration and automation to build, test and release software faster.
Mobile Device and DevOps Adoption
Over the years, DevOps began to grow and pick up fame. And with the emergence of mobile phones, the trend picked up rampant speed.
According to a market intelligence company App Annie, global economic activity concerning mobile applications will be worth USD $6.3 trillion by 2021, five times 2016’s revenue.
Creating software for mobile devices requires an elevated software development approach. Now, organizations have to deliver new, high-quality software to customers as soon as possible to remain atop the competitive market.
In this new approach, organizations need to focus on two prime factors: agility and adaptability. While the former defines the ability to code and recode, the other marks DevOps’ ability to include changes along the SDLC.
Companies are constantly working on getting these qualities into their workflows to promote DevOps concepts of continuous delivery, continuous integration and eliminating bugs to allow users to enjoy a hassle-free software experience.
DevOps Over the Past 18 Months
According to Gartner, about two-thirds of organizations were using DevOps to support the certain business process by the end of 2017, which Forrester termed as the “Year of DevOps.”
Forrester awarded the term to DevOps owing to increasing clients’ questions and discussions around, “How do I implement DevOps at scale?” rather than just, “What is DevOps?”
Going by DevOps’ potential to support modern business processes, more and more companies are adopting DevOps, leading to a steady rise in the trend.
As DevOps continues to be successful within and across the organization, the IT industry will continue to see what it looks like to be “DevOps successful.”
The Emergence of DevSecOps
While DevOps continued to change the software industry, however, security within the process still remained an unanswered question.
In 2012, Gartner analysts Neil MacDonald and Cameron Haight introduced the concept of DevSecOps. This meant security’s introduction right from the beginning and continuing throughout SDLC.
Security’s entry into the model made DevOps a more alluring practice. Six years out, DevSecOps has been a much-discussed concept across seminars, conferences and written content.
But it is still a question whether organizations are implementing DevSecOps considering the time constraint, the need for accelerated delivery and the understanding of security’s importance in the cycle.
For many organizations, security is still not a part of their production cycle. But, it is essential for any organization to consider security if they want to remain in the competitive market and importantly at the top.
Negligence in this area may cost them bankruptcy or even elimination from the industry. According to the “Global State of Digital Trust Survey and Index,” one-third of British customers stopped using an organization’s services owing to a security breach.
10 Years of DevOps: What’s Next?
The emergence of AI and automation have made some waves in the IT industry, and DevOps is no exception.
DevOps holds high predictions for the future. In the years to come, AI and automation will change the software development process. Apps will be self-written and coded. Tasks such as integration, security, testing and deploying will be dictated by AI, data and machine learning in the all-new AIOps.
A data-driven approach will have software dynamically adapt its behavior on the basis of what is in its environment, much like other AI-driven technology including Apple’s Siri and Alexa.
Moreover, this significant shift from programmed to non-programmed apps will call for improved security and functionality management and allow developers to focus on value-addition activity.
As DevOps functionality continues to expand, we’re looking at a new vista for freedom of applications. Our apps can be moved around as we need them; we can build-test-deploy on the fly; and we have the ability to monitor whatever is needed. The price we pay is one we gladly take up: implementing new toolchains to drive the transition and getting those toolchains integrated into the entire development and operations process. Yes, we have an entire organizational side, too, but for this blog, we’ll talk about tools.
Why Tools?
Why are we going to discuss tools alone if this is about DevOps? Because it is a different set of problems to move from one organizational structure to another versus moving from one toolset to another. While DevOps is giving us independence from a lot of traditional infrastructure issues, the reality is that it is creating dependencies upon the very tools/toolchains we are using to gain that independence.
Confusing? Let’s try looking at it differently. The new vendor lock-in will be DevOps tools. While they are freeing us from vendor lock-in on several fronts, you are unlikely to make a quick move from, say, Ansible to Puppet or vice versa. It can be done, and some organizations even do a little of both to make sure their options are open. But most shops will pick one and run with it, because that creates a single training and maintenance direction.
Why Now?
I (and others) have alluded to this issue in the past, but it is of growing importance as vendors start to take what should be cross-portable tools and customize them to limit portability. Container management, for example, is available on every major cloud vendor these days, but using the vendor’s bundled container service is convenient and more difficult to move away from. That is but one example of many.
And the importance of these tools/toolchains is growing at the same time that their use is growing. Soon, in a DevOps-heavy shop they will be essential and locked-in as if they were cemented.
So, What Are You Saying?
Pick DevOps tools as if they will be there for decades. This is not how DevOps teams are encouraged to think, but you are building a DevOps infrastructure that, if adopted broadly in the organization, will be difficult to move away from to any significant degree. Just like these days you would need a driving reason to port thousands or hundreds of thousands of lines of code, DevOps scripts are growing and the number of DevOps scripts are multiplying. As you fold security, storage and networking into that environment, and build real-time reporting that you are going to rely on for years to come, make certain what you have is good, if not great. In many ways, DevOps thrives on “good enough” without thought to long-term futures, but the investment required to have a fully developed DevOps environment strongly implies that what an average enterprise is doing today will be what it is stuck with for a good long while. That’s not a place for “good enough” if you know there is better out there.
It’s Not All Unchangeable
Some bits of the environment can change over time, of course. Better data collation/integration technologies no doubt will take over for whatever you are using for reporting now, as well as AI-based solutions that can plow through the growing pile of machine-generated data to get a systemic view of what is happening. But for the most part, build it like you’ll be maintaining it for years to come, because much of the environment you will be.
Cisco Systems is making a concerted effort to create an era of détente between networking professionals and the rest of the IT operations teams that have embraced DevOps practices.
Susie Wee, senior vice president for the DevNet community at Cisco, said the goal is to provide network operations (NetOps) teams the skills required to programmatically expose a range of self-service capabilities to developers.
At the same time, Cisco is encouraging developers to deploy applications directly on routers and switches that now incorporate general-purpose processors. Those processors would run applications in a way that significantly reduces network latency, while the custom ASICs that Cisco develops would continue to handle all the networking functions.
Wee said that dichotomy is now driving Cisco to classify two classes of developers within its DevNet community. The first is aimed at traditional application developers and the second community is more focused on managing infrastructure as code.
Cisco is now extending that latter initiative into the realm of software-defined wide area networks (SD-WANs). The company has added integrated firewall, intrusion prevention and URL-filtering technologies to its SD-WAN platform. In addition, there are code samples, videos, learning labs and sandboxes available on Cisco DevNet that are intended to make easier to learn how to programmatically manage Cisco SD-WANs using open application programming interfaces (APIs).
Cisco claims to now have 530,000 developers participating in DevNet. One the one hand, the company is trying to entice NetOps teams that have been reluctant to give up legacy command line interfaces to modernize their networking environments. On the other hand, Cisco is trying to attract more application developers who increasingly find themselves encountering latency issues as they move to deploy applications in highly distributed computing environments, such as an Internet of Things (IoT) application.
Over time, Cisco and other providers of network operating systems also will be moving away from monolithic architecture to embrace microservices, which would make it possible to programmatically invoke networking services on a more granular level.
Of course, there may be some developers who would prefer to programmatically take control over networking along with the rest of the IT infrastructure. But Cisco is clearly betting most organizations will want to make it easier for their NetOps and DevOps teams to work more closely together. Today, most enterprise IT organizations can spin up a virtual machine in a matter of minutes; provisioning the network connections to that virtual machine, however, is still measured in days and weeks.
Other networking infrastructure companies besides Cisco have realized the need to make networking more programmable. But in terms of the resources the company is making available to solve that problem, it would appear to ready to make a significantly larger commitment.
As DevOps has become more common, these teams are tasked with uniting everything from coding to deploying and monitoring applications. Build Engineering forms the foundation for all those processes and is essential for building code. Its tools vary according to the operating system you’re building on and by programming language. DevOps needs to work with Build Engineering, but the two don’t usually connect. And that’s a problem.
Over the last 10 years, there’s been a shift toward DevOps. Previously there was a discrete division between coding, done by developers and engineers, and deployment, done by Ops. However, it became apparent that for companies to optimize benefits such as productivity, development and operations shouldn’t be so disparate. The result of which is DevOps.
Along with the shift to DevOps have come several trade-offs—namely trading security for speed, certainty for agility and proprietary code for open source:
Speed: Getting to market quickly has become critical to success. Speed now takes greater priority over fixing vulnerabilities, which can always be addressed in the next release (hopefully). And retrofitting for security and vulnerabilities after the fact becomes a big blocker for development and engineering teams. At the end of the day, the risk of lost profits is greater than the risk of potential unknown threats.
Agility: Agility trumps certainty in the race to market. For instance, iterative pushes to production do away with fixed, known road maps in favor of implementing whatever this week’s most important feature/functionality is. The trend toward immutability is also important here; it’s generally simpler and quicker to throw away something that’s broken or outdated rather than trying to fix or update it.
Open source: Due to the greater innovation, reduced costs and transparency it delivers, open source has won over proprietary code. Rather than focusing on creating proprietary code, developers can quickly iterate on a solution by assembling open source code, accelerating time to market. Plus, the ability to take from what has already been created can free up your dev teams to innovate and build IP for your enterprise efforts.
For most, the tension between fixing something and knowing if it will break something upstream hasn’t been eliminated. You may be able to quickly update and patch a single solution, but how will this impact you once you push to your CI stream? Even for mature DevOps organizations, there’s a hidden opportunity cost associated with two factors that are overlooked: open source programming languages and Build Engineering.
Overhead is added to your DevOps teams when you retrofit your open source language of choice with new versions, dependencies, security patches and so on adds. Teams should be focused on the things that matter—creating and delivering the latest release.
On top of that, Build Engineering takes a hit each time the underlying open source language is retrofitted. You need to rebuild all your developer environments, CI/CD systems and production environments. All of this build engineering work is manual, slows down time to market and eats up valuable engineering resources. Plus, you’re not certain your rebuild and update will provide performance improvements that warrant the update.
Since DevOps already relies on automation, it makes sense to apply this tool to the retrofitting and building of your open source languages and seamlessly connect to the rest of your DevOps cycle. You could ensure build reproducibility, dependency management and compliance with your corporate security and license criteria. For example:
Build impact – What if you could understand the performance improvements (if any) that could be gained by taking the latest build?
System update – Consider the benefits of being able to update all of your impacted systems across dev, test and even production environments with the latest build using a single command.
Packaging – If each continuous build could be packaged in multiple ways—any form factor for any operating system—how would it change your workflow?
Build validation – Imagine that every continuous build could be vetted against your smoke test, inclusive of compliance with your security and license criteria, so you knew whether it was ready to deploy.
Continuous builds – What if every time a new version of a library or patch for a vulnerability was found, a new build was kicked off and you were notified when it was done? Then you could keep up with your continuous deployments.
The domino effect is a phenomenon that DevOps must deal with on an all-too-often basis. Dev and Ops came together for greater productivity, but unforeseen dependency issues regularly gum up the works. That’s the reason automation is such a useful tool, one that you can used to stop the domino effect as you automate your build pipeline for open source languages. It’s truly possible today to benefit your business by ensuring build reproducibility, dependency management and compliance with your corporate security and license criteria.
IBM and ServiceNow today announced an alliance under which IBM Global Technology Services (GTS) will collaborate with ServiceNow to make DevOps processes and events managed via the IBM Cloud Migration Factory visible to a cloud-based IT service management platform from ServiceNow based on the ITIL framework.
Jim Comfort, general manager for cloud services at IBM GTS, said that from a governance and compliance perspective it has become clear IT organizations need more visibility in to DevOps processes. In general, enterprise IT organizations are increasingly adopting DevOps processes to accelerate application development and deployment, while still relying on ITIL to manage governance and compliance, he said.
IBM GTS, via the alliance with ServiceNow, will be able to bridge that divide by integrating ServiceNow with a set of tools and services IBM provides across hybrid cloud computing environments that are based on DevOps process.
The IBM Cloud Migration Factory is based on a methodology IBM developed to move applications to cloud. IBM Services claims it has completed more than 100,000 migrations. In addition to allying with ServiceNow, a new toolset is being added to IBM Cloud Migration Factory that employs algorithms based on the migrations IBM has completed that improve planning accuracy and uncover application dependencies. Included in that portfolio is cloud brokering software that IBM gained via the acquisition of Gravitant in 2015.
Comfort said the alliance with ServiceNow will make it possible to have DevOps processes and events register automatically in the change management database (CMDB) provided by ServiceNow. That’s critical because it should allow IT organizations to maintain governance and compliance without having to impose an overhead that otherwise would slow down application development and deployment using DevOps processes, he said, adding there’s no need to impose an artificial construct on top of a DevOps process.
As hybrid cloud computing continues to evolve, Comfort said IBM GTS expects IT organizations will be relying on a mix of ITIL and DevOps processes for the foreseeable future. In fact, he noted that while managing one to ten DevOps processes is relatively simple, keeping track of hundreds of DevOps processes requires a more static set of processes to maintain governance and compliance. The IBM approach is to create a unified approach to integrating DevOps and ITIL processes that doesn’t require organizations to integrate a variety of point products on their own.
Proponents of DevOps processes have been making a case against ITIL processes that they tend to view as being too rigid to support agile application development. Comfort said the next iteration of the ITIL framework will incorporate many of the concepts advocated by DevOps proponents. But the two frameworks will remain distinct.
The alliance between IBM and ServiceNow represents a clear signal that DevOps is going mainstream in enterprise IT environments. The challenge now is finding a way for organizations that have spent more than a decade adopting ITIL processes to embrace DevOps processes with a minimal amount of friction possible.
Emerged out of Agile methods in 2001, DevOps basically represents a fundamental departure from the traditional waterfall software development approach.
DevOps is a modern software development method that focuses mainly on the effective collaboration between software developers, project managers and operations staff, while also aligning with business goals at the same time.
DevOps came into being when developers and operations staff came together to more effectively overcome the challenges associated with continuous software delivery.
DevOps in Mobile Applications
Peerbits takes a holistic view of mobile DevOps, defining it as a capability for continuous mobile app delivery that enables our clients to reduce time-to-market new features and seize new market opportunities simultaneously.
The key principle in DevOps is continuous delivery, which means deploying the software (in this case a mobile app) and the environment on which it runs, automatically and on-demand, during any stage of the mobile app development cycle.
DevOps basically applies the same principles whether you want to build an app for either mobile or web. However, the challenges in mobile DevOps are bit different and specific.
Mobile DevOps Challenges
The challenges in mobile DevOps mainly include:
Different OS Versions: Most mobile apps have multiple target environments and multiple devices, which require dealing with different specs, different OS versions and different form factors.
Furthermore, not only is there a wide range of operating systems running in the market, but there are also different installed versions.
Take Android, for instance: The Android market accounts for almost 85 percent of the smartphone market. The below mentioned chart shows the breakdown of installed Android Versions as of Oct. 26.
Image source: www.developer.android.com
As you can see from the chart, the Android version 6.0, which according to Wikipedia is now unsupported, is still being used by 21.3 percent of Android users.
This clearly indicates that only a few users update as soon as the new version comes along, while others don’t, either because they don’t want to or because their smartphone manufacturers are not providing newer OS updates.
In addition to this, most Android smartphone manufacturers also tweak newer Android OS slightly, resulting into an increase in the OS fragmentation and making the launch times slower.
Hardware: Smartphone hardware presents two primary challenges. First, smartphone manufacturers offer a good range of chipset, memory, storage options and, most importantly, different screen sizes. This makes it difficult to support each of them.
Second, the mobile device war no longer is being fought only between Apple and Samsung. There are new smartphone manufacturers entering market more often than before. Out of these new rivals, there are also many manufacturers that offer low-cost models.
Though Apple and Samsung might be dominating the Western markets right now, in China both of these brands account for only 13.7 percent and 20.7 percent respectively.
In 2015, at least 18 new smartphone manufacturers entered in the market in Asia.
This localization and fragmentation of available devices and their different OS versions adds a considerable complexity to the mobile application development as a challenge.
Balancing Mobile App Quality with Always Changing Consumer Demand: As the app economy grew in last recent years, the need for businesses to get their mobile apps released in the market quickly also grew along with it.
However, since the consumer expectations are also increasing, the quality of a mobile app is the most essential factor in retaining existing customers as well as attracting the new ones.
In 2018 especially, this is more and more difficult, due to the ease in which customers can delete apps and give poor ratings and review to low-quality mobile apps.
As a result, businesses need to be extra careful in terms of their app quality, while consistently adding value to keep their existing customers happy.
All of these factors are putting strain on mobile app development teams, as the waterfall model approach is not a sustainable practice for today’s consumers.
3 Rules for Successfully Adopting Mobile DevOps
Adopting DevOps in your business mobile app needs only three considerations, which are explained below.
Continuous Planning and Integration
Continuous planning refers to bringing your entire mobile app project team—developers, project managers, operations staff, and other stakeholders—on the same platform to finalize the scope of the mobile app in the preparation of a launch plan.
Continuous integration, on the other hand, ensures that the code submitted by one developer works with the code submitted by the other developer(s). Simply put, continuous integration in mobile DevOps demands frequent builds, which must be integrated with the last developed code.
Continuous Testing and Monitoring
Most mobile application testing is executed on simulators instead of actual devices. Plus, the testing is also manual. And given the number of OS versions alone, it is impossible to do manual testing for all versions.
A mobile app can perform well in test environment, but it is very common that it will fail in the live environment.
The reasons of these failures can be many, such as memory, network conditions, power, etc. It is necessary, therefore, for developers as well as businesses to make sure that the continuous monitoring is done via third-party SDKs for log, crash report and so on to find the root of the problem.
Continuous Delivery and Deployment
Continuous delivery refers to the practice that makes sure the code is deployed to the production environment by submitting each change to the production-like environment.
Continuous deployment is the next step of continuous delivery. In continuous deployment, each change that has been approved by continuous testing gets deployed to the production environment automatically.
Takeaway
In our perception, there is no such thing as a separate DevOps for mobile application development. DevOps is a general approach that works for all components as well different kinds of application development.
While there are certain challenges involved in adopting DevOps for mobile apps, but there also measurable benefits that makes the entire journey worthwhile.
All in all, despite the cost to build an app and its initial pains, the benefits of DevOps make the investment worthwhile.
It was 6 a.m. Saturday when the phone rang and awoke the American Express CIO. When the phone rings at 6 a.m., it is never good news. Never. This instance was no different: In this case, a third-party supplier just suffered a breach, and that breach would impact American Express cardholders.
Immediately following that call, the CIO activated the company’s Cyber Crisis Response team. The job of the Cyber Crisis Response team is to help identify impacted cardmembers and prepare to reach out and assist any who have questions or need help regarding the breach.
The need to respond swiftly to customers and provide them with the correct information is essential today, not only to stay on the right side of regulators, but to also help customers avoid fraudulent transactions and identity theft. When it comes to such effective data breach incident response, rarely does DevOps come into play—but the experience American Express shared at the most recent DevOps Enterprise Summit revealed just how better DevOps organizations can be when it comes to effective breach response.
Within hours of the initial call between the CIO and the Cyber Crisis Response team, the primary breach response was divvied into three teams. The first team focused on what data breach bridge teams typically focus on: how to identify impacted customers. The second team was comprised of business and product owners, as well as customer care staff, whose objective was to take the findings from the response investigation and communicate them to American Express customers.
The third team consisted of DBAs and system specialists, who understood all the systems, as well as enterprise architects, who might be able to quickly solve any technical challenges that arose.
Getting to the Right Breach Information
By 3 p.m. that Saturday, the first team determined that it could pull together all of the information it needed to identify affected cardholders. That was the good news. The bad news was that there would be tens of millions of production records that would need to be evaluated to make the final determination.
If the team was to pull those tens of millions of records in production, the demand would start to slow those production systems. “How do we pull these records out of production without impacting our availability? That was the challenge,” said Aimee Cardwell, vice president of Engineering, Consumer Product Development at American Express.
As the day moved forward and the teams worked to find a way to access those production records without impacting availability, one of the engineers on the team pitched an idea that would have been scoffed at in most organizations: Why don’t we clone production? But the novel idea wasn’t immediately rejected. The team on the call began to weigh the pros and cons of cloning production, and concluded that it could, in fact, clone production quickly, and all of the impacted card data could be aggregated without negatively impacting American Express’s servers and associated availability.
After the team successfully cloned the production systems, it worked all night to identify the impacted card members, which required cross-referencing the cloned production system with other data stores, Cardwell explained.
“What was really important here was the comfort level everyone had when it came to bringing up an idea that was really off the wall. And, in the end, they came together to make it happen because together the various teams had a string understanding of the people, technologies, and processes in place necessary to succeed,” said Chad Avery, director, DevOps implementation at American Express.
It was now 6 a.m. Sunday and the teams had worked nonstop since the CIO’s phone rang Saturday morning. The teams managed to evaluate the relevant data from various systems and collected a list of potentially impacted card members. And after a careful analysis, they were able to determine who had been affected by the breach and who hadn’t.
What made this success possible? Both Avery and Cardwell believe that it was the integration of business, product and technical teams. “The fact that we had business, product and technical teams working together in this incident was a huge win for us,” Cardwell said.
Avery contended the ability for an organization to successfully integrate its technical, business and product teams makes the difference. If those teams hadn’t been working together from the beginning, he said, they may not have been able to find a solution and, if they had, it would have taken much more time to do so.
Disclosure Day, and a Final Big Win
It’s a good thing it worked out quickly. By Monday morning, the breached third-party was ready to go public with its data breach disclosure. That announcement would immediately impact not just American Express cardmembers, but those of most other credit card providers as well. Many customers would have pressing questions about whether their accounts were impacted, and, if so, what to do.
Customer service lines had to be ready.
That proved to be the final challenge that needed to be solved: How would customer service representatives know which customers calling were affected? Tight team integration and collaboration proved to be a deciding factor once again.
The engineers on the product team had spent the previous 24 hours building a tool that they shared with American Express customer care professionals. When individuals called the customer care team, the tool could alert the team whether a customer was in the impacted group.
The tool also came equipped with a data collection feature that would enable everyone to learn from the event and even prepare better for next time. “We want everyone to get better. We want ourselves to get better,” Cardwell said. “We want to know what percentage of customers who called were impacted, and what percentage of customers that called were not impacted. We wanted to know our response time on those customers. We wanted to know any other data that would help us to do better next time.”
And it will have been that dedication to collaboration among technical teams, business units, developers, operations and security teams that enabled them to quickly and successfully manage a third-party data breach and prepare themselves to do even better next time. “They all were able to develop a solution quickly. They were able to identify the customer impact, design a solution, build it, test it and get it deployed. And, oh by the way, the teams did it in a weekend,” Avery said.
Despite what you may have heard to the contrary, not everything that happens in Vegas should stay there. Take the DevOps Enterprise Summit 2018 (DOES18), for example: The three-day event attracted the best and brightest in the DevOps world—and me—to discuss all things DevOps. Some of those great conversations were captured in a series of video interviews, which are now available on DevOps TV.
During the two days of interviews in our glass-walled video booth, a variety of folks stopped by to chat about topics ranging from audits and compliance to NextGen Ops and the future of DevOps. Below is a recap of the Day 1 video interviews from DOES18 (Day 2’s recap will post next week):
The Future of DevOps
Wesley Pullen of Electric Cloud, Torsten Volk of EMA and Chris Nowak of FlowStates stopped by to discuss the latest DevOps trends, where DevOps is headed, and understand key emerging practices and technologies that are driving success.
Venture Capital and DevOps
Sid Sijbrandij of GitLab, CJ Napolitano of Electric Cloud and Ubaid Dhiyan of Union Square Advisors chatted about the current state of VC investment into DevOps, how this could evolve in the future and the impact this investment ecosystem has on the global economy.
ITSM/DevOps Crossover
IT Skeptic Rob England and his partner Dr. Cherry Vu discussed what holds ITIL back from more successful transformation of IT work, and how we can help it succeed, as DevOps and ITSM finally converge on each other in the common ground of service operations.
‘Dear Auditor’ — Compliance and DevOps
Sam Guckenheimer of Microsoft, James Wickett of Signal Sciences and Topo Pal of Capital One talked about the current state of DevOps and audit/compliance. They are part of a team that wrote a love letter to auditors from DevOps, in which a promise was made to make life better. They also compiled a list of audit concerns and documented them in a DevOps Risk Control Matrix with lot of details around the controls, practices and evidences that are collected to support the control.
NextGen Ops
Damon Edwards of Rundeck, Cornelia Davis of Pivotal Software and Tom Limoncelli of Stack Overflow chatted about the next generation of IT Ops and what emerging patterns and practices teams should consider for the future of IT.
Workplace Burnout
John Willis (Botchagalupe) is passionate about so many topics around DevOps, cloud and technology. One of the areas he is really passionate about, though, is the topic of burnout and fatigue in the IT field. He and Dr. Christina Maslach discussed the concept of burnout and examined the causes and effects of this problem, and the most promising strategies for dealing effectively with it.
Project to Product
Dr. Mik Kersten, CEO of Tasktop, reviewed the transition from project-based work to product-based workflows—the most optimized and scalable DevOps environment will be a highly prized gem for any business only if the product management team is prioritizing the right capabilities into the DevOps process.
Tech for Tech Leaders
Anders Wallgren of Electric Cloud and Erica Morrison of CSG discussed the technology landscape and what things tech leaders need to not only understand from a tech and architecture perspective, but also what they need to know to help build technology and tech platforms for the future of software delivery.
Banking on DevOps
Mark Howell of Lloyds Banking Group, Shaun Norris of Standard Chartered Bank and John Rzeszotarski of KeyBank reviewed a few of the top DevOps journeys in banking and what other financial institutions should consider along their DevOps journeys.
Well, there ya have it: An entire day’s worth of videos compiled for your viewing pleasure. Be sure to check back next week for a recap of our Day 2 video interviews from DOES18 Las Vegas.
We’re well past software “eating the world.” According to Tricentis CEO Sandeep Johri, software has become irreversibly intertwined with the world. You can’t find a modern enterprise that doesn’t depend on software, or a business transaction that doesn’t require software at some point in the end-to-end process. In the new world of digital business, companies that deliver differentiating customer/user experiences through software have the clear competitive advantage. Those that cannot are living on borrowed time.
In response, every CIO is now focused on digital transformation initiatives that ensure the company is disrupting, not disrupted. From an IT perspective, this requires faster delivery of innovative software and greater agility—the ability to pivot as soon as you identify a new opportunity or challenge.
Everyone agrees that this is a priority. Developers are producing software faster than ever before, and Ops can now deliver applications in a matter of seconds. So, what’s holding us back? Testing.
When Johri talks to CIOs about how their digital transformation is progressing, the same basic story always emerges: Organizations transform Dev and Ops, but then testing cannot keep pace … and they get stuck. The tools and processes architected for traditional months-long release schedules simply don’t fit modern delivery cadences, which require immediate quality feedback with each new build. It’s a sad but simple fact: If you have a slow testing process standing between highly accelerated development and operations processes, there’s just no way that you can achieve the desired delivery speed.
Many companies have already recognized this—and they’ve galvanized their digital transformation initiatives by modernizing their testing. By re-examining and re-inventing software testing across their organization, they went beyond removing the “testing bottleneck”—they’re also driving the positive user experiences that can now make or break a business.
Many of these distinguished quality leaders joined thousands of software testing professionals from around the world at Accelerate 2018 in Vienna, Austria. This year, the conference theme was “Digital Transformation Requires Continuous Testing.” To ensure that the conference thoroughly covered both the digital transformation and continuous testing sides of that topic—as well as the core intersection between them—we invited CIOs as well as quality leaders to share their perspectives. Here are some of the key insights that are most pertinent for executives leading digital transformation initiatives.
Archaic Testing Approaches Can Cause a ‘Brain Drain’
Merck sought out top development talent to advance their digital transformation initiatives. When those developers started quitting, (now retired) Merck CIO Clark Golestani wanted to know why. He learned that the No. 1 reason for these developers leaving the company was the company’s testing practices. Developers were “fed up” because the company was still using traditional testing practices within Agile development processes—and developers believed it was undermining their hard work and dedication.
That’s the precise moment that Merck launched its testing transformation. Ultimately, the testing organization changed the way testing was done, dramatically reducing attrition and enabling faster delivery with superior levels of quality. According to Golestani: “If it weren’t for the testing organization driving that change, they would have been an inhibitor. We would have never been able to release the types of products we do, and the technology would never work the way that it needs to. It’s absolutely essential.”
Agile, DevOps, Cloud and Testing are Inextricably Linked
Agile, DevOps, cloud and testing transformation are all interlinked as part of a broader digital transformation. Andy Sturrock, CIO of one of the world’s largest energy companies, stated: “All of these things are self-reinforcing. You certainly don’t get the full benefit of one without doing the rest.”
He explained the Ops team used to be the primary bottleneck to moving applications to production (just getting servers provisioned could take weeks or months). Now, the cloud relieves that bottleneck—and that places a lot more pressure on the “apps” team, which tends to emerge as the new bottleneck. “You can write code as fast as you want, but if you can’t test it, you’re not going to get into production quickly,” he said. “Testing has to become automated, and the role of the tester must evolve.”
When Software Runs Your Business, Software Testing Becomes a Business Issue
According to Golestani, nearly $1 billion US in revenue passes through Merck’s SAP instance each week. Consider that this instance is updated at least four times a year, and more than a thousand changes are made each time. Manual testing is simply not an option. No amount of manual testing can thoroughly de-risk the core business processes, given the frequency of updates, scope of changes and breadth of functionality that’s impacted. Extreme test automation is the only way to achieve the required business risk coverage with the necessary speed.
Due to the nature of Merck’s business, lives also are at stake. Merck ships billions of medicines and hundreds of millions of vaccines each day. The company’s continued focus on digitizing the business means that software runs everything from the shop floor to the way that they interface with customers and educate 4 million healthcare practitioners. As Golestani explained, “When the software driving all this fails, nothing ships, manufacturing shuts down, research shuts down … and people die.”
It’s Not Easy, but Transforming Enterprise Testing Pays Off—Significantly
Global 2000 organizations that want to implement continuous testing—the extreme test automation required for DevOps—need to overcome challenges such as:
Highly complex application architectures (e.g., processes that cross browsers + APIs + mobile + SAP + other packaged apps + custom Java/.NET apps + mainframes).
Different application types, planning cycles, development methodologies and tools existing in parallel.
Deeply ingrained quality processes across different groups and projects.
Scarce test automation resources.
“Bloated” test suites that delay the process while providing limited business value.
There is no silver bullet. Transformations take time and they must involve all of the people, processes and technologies associated with your application delivery pipeline.
Nevertheless, you can rest assured that your efforts will pay off. Consider just a few examples of how organizations that committed to transforming testing are now growing the business and enhancing the customer/user experience:
Duke Energy achieved a $9 million cost savings during the first year of its testing transformation—and it also reduced testing time by ~ 85 percent. This enables the company to more rapidly deliver software that transforms the customer experience, modernizes its energy grid and creates a smarter energy future.
SPAR accelerated time to market 3X, enabling the company to expand its online presence across Europe, deliver more customized shopping experiences and provide seamless “omnichannel” experiences across the customer journey.
EdgeVerve now releases 30 percent more new functionality in cycles that are 60 percent shorter with 50 percent fewer escaped defects. This enables the company to help its global banking clients onboard, sell, service and engage customers with flawless personalized experiences.
ExxonMobil saved $140 million in cost avoidance and increased productivity 3X. This enables the company’s IT department to take on more projects and complete them faster—which in turn helps the company boost sales and increase customer loyalty.
Vantiv (now Worldpay) transitioned from eight-week testing cycles to daily testing, which has been critical for enabling rapid growth and agility during the recent Vantiv-Worldpay merger.
Continuous testing was once just an aspiration. Now, the drive toward digital transformation has made continuous testing a business imperative. Organizations such as these are paving the path forward, demonstrating that continuous testing can—and must—become a reality in enterprise organizations.
