If you lead big honkin’ software development projects at a big honkin’ enterprise, you’re probably doing everything you can to shift QA left. That’s because the earlier you discover defects and shortfalls, the faster and less expensively you can bring great software to market.
Unfortunately, above a certain scale, it takes more than just better processes and a more aggressive culture to shift test and QA left. It also requires a different approach to the infrastructure you use to distribute your builds.
Infrastructure and the Problem of Scale
Most software developers understand the scale issue primarily as one of complexity. The bigger the project, the greater the chance that something won’t work with something else exactly as planned.
But large software projects also canget badly impeded by the sheer physics of build distribution.
Take, for example, a 10GB build that you have to distribute to five different domestic and offshore locations from your primary facility over your company’s 10Mbps MPLS connection. If your two of your locations are domestic and have 5Mbps connections, it will take five hours to get them each build. If the other three are offshore and have 2Mbps connections, it will take them each 12.5 hours to get their copies of the build.
That’s a “time tax” you’ll have to pay again and again as you keep having to ship massive files over the network to your globally distributed test/QA team. Worse yet, the more Agile and iterative you get, the more you keep paying this “time tax” over and over.
So hours expand into days, and days into weeks—constantly frustrating even your best efforts to more aggressively shift left.
Avoiding the Scale-Driven Time Tax
Here’s how you can successfully avoid the time tax that chronically prevents you from shifting your biggest projects further left.
Instead of continuing to use IT infrastructure that was never designed to support time-compressed software development at scale, you can instead implement a hub-and-spoke build distribution system that virtually eliminates network-related delays.
This hub-and-spoke model lets you maintain a “gold copy” of your current build(s) in the cloud—while providing all your remote locations with their own local copies that get continuously and automatically updated with any changes as they occur.
This model eliminates network-related bottlenecks while allowing your geographically dispersed teams to collaborate without tripping over each other’s work. As a result, you can shift left much more aggressively.
Of course, infrastructure decisions typically aren’t made by the same folks who lead software development. So the two groups have to collaborate closely to make sure the network doesn’t impede critical development processes.
But both parties have lots to gain—because hub-and-spoke saves infrastructure owners lots of money in storage, bandwidth costs and network acceleration hardware at the same time as it empowers development to shift further left.
That’s why DevOps leaders and enterprise infrastructure owners need to work together ASAP. The digital success of the business depends on it!
Around the world, enterprises are adopting the concept of Agile development. This methodology is based on a set of principles for software development by which requirements and solutions evolve through the collaborative effort of teams. User stories form the foundation of the Agile requirements process, and teams gather to create them by exploring how users will interact with their solutions. To facilitate planning and discussion, teams typically jot down stories on pieces of paper and then physically arrange and rearrange them. It is a curiously old-school process for such a forward-thinking method.
User stories are so vital because they give developers just enough information to estimate the level of effort needed to implement the functionality described by the story. When written well, they can be powerful, because they help developers and testers view requirements from a customer’s perspective. They provide context and an understanding of what motivates the people who will use the solutions they deliver.
Struggles with Stories
Agile represents a significantly different workflow than developers have previously used. As enterprises work to expand and mature their Agile practices across their projects and teams, they struggle to figure out how user stories fit, because:
Developers and testers may prefer user stories over a lengthy, text-heavy business requirements document, but user stories are new to most business stakeholders and business analysts. As organizations try to transition to Agile, these individuals struggle to write high-quality user stories that accurately and sufficiently describe all customer needs. They get bogged down writing and managing them and lose focus on the bigger picture.
The number of user stories can get out of hand quickly due to complexity. Most enterprise projects involve the development and enhancement of multiple, integrated systems to deliver functionality for a variety of roles. Users can take many paths that often interconnect and overlap with other processes. This can mean hundreds of user stories for a project of any size. Manually recording and managing the wealth of information that the development team needs is not feasible, and user stories can be missed or misinterpreted in the process.
Critical nonfunctional requirements—such as those for compliance, security and performance—complicate the adoption of user stories because they weren’t designed to represent those types of requirements. User stories scribbled on sticky notes or created in an Excel spreadsheet don’t support the rigor enterprises need for audits, change management, history and traceability.
Enterprises want to reap the benefits of Agile for their complicated, expensive, high-risk projects, but they want to manage complexity and risk as well. They need a robust Agile requirements tool to help them, especially for the creation of reliable, consistent and high-quality user stories.
Making the Shift to an Agile Requirements Tool
Just as approaches to development have changed, so has the workplace, necessitating a new approach to teamwork. PricewaterhouseCoopers noted in its paper, “Adopting an Agile Methodology, Requirements Gathering and Delivery“: “As companies shift from small projects and teams engaged in Agile to more complex projects and potentially distributed teams, there is a need to shift from paper and spreadsheets to tools that provide workflow, persistence and traceability.”
The power of best-in-class requirements management tools is that they enable teams to use user stories for development while making it easier for business stakeholders and business analysts to create them. These tools eliminate the need to manually create user stories within the development teams’ Agile tools. They bridge the gap between traditional and Agile requirements, enabling enterprises to scale Agile while managing enterprise concerns.
The powerful capability sets of tools such as this help teams to:
Generate high-quality user stories and tests automatically from process flows. Product owners and business analysts use customer journeys to automatically generate user stories and acceptance criteria with the click of a button. They push these artifacts into the development teams’ Agile management tool of choice, where developers and testers also have access to related requirements information, such as regulatory information, visual models and constraints, supporting a comprehensive understanding. User stories are reliable and consistent, and there is no longer a need to spend time and money to teach business stakeholders or business analysts to write them.
Work together to visually define customer journeys. This allows stakeholders to “tell their stories” as they work with product owners and business analysts to collaboratively define customer journeys. Using the familiar construct of user models—with steps, decision points, actors and condition statements—the entire team collaborates to record and analyze processes in a shared workspace. Teams maintain a focus on strategic objectives when making prioritization decisions and spend less time managing a huge list of user stories manually.
Reap the benefits of Agile while using enterprise-level capabilities for visualization, traceability and reuse. Support for visual models and the ability to relate them to one another and other requirements artifacts helps teams establish the precise traceability they need to ensure full requirements and test case coverage. It also supports improved change management and decision-making, ultimately leading to higher-quality software. Customer journey models can be reused across projects and teams, as can user stories and other requirements artifacts, saving time and improving consistency.
Great strides have been made in Agile requirements management tools recently, but many enterprises are still using sticky notes to create and organize their user stories because they don’t know about what these tools can do. A next-generation tool enables organizations to scale Agile across the business and create a common ground for IT and business to meet. With capabilities including visualization, collaboration, traceability, management analytics and reuse, teams are more effective and applications are delivered faster.
About the Author / Ruth Zive
Ruth Zive is Vice President of Marketing at Blueprint. She is a metrics-driven marketing strategist who has worked for two decades serving B2B clients in the technology, healthcare and financial services industries. At Blueprint, Ruth is responsible for product marketing, analyst relations, branding, demand generation and inside sales initiatives. Connect with her on LinkedIn and Twitter.
The agile approach is everywhere. Born in the software development world, agility means adaptive planning, ongoing development and continuous improvement. Rapid and flexible response to change is critical. It is not surprising that the best practices of agile software development penetrated into other areas, and IT service management (ITSM) is no exception.
The Service Desk Can Be Agile
Unless you’ve been away from ITSM for the last 10 years, you know that an agile IT service desk is flexible, automates processes for quicker incident resolution and emphasizes collaboration between technicians and interactions with users.
But how do you adopt these seemingly tough principles to make your IT service desk truly agile? We’ve put together five simple ways to help you assess your IT service desk and move toward agility.
Keep Interactions in Real Time
IT service desks are constantly nestled in webs of interactions between technicians and users. It’s easy for technicians to get lost in endless emails and delay resolution. To avoid chaos and ensure timely ticket resolution, interactions should stay in real time and communication should be as proactive as possible.
Let’s say the sales department of a company uses a sales application. But the application crashes, and the department’s users are unable to connect to it. So, they raise a ticket, and the IT admin immediately checks the server and realizes it’s down. They then check the code management database (CMDB0, which indicates that the application is also used by the finance department. Before attempting to resolve the incident, the IT admin should send a notification to the sales and finance departments to inform the users of both departments that the application is down and that the IT team is working on resolving the issue. That way, they can prevent duplicate tickets and focus on fixing the issue.
Eliminate Productivity Leaks
Most IT service desks are built on ITIL processes, which means that they are already efficient. But to become agile, you have to track IT service desk performance with key performance indicators (KPIs) such as lost business hours, SLA compliance and cost per ticket—all of which monitor both cost and productivity. While KPIs help you measure productivity leaks, reports help you identify their root cause. For example, your service desk KPIs indicate that there have been a lot of SLA violations. A quick look at your reports shows that a lot of time is spent gathering information rather than on finding solutions. By creating a template for common incidents or service requests, all the information you need is gathered in one place, making it easier for technicians to quickly resolve issues and meet service level agreements (SLAs). Similarly, you can reduce the time spent on routine tasks, such as assigning tickets and tasks to technicians, sending notifications to users and more, by automating these processes.
Encourage Collaboration Among Stakeholders
Every IT service desk has stakeholders across various disciplines and geographies. And IT service desks are often rated on how seamless the flow of information is between stakeholders. Therefore, it’s important to have a service desk that provides visibility and encourages collaboration. Let’s say that a company is running its financial application on a legacy server. To cope with increasing transactions, IT proposes a server upgrade. Since this application impacts everyone in the organization, any disruption in service will impact the business significantly. So, IT should keep all business groups and admins in the loop, ask them for recommendations on scheduling the change, ensure backout plans and inform users about the change schedule and expected downtime. An IT service desk application that provides room for this collaboration ensures a smooth upgrade with minimal interruptions.
Establish Self-Organizing Teams
An IT service desk is incomplete without its team. And since IT is essentially the command center of a company, it’s imperative to establish a self-organizing IT team. This starts with looking at the current team’s structure and creating workflows around it. In most IT service desks, the technicians are grouped based on their skill sets and level of expertise. So, you can easily assign a technician (or group) to a workflow that requires a certain type or level of skill. You can also auto-assign tickets to technicians or groups, assign a change manager to coordinate with different teams, automate escalation rules for complex issues, configure a backup technician when a one is not available and more.
Learn Through Iterative Increments
IT service desks are expected to meet the changing demands of their organizations. This means that they have to make a number of small changes over a period of time. In agile terms, this is called iterative increments. Let’s take, for example, a growing organization that has several employees joining every week. A typical employee onboarding and provisioning process requires the IT team to coordinate with different teams including HR, maintenance and administration. This can take up to 10 hours for each new employee. But, you can simplify this process by dividing the request into smaller tasks and assigning the tasks to corresponding departments. You can also automate this whole process, significantly reducing the time it takes to onboard a new employee.
Just follow these simple steps and you will be on your way to creating an agile IT service desk that delivers quicker resolutions, increases user satisfaction and evolves with rapidly changing technologies.
About the Author / Aparna TA
Aparna TA is a product analyst at ManageEngine, the real-time IT management company and division of Zoho Corp., and works in the ITSM products division for the flagship product ServiceDesk Plus. She keeps an eye on various IT practices and studies their impact on IT service management. In her free time she likes to relax on a couch and read the latest crime novels. Connect with her on LinkedIn and ManageEngine on Twitter.
Almost every product we buy or use today goes through some form of extensive testing before it’s released on the market; typically, these tests are designed to replicate the worst-case scenarios that a product may encounter under real-world conditions.
Automobiles, for example, go through extensive hot and cold weather testing in remote locations, and are driven over obstacle-laden tracks designed to push the limits of their durability. Even seemingly mundane objects such as children’s toys are exposed to abusive simulations to ensure that they don’t harbor any potentially hidden and harmful effects.
On the current landscape, applications are the face of your business, often serving as the battleground where consumer loyalty is won or lost. Given this reality, it seems crazy to think that any applications are released having only been exposed to a minimum of tests. Yet, in the name of getting to market quickly, that’s far too often the case.
As disciplines including agile and continuous delivery continue to increase the speed at which development teams can release software, it’s easy to put the crunch on testing in the rush to get an update released. But that’s a costly mistake in the long run, as research has shown that it is 50 percent less expensive to find an error in testing than to address the same error once it has reached production, and more than 90 percent cheaper when addressed in the development/coding phase.
Given the dramatic cost savings of addressing errors in development, clearly the goal of any organization focused on continuous delivery should be to shift testing left so a greater number of defects are found early in the development process. This is what we refer to as continuous testing, and it’s a huge part of the reason why I am so excited about CA’s acquisition of BlazeMeter.
Performance load testing should be a core element of any continuous testing strategy because it is a critical factor in determining system reliability. Similar to the worst-case scenario testing that consumer products undergo, performance testing can quickly uncover applications defects that might otherwise go unnoticed. After all, the manner in which a system operates with a few hundred concurrent users may be very different than how it runs with thousands of users.
BlazeMeter enables today’s development, operations and DevOps teams to conduct this critical performance testing earlier in the development process, saving countless hours and resources. The company is also competing—and winning—in a highly competitive marketplace where it has been pitted against much larger, more entrenched competitors.
Part of the reason BlazeMeter is winning is its focus on offering a SaaS-based solution that is highly cost-effective and easy to engage. Using this model, customers can try out the testing software with a few clicks, and then quickly move forward to broadly adopt the system. Another catalyst to its success: BlazeMeter was built using open source tools, such as Apache JMeter, used widely in the development community for performance and load testing. As a result, BlazeMeter can easily run and analyze JMeter and other open source-based load tests—such as Selenium, Gatling and Locust—within a commercial-grade performance testing system.
This combination of a SaaS delivery model and incorporation of open-source tools, augmenting strong performance testing functionality, make BlazeMeter very complementary to CA’s existing continuous delivery solutions, including its CA Service Virtualization, CA Test Data Manager, CA Agile Requirements Designer and CA Release Automation products.
At CA, we’re committed to providing the most comprehensive, integrated and open continuous delivery ecosystem, allowing our customers to release swiftly, test agilely and constantly increase reliably. The addition of BlazeMeter will further enable customers test continuously, thereby increasing agility and the overall quality of their software, to ensure that applications don’t break down on the side of the virtual road.
Many business leaders are planning to make DevOps the cornerstone of their IT transformation strategy. But, as surprising as it might sound, skepticism abounds regarding how major vendors address the DevOps transformation challenges—they confuse DevOps with continuous delivery (CD) infrastructure.
“We wanted DevOps and we got after a nine-month-long implementation a continuous delivery platform which speeds up IT operations without significant business value. How differently can you help?” the senior business executive of a major financial institution in New York sarcastically asked me during a conversation three months ago.
The business executive is right: DevOps isn’t a technology as I’ve been implementing it, it’s primarily an accelerated application delivery approach implemented in the form of a technology-enabled application delivery operational model. Adam Jacob, CTO of Chef, substantiates that fact in “The Secret of DevOps: It’s Always Been About People, not Technology,” when he says, “DevOps is about bringing together all the people you need to build and run your business effectively, and empowering them to move as quickly as possible towards their goals.”
Agile Admin’s Ernest Mueller in, “DevOps: It’s not Chef and Puppet,” confirms Adam’s point: “Chef and Puppet are individual tools that you can use to implement specific parts of an overall DevOps strategy if you want to use them—but that’s it. They are fine tools, but do not solve DevOps for you.”
How can we implement DevOps in line with business leader expectations? Three Amazon Web Services (AWS) Design Patterns I developed can help to get the best of DevOps on AWS cloud: Software Organization Factory, Digital Organization Factory and CD Infrastructure Factory. This article discusses the Software Organization Factory.
The Foundation of DevOps Value: Consequences to IT Service Delivery
Before delving into these AWS design patterns, a quick reminder of what DevOps is, its value propositions and, more importantly, what changes are needed in how IT services are delivered.
The Ignored DevOps Business Logic We All Must Consider
The combined effect of cloud adoption and the rising digital economy is changing competitive environments; it forces business leaders to focus on innovation and market responsiveness as the competitive differentiators.
The fact of the matter is, by facilitating access to technology, cloud accelerates entrance of several thousands of tech startups that take advantage of the recent technology innovations to substitute traditional products and services into digital. This creates a context where new entrants increasingly compete with established brands, forcing business leaders to seek value paradigms other than IT efficiency.
The business executive argues, “In these new markets, our concern is to increase customer value by delivering the right services to the right segments at the right moment. You guys need to change the IT paradigm; IT alone does not create that kind of value.”
Then, what should be changed in today’s IT approach?
The Two Dimensions of Modern IT: The Operational Model and the Continuous Delivery Infrastructure
The new IT paradigm advocated in my recent book, “Rethink Your IT with DevOps: The Complete Guide to Aligning Your IT to DevOps,” is the idea that business value results from two dimensions: organizational agility and technology efficiency. In other words, it defines modern IT as a two-dimension asset including the operational model and the continuous delivery infrastructure.
The timely delivery of innovation transcends IT issues; it not only cuts across your sales, marketing, post sales, application development and IT operations, but also demands concrete answers as to how to accelerate business prioritization, decision-making and problem solving. The operational model is where business and IT people, processes, practices and tools are combined to create agile and cross-functional environments. The following exhibit shows a typical DevOps operational model:
Typical DevOps Operational Model
Organizational flexibility for the sake of flexibility doesn’t make sense unless technology is leveraged to speed processes. This is where CD infrastructures come into play; throughout the delivery process, they build on IDEs to accelerate coding and testing, on versioning tools to convert software version into releases, on continuous integration (CI) tools to build releases and on delivery pipelines to ensure continuous delivery.
So, what is DevOps?
DevOps is a Technology-Enabled Application Delivery Model
DevOps is a combination of value, principles, processes and practices shared not only by developers and IT operations but also by the business. The goal is to help the organization deliver the right service to the right market segments at the right moment.
DevOps from the Business Perspective
As I’ve been implementing it, it’s a cross-functional application delivery model involving the business and IT and enabled by continuous delivery infrastructures.
By enabling and automating agile practices, the continuous delivery infrastructure makes the operational model agile and provides the business with the flexibility and velocity needed to meet competitive challenges.
Design Patterns as DevOps Implementation Engineering Framework
As suggested by the business executive, today’s DevOps implementations struggle to deliver the promised business benefits. The reason is, the operational model dimension is ignored in favor of tools and infrastructure.
Another reason is, DevOps is young and is still at the early stage of the technology maturity model; the expertise and engineering methodologies to implement it as what it is—a technology-enabled application delivery model—are lacking. That’s where DevOps transformation design patterns help.
What are DevOps Transformation Design Patterns?
In this article, the term DevOps transformation design patterns refers to general reusable architectures addressing specific DevOps strategy. They’re not necessarily implementable solutions, their primary goal is twofold:
Offer a neutral language for designing and implementing technology-enabled operational models
Offer an engineering approach that reduces design and implementation duration and effort
They’re formalized architecture design and implementation best practices for DevOps and platform-as-a-service (PaaS) services.
Design Pattern as DevOps Implementation Engineering Methodology
To meet its goals—technology-enabled application delivery model—the design pattern is structured around four items including purpose, principles, general solution and implementation techniques.
Purpose: Clarifies the purpose of the design pattern along with the DevOps strategy to address.
Principles: Are best practices formulated as rules and beliefs supposed to guarantee expected DevOps strategy benefits? They’re the determinants of the solutions to implement the DevOps strategy.
General solution: Are blueprints resulting from the application of principles? They summarize the functional and technical architecture of the DevOps platform to implement.
Implementation techniques: What are recommended technical approaches, solutions and tools to implement the general solutions?
Why AWS? Why would it be the catalyst for DevOps value?
AWS-ome Design Patterns That Will Change DevOps Implementation
AWS offers such a rich, varied and easy-to-implement services portfolio that it would be unfortunate not to take advantage of it to modernize IT. The rationale underpinning the value businesses can derive from their DevOps transformation experience with AWS is summarized in the following figure:
Rationale of the DevOps Design Pattern
It’s the belief that:
Certain principles determine the AWS services needed to deliver the application delivery model (ADM) architecture that guarantees value.
The implementation of the ADM using these AWS services results in AWS-based DevOps platforms that tangibly deliver expected benefits.
The Four DevOps Principles that Yield AWS Value
Let’s explore the principles I’ve been using to make CIOs happy with their AWS cloud:
Think holistic, not only IT: Reminds that transformation isn’t only about IT but the entire organization’s value stream from marketing and sales to development and IT operations.
Think cross-functional, not only IT: Reminds that the bottom line is to set up a cross-functional application development environments to accelerate business prioritization, problem-solving and decision-making, and meet time-to-value objectives. They’ll be the core value propositions of the DevOps platform.
Make it agile, flexible, and speedy: Reminds that agility, flexibility and speed are the catalysts for competitiveness and business value. They’ll be part of the DevOps platform.
Think PaaS: Reminds that AWS provides varied services for implementing state-of-the-art CD infrastructure.
Let’s illustrate three AWSome design patterns that’ll change DevOps experience with AWS!
Implement a Startup Mindset with the Software Organization Factory
The purpose is to implement DevOps strategies focused on time-to-market and time-to-value. It’s adapted for tech startups.
To implement the startup mindset, the DevOps four principles for AWS are implemented as follows:
Think holistic and cross-functional, not only IT: Relevant business and IT resources are put together in a single and dedicated service development team supported by appropriate Scrum and XP agile practices.
Make it agile, flexible and speedy: PaaS services building on agile practices and AWS infrastructure-as-code are provided to make application delivery processes agile, flexible and speedy.
Think PaaS: AWS infrastructure as code and DevOps tools are combined to implement a CD infrastructure in the form of a PaaS platform.
The DevOps platform resulting from the principles application is illustrated below:
Software Organization Factory
The “Startup Mindset” platform is structured around three components including the cloud-enabled operational model, PaaS and the CD infrastructure.
The operational model builds on a three-stage application delivery life cycle:
Plan & measure: It mobilizes business users, development, QA and operations staff on business prioritization issues using agile practices such as Scrum product backlog planning or XP’s user stories.
Develop & Test: It mobilizes business users, development, QA and operations staff across the development life cycle using agile techniques such as Scrum iterations, XP’s test-driven development (TDD) and behavior-driven development (BDD).
Release & Deploy: It leverages continuous integration practices to build and test releases prior to deploying to production.
Monitor & Improve: It’s outsourced to keep the service development team on its core business: developing software.
The CD infrastructure combines the followings services:
AWS CodePipeline and CodeDeploy to set up the delivery pipeline.
AWS infrastructure as code including CloudFormation, OpsWorks and Elastic Beanstalk to generate the CD infrastructure.
Summing It Up
The DevOps business benefits are real, yet, it’s young and is still at the early stage of the technology maturity model. The expertise and engineering methodologies needed to implement it as what it is—a technology-enabled application delivery environment—are lacking.
DevOps design patterns such as the software organization factory fill the gaps of today’s transformation practices:
They provide the architect team and the executive team a neutral language to discuss the business, technological and technical challenges and solutions.
By pointing out the agile practices to leverage, they simplify and accelerate the design and deployment of the operational model.
By pointing out the AWS services and features to leverage they facilitate the design and implementation of the CD infrastructure.
Deploying software has always been manual and labor-intensive. That’s fading into the past as continuous innovation has become a competitive differentiator for companies of all stages and sizes across every industry.
DevOps has become a hot investment market in recent years. The mega-rounds we’ve been seeing show the massive demand for tools that enable optimal runtime. Moreover, IT leaders are ready to spend to promote the DevOps culture.
Companies increasingly are turning to DevOps processes and tools to automate the development life cycle and position themselves to adapt and scale nimbly. But for some companies, the idea of completely overhauling their IT operations processes can seem daunting—they don’t know where to begin. The thing is, adopting DevOps is actually quite simple.
Examining the Stack
Developers should bake in automation and agile development from the beginning. The good news, though, is that a defined set of tools to automate each step of the deployment process has emerged. Here’s a breakdown of the new DevOps stack, layer by layer:
Source Code Repository: Let’s start from the bottom of the stack—from the moment you write the first line of code. Before the open-source movement took off and the cloud enabled easy sharing, developers had to work much more slowly to ensure that the necessary improvements were inserted in the right version of a product. Companies such as GitHub, Atlassian and GitLab provide source-code repositories that facilitate collaboration. Now, engineers can work on the same projects at the same time from halfway around the world without worrying about duplicating efforts.
Continuous Integration: Newer tools such as CircleCI and TravisCI, as well as older tools such as those offered by Jenkins, automate testing and integration and provide repeatability in the deployment process. Using these tools, developers are able to test their code for failures and bugs as they write it so the code can be fixed before users ever experience the bugs. CI tools give developers something they desperately need to succeed in today’s competitive marketplace: timely feedback. By spending less time waiting on feedback for new code, engineers can resolve software bugs before they grow into full-scale crises.
Binary Repository: Jfrog and Sonatype provide an end-to-end solution for storing and managing binary artifacts. The binary repository is the system of record closest to the production runtime environment. When something goes wrong, a binary repository such as Artifactory is the first port of call for operations teams. In addition, these tools scan for security vulnerabilities and make it easy to seamlessly distribute software to every corner of the globe.
Continuous Development: Companies including Chef, Puppet and Ansible automate the next phase of taking data into production. The holy grail for software developers is zero downtime; continuous development tools allow developers to update software while everything remains up and running. These tools also streamline the delivery process by automatically configuring servers and ensuring repeatability of the deployment process.
Production Environment: Google Cloud, Amazon Web Services and Microsoft Azure are in a war over runtime market share in the public cloud. Behind the firewall, Mesosphere, Kubernetes and Docker are trying to dominate the orchestration layers of enterprise private clouds. Customers want the agility and automation that comes from a cohesive DevOps stack but are refusing to be locked into a single vendor at any level. Whichever stack emerges as the default DevOps stack will have to ensure compatibility across public and private clouds.
About the Author / Andy Vitus
Andy Vitus is a Partner at Scale Venture Partners. He has been investing with Scale for over 12 years and focuses on next-generation enterprise software and new data center technologies, including virtualization and cloud computing. Andy sits on the boards of Arena, Crittercism, DataStax, JFrog, PubNub, Realm Stormpath and Treasure Data. Connect with him on LinkedIn and Twitter.
This blog stems from my experience of being one of the earliest DevOps Foundation trainers of DOI and successfully completing 19 batches of DOI Foundation course in the last year.
In retrospect of the year starting when TaUB Solutions signed up as a registered education partner (REP) of DOI, my opinion is this has been the best service portfolio offering for our customers and partners.
There are few things that appeal best when it comes to DOI Foundation course and I can summarize them with the collective feedback I’ve received so far:
“Challenges the preconceived myths and practices around DevOps with practitioner stories.”
“Gives a holistic overview of philosophy behind DevOps. Philosophy should precede science and science should precede engineering.”
“This course maintains the discipline to be truly advisory in nature by illustrating tool chain without recommending vendor-specific tools.”
“The group exercises during the class and the facilitator practical insights helped us to put the aspects in to our organizational context of delivering products and services.”
“We have never seen a course such beautifully designed that could be the first basic building block to know ‘Do’s’ and ‘Don’t’s’ of DevOps across application developers, infrastructure operations, project managers, infosec professionals and business leaders.”
To enhance the overall learning curve of the audience, we put them in to the third day of experiential learning using the Phoenix Project Business simulation. The simulation was divided into four rounds and played with maximum of 12 people per batch. The Team was entrusted to achieve a target of $300,000 within four rounds and take the share value from $21 to $45.
While the target milestones were clearly laid down at the beginning of game it was difficult for the teams to get hold of the situation due to various unknown parameters. It took them quite sometime to understand the different moving parts of flow, dependencies, business impact of projects, issues and features that came up in every round and make appropriate choices and decisions.
The teams experienced significant learning, sharing and collaboration over the four rounds and had loads of fun. One common feedback from the teams was for the game to be mandatory after every two-day DevOps Foundation course.
The consolidated feedback after playing the DevOps business simulation after the two-day course:
“It demonstrated that it is not easy to build collaboration, communication and integration. It takes time and there is a lot of patience and team work needed to make this a reality. The DevOps culture had to be nurtured through self-organizing teams with mutual respect, sense of pride and collective ownership to work toward a common goal.”
“It gave a working knowledge on how ITSM and DevOps are complementing each other [Ex. Change Management and Improved Flow].”
“The Principles of DevOps, namely the First Way (Improve Flow from left to Right), Second Way (Shorten Feedback Loops) and Third Way (Repetition through Mastery) was visibly evident over the course of four rounds.”
“The speed of execution improved over the rounds using short feedback loops, continual improvement and applying lessons learned from previous rounds.”
“It gave a good understanding of how different roles and responsibilities in the organization contribute to the overall success of the project.”
Finally, the teams made over and above the required target of revenue and stock price. The winning moment was enhanced by the significant learning opportunity and ability to bond with diversified cross functional teams.
The experience of playing the business simulation reinforced practical understanding of DevOps and also transferred skills that they could implement immediately at their workplace.
About the Author / Suresh GP
Suresh GP is the Managing Director of TaUB solutions and Regional leader of BRMI (APAC). He has more than 15 years of experience in IT Service Management, IT Governance, Program Management and Organizational Change Management. He is regular blogger and International speaker at itSMF UK, itSMF Australia, itSMF USA, itSMF Finland and itSMF Norway. He is a Certified DevOps, Agile Service Manager, Business Relationship Management Professional, PMP, CGEIT, ITIL V2 Service Manager, ITIL V3 Expert, ISO 20000 practitioner. He was awarded the ITSM contributor of the year by itSMF Singapore in 2013.
In “The Phoenix Project,” a 2013 novel about an IT manager rescuing a late, over-budget IT initiative, Gene Kim demonstrates how collaboration between IT operations, development and business users can impact the business. In the era of digital transformation, the need to bring development and operations together—in what is often called DevOps—is more important than ever, as IT is now more than an enabler of the business. IT is the business.
What’s at stake today is more than just a project or someone’s position or even the viability of an in-house department versus outsourcing the function. Instead, it may be the very existence of the company. According to Gartner, 25 percent of companies will lose business due to digital incompetence by 2017 while IDC predicts that 75 percent of the S&P 500 will be replaced by 2027. To avoid the fate of becoming an industry has-been, today’s market leaders need to ensure they won’t be overtaken by more agile competitors that respond faster to a rapidly changing marketplace.
Automate Everything
Today’s businesses need to transform into digital enterprises to thrive in today’s digital economy, and this transformation needs to be far more profound than simply investing in the latest digital technology. Because the pace of the digital economy requires companies to roll out new services faster than ever before, businesses are challenged to move from idea to revenue as soon as possible. This need for speed leads to a desire—really a requirement—for agile applications.
For many of today’s enterprises, this means they have to stop thinking that a successful future depends solely on a traditional strategy of product innovation, using data analytics to better understand and serve their customers or building a higher quality, more profitable product portfolio. Instead, like the mythical phoenix, their survival may well depend on a complete rebirth around making the practices of agile application delivery and DevOps a core competency.
DevOps teams need the ability to efficiently develop, test and deploy new releases faster, with high quality and stability. Using automation technologies to design-in this speed, quality and scalability within the development life cycle is critical. The continuous integration and delivery processes at the core of DevOps and agile practices are enabled by many new automation tools and capabilities. Each of these tools typically targets discrete elements or steps in the entire process and DevOps teams use them as such, finding the right tool for the job. In addition to faster and higher-quality development throughput, building automation into the code of the application ensures better application services delivery in the production environment.
The good news is that 60 percent to 70 percent of these enterprise application services or jobs within most organizations are micro-batch, batch or event-driven—and can be managed with a comprehensive job scheduling solution. By automating jobs as part of the application code and subjecting the job artifact to all the other automation processes, companies can substantially reduce time and effort during and after the application delivery process.
According to Enterprise Management Associates, companies with workload automation reduce application outage time by about 70 hours annually, raise service level agreement standards, free 41 percent more IT staff for strategic projects, and defer or eliminate an average of 50 percent of IT staffing requirements.
Traditional vs. Innovative IT
Too many companies intent on being agile and innovative believe that a successful DevOps organization must be free of the trappings of rigorous change management. Business innovation and increased employee productivity, however, can’t exist without a stable IT foundation. The goal is to have a balance of both strategies in an organization. Gartner refers to this as Bimodal IT, which has two modes: “Mode 1 is a traditional, plan-driven approach that emphasizes safety and accuracy in the pursuit of reliability. Mode 2 is iterative, being largely outcome-focused and emphasizing agility and ‘just enough’ with respect to areas such as planning and process.”
By committing to balancing the two strategies, organizations can increase agility without sacrificing stability. More importantly, companies can increase employee satisfaction by different teams successfully doing the jobs they are paid to do, whether it’s to drive innovation or keep the lights on. It’s worth noting that Gartner believes many elements in DevOps apply to both modes and that by 2020, at least 80 percent of practices identified today with DevOps and Mode 2 will be adopted by traditional Mode 1 groups.
While thinking of your company as a phoenix may be essential for survival, the benefits go way beyond that. Just like the revitalized bird, your company can once again be youthful and vibrant, capable of soaring to new heights. This is a strategic goal worth investing in.
About the Author / Gur Steif
Gur Steif is the president of the Workload Automation product line for BMC Software. Under Steif’s leadership, the Workload Automation business has more than doubled in size. Previously, Steif was the general manager and vice president for both the Workload Automation product line and the Performance & Availability business line. During this time, he grew the Workload Automation business at triple the market growth rate while also delivering a double-digit increase in customer satisfaction for Performance & Availability. In his 25-year career in technology, Steif has held leadership roles in strategy, product management, R&D, sales, marketing and M&A in IT operations management, application management and workload automation. Connect with him on LinkedIn/Twitter.
Some say DevOps is the next step in Agile’s evolution.
Software development is changing rapidly; if you don’t change with it, modern development may leave you behind. DevOps connects information technology operations with development and vice versa; this is a big change for the art of software creation. Agile was not designed with this new level of focus on information technology operations in mind, and must change to include this focus to thrive.
According to Tony Higgins, CTO of Blueprint, the hot new trends that are supposed to replace the methodology actually are extensions of it. DevOps, for example, aims to extend the collaboration happening between development and QA teams to the operations people who are running the production infrastructure. “It’s the next step in the Agile process,” he says.
Survey data correlates with assertions it is growing in large enterprises even as DevOps is expanding. According to VersionOne’s 2016 State of Agile Report, more than 24 percent of respondents who use the methodology work for organizations with more than 20,000 employees, compared to 21 percent last year.
Addressing the State of Agile
Some enterprise software development stakeholders are shifting their sentiments by moving into enterprise Agile from software project-based/development side-only Agile.
To some, the original creed should be unwavering, even though it addresses only software development and not information technology operations. “In an ISV context, which is where Agile ‘grew up,’ this understanding did really well. But now with [it] making rapid inroads into information technology, the context is very different and, in a nutshell, the manifesto is insufficient,” says Higgins. The needs of information technology operations such as those that DevOps is meeting are driving new frameworks so it also can morph to include a Dev and Ops methodology.
“Rather than dying, Agile is transitioning from a competitive differentiator to a core competency,” says Higgins. Whether as part of Agile, what the industry calls DevOps or some new methodology, development is accelerating by building new features and innovations in small, rapid, incremental trips through the pipeline; that approach is originally a core concept of Agile. “Industry folks declaring it’s dead don’t realize that businesses have not given up on it; they’re just looking for better ways to do it,” he adds.
Agile is Growing
According to Higgins, the methodology isn’t something that just “happened” in 2001 when its manifesto came to into existence. “Software development organizations were practicing and evolving the various principles of Agile all along and the manifesto was really a culminating event that codified it,” Higgins explains.
The methodology is spreading much like DevOps, appearing in executive groups such as marketing, sales and information technology. “In marketing, they’re time-boxing their work into iterations with tangible outputs/deliverables. You’re seeing regular ‘stand-ups’ happening with the inside sales group. Agile is making fast and strong inroads into information technology, driving tighter collaboration with operations, and enabling faster delivery,” says Higgins. It is proving to be a repeatable business model that applies to every part of the organization that cares to use it.
Proof it is a core competency lies in the fact that technology must now be able to support it. “Analysis tools that help determine requirements must be able to do so in an Agile environment,” says Higgins.
Enterprises are approaching the methodology in better ways as organizations are seeing that requirements are keys to its effectiveness. “Requirements often have been overlooked in Agile processes, but recently we have seen research firms (Gartner, Forrester) and the market shift their focus to requirements to improve their deployments,” says Higgins.
Both Agile and DevOps are useful in extending the collaboration of development and QA not only to operations people but ultimately also to other departments. “Instituting DevOps, continuous integration tools or any type of automated deployments will inherently require collaboration between departments; this must actually extend well beyond just Dev, QA and Ops and to the rest of the business to ensure that end-to-end visibility is achieved and decisions are not made in isolation,” says Higgins.
While there’s no fundamental conflict between IT service management (ITSM) and DevOps, DevOps does bring new and different techniques for getting work done and interacting with the rest of the organization.
ITSM people who are used to frameworks such as ITIL and COBIT tend to regard the IT department as a “service provider” and the rest of the organization as “the customer.” But DevOps has come from web-scale organizations “where IT is the business,” and that means the prevailing view is more like the Toyota Production System mentality, wherein the aim is to move things along a production line with maximum speed and maximum quality, says Matt Hooper, product evangelist at LANDESK.
DevOps thus has already broken down some of the barriers between “IT” and “business”—practitioners realize they have to determine what the business needs, and then provide it. But Agile or Agile-like practices also are part of this approach, so there’s lots of experimentation, learning and redeployment in “lots of tiny little steps,” and, consequently, there’s not much risk of going very far wrong with any of them.
An issue is that this conflicts with ITSM’s view of life cycles, partly because the whole idea is to merge development and operations—for instance, by automatically spinning up software-defined infrastructure as needed.
What’s needed, Hooper suggests, is “service management with a digital mindset” rather than ITIL’s “human mindset.” It’s all about feedback, he says, but for ITSM and DevOps to sit comfortably together there must be a realization that feedback can come from humans or systems.
And even though the command and control mindset tried to avoid failures, systems still failed. So a better approach is to accept that “stuff breaks” and make small changes at a time, respond to issues as they arise and keep learning. In the longer term, this will lead to more durable systems, he says.
It’s a myth that you’ll be able to stop bad things happening, because people and systems aren’t perfect. DevOps, in contrast, accepts that things will go wrong but aims to respond quickly—for example, by keeping developers close to operations so they can see problems occurring, realize what mistakes they made, and put them right.
“That’s challenging to a traditional ITSM person,” Hooper says. It is important to accept that failure is an option, even though massive failure isn’t.
(Before you ask, he accepts that certain systems need to be extremely reliable. But others provide room for a somewhat experimental approach, in which case DevOps helps avoid overengineering.)
While some ITIL practitioners see value in DevOps’ continuous deployment approach, their premise is still that IT is a “service provider” rather than a “partner,” but the business side of the organization wants the latter. You don’t get home from a business trip and tell your spouse you were 99.98 percent faithful while you were away, he observes, and that sort of SLA-based thinking works against becoming a learning organization, achieving good demand management and so on.
ITIL v2 served a good purpose, but ITIL v3 arrived too late, Hooper suggests. The growing influence of the Lean movement means systems are going through rapid change cycles, and too much “process” leads to waste and lower performance. DevOps and software-defined technologies provide ways to better handle rapid change—infrastructure can be spun up and back down in seconds, providing opportunities for self-healing systems with better scalability and reliability.
Where ITIL focuses on meetings and human processes, DevOps prefers to use analytics to determine the right time to release and then whether then release was successful. “The feedback loop is an essential part of DevOps strategy,” just as it is with ITIL, but ITIL has a more human-centric idea of feedback.
“ITSMers tend to hide in the back closet” waiting for the business to tell them what to do, but DevOps takes a more active approach, he suggests.
Ruth Zive is Vice President of Marketing at 
Aparna TA is a product analyst at 
Andy Vitus is a Partner at Scale Venture Partners. He has been investing with Scale for over 12 years and focuses on next-generation enterprise software and new data center technologies, including virtualization and cloud computing. Andy sits on the boards of Arena, Crittercism, DataStax, JFrog, PubNub, Realm Stormpath and Treasure Data. Connect with him on 
Suresh GP is the Managing Director of TaUB solutions and Regional leader of BRMI (APAC). He has more than 15 years of experience in IT Service Management, IT Governance, Program Management and Organizational Change Management. He is regular blogger and International speaker at itSMF UK, itSMF Australia, itSMF USA, itSMF Finland and itSMF Norway. He is a Certified DevOps, Agile Service Manager, Business Relationship Management Professional, PMP, CGEIT, ITIL V2 Service Manager, ITIL V3 Expert, ISO 20000 practitioner. He was awarded the ITSM contributor of the year by itSMF Singapore in 2013.
Gur Steif is the president of the Workload Automation product line for 
