Author: JP Morgenthal

The Executive’s Guide to Microservices Chapter 3: Choosing A Starting Point

In the previous chapter, Steve had been asked to provide a comparison of costs and resources needed to migrate to a commercial system versus modernize the current system. He works with Stacey, head of development, who comes up with a modernization plan that includes certain changes that the team will implement to limit downtime.

Steve has discussed Stacey’s plan with the business, which agreed with the approach toward modernization of the supply-chain system. The business particularly likes the idea that in the future the reduced fragility of the system might mean its IT teams can introduce ongoing improvements even through the holiday season, rather than having to freeze all systems development for a period of four months.

Now Steve and team are challenged with where to start.

The team has decided that any changes will require decomposition of the existing monolithic design. The approach to date had been to address requirements functionally, which meant that every time the business requested changes to the system, the code was changed to support those requirements by adding specialized logic to various modules.

Unfortunately, this approach has resulted in significant amount of technical debt. Each compounded change introduced conditional logic to support prior requirements. Moreover, each new change had the potential to break prior changes, so time-consuming impact analyses had to be done before any changes could be made.

Steve and team want to decompose the system into a set of microservices that closely resemble the business function they support. Moreover, they want to segregate the specialized business logic into modules that consume the generic business services.

Obviously, this is a long-term strategy, so Steve and team need to choose how to approach incrementally to enable them to make major strides before this year’s systems freeze. Given the proximity to the holiday buying season and the availability of resources, Steve and team will need to choose wisely which microservices they will create. Moreover, Steve and team know they will be limited by the level of entanglement within the existing code.

After an initial analysis Stacey’s team determines that the costliest issues stem from the logic to support multichannel customer engagement. That part of the system integrates custom logic with some off-the-shelf ecommerce products. During the holiday season, customer support calls increase in volume and each call can last five to 50 minutes. The large variation is often caused by the inability of the system to address changes in a particular channel post order-placement. For example, if the customer wants to add a gift receipt or wrapping after the order is placed, it must be dealt with out of band. This means that the business has had to add support people during the holiday season just to deal with these issues.

Stacey and team also believe they can easily isolate the logic for multichannel support within the existing code base and decompose into a set of microservices. The team decide that the existing customer records system must stay as is for the time being, but they can create a new API to more easily access that data from the new microservices. Stacey calls these macroservices.

The team will create a set of microservices to support the web channel and a set to support the distributor channel, since these each have very different requirements. The current system combines the logic to support both of these in one module today. Since microservices are supposed to be focused on doing one thing well, the team have decided on the following bounded contexts: order creation, order execution (payment), order fulfillment and returns.

Steve asks Stacey, “Why not just have an order service?”

Stacey replies, “That is one approach, but we would call that a miniservice as it manages all aspects of orders. However, our goal is to allow parts of the system to change over time with minimal testing and maintenance required. A microservice change should not have broad system implications. Changing the order miniservice could mean that an entire channel could be impacted. With this approach we could change any of these supporting order processes without impacting the entire order function within the company.”

Steve thanks Stacey for her answer and tells her he has a much better understanding now for what the team is trying to achieve.

— JP Morgenthal

February 16, 2018
The Executive’s Guide to Microservices Chapter 2: The Business Challenge
Steve and team have been working diligently to analyze the intricate web of code that has been organically developed over the past 12 years to design a program to modernize the application. Application is actually a poor name for all that Steve and his team manages. Steve’s team manages a complex custom supply-chain system that manages inventory, warehouse, picking, packing, shipping and order management. A hiccup in any module could lead to a serious financial impact to the business.

The business has discussed multiple times over the years switching to a commercial product, such as Manhattan or SAP, but the costs and challenges of doing so without interruption to the business resulted in a decision to keep maintaining the current system. The business has been judicious in its requests to update the system, understanding that changes take a long time to implement and test due to the design. However, competition from major online retailers are starting to seriously impact profitability, forcing the business to make a considerable investment.

The supply-chain system is comprised of multiple modules mostly written in the Java programming language, but some of the older modules are in C++ and require a Windows Server operating system. This combination of technology has been a source of strife for Steve’s team. They have tried multiple technologies over the years to make the system more stable and to isolate the older C++ modules in a way that reduces risk of downtime should they crash. The Java modules were written to run inside of a web application server, which provides some linear scalability, but it’s still an issue to scale the C++ modules in a relative manner. Now, increased demands from the online channel is stressing the system in ways that it was never designed to handle.

The business has asked Steve to provide a comparison of costs and resources needed to migrate to a commercial system versus modernize the current system. However, there is one constraint that is limiting the business moving to a commercial version at this time: There are five months until the winter shopping season freeze, when all changes must be completed with no further changes to the systems until February of the following year. Most of the time estimates from integrators for the commercial version are more than 12 months in duration.

Steve’s company is really between a rock and hard place. If it doesn’t fix the system Steve is managing, it may not be able to scale to the volumes the company needs to be profitable this holiday season, but the company has waited so long to decide to move to a commercial package that it won’t be ready in time for the holiday buying season regardless.

Likewise, Steve and team are in a similar difficult position. Their efforts around a modernization program have yielded some potential wins that could allow the business to achieve many of their desired outcomes before the winter shopping system freeze. However, there is a culture of fear that permeates Steve’s team. Whenever this system has failed in the past, there were significant reprisals from the business in the form of multi-day “all hands on deck” sessions, where the entire team was expected to be there from 7 a.m. until 8 p.m. until the system was operational again, not to mention letting go those who introduced bugs into production releases.

Stacey is Steve’s head of development. She brings the modernization plan to Steve for review, but makes it clear that the team believes there are some potentially high risks associated with the approach. While the Stacey and team believe they will be successful, they want assurances from the business that they have the leeway and support to take the risks without reprisal. In return, Stacey tells Steve that there ares certain changes that the team will implement to limit downtime if something should occur:
1. They will adopt a strategy that developers are responsible for fixing anything they break.
2. They will be implementing a new continuous integration and deployment tooling that will automate the integration build, unit testing and review of any code changes speeding release of updates and fixes into production.
3. They will be adopting the Agile Scrum software development life cycle to deliver functional changes every two to three weeks.
4. They will set up a Kanban board online that will allow the business to track and understand where the proposed capabilities are and when they will be completed.
5. Before each sprint the business will have a chance to reprioritize any features or capabilities so as to ensure that the most important features get completed prior to the system freeze.
Steve likes Stacey’s plan and agrees to take it to the business. He explains to Stacey that there will be a natural tendency to push back, but given the urgency and lack of options, this may very well be the best time to push for these changes in their company. In the past, there has not been a lot of tolerance where IT systems impacted financial performance. It’s now Steve’s job to help the business understand that, with the growing dependency on software in business, its expectations need to change for the business to successfully compete.

— JP Morgenthal
December 18, 2017
The Executive’s Guide to Microservices: Chapter 1

Meet Steve. Steve is responsible for a mission-critical application for a very large enterprise. He has a problem: It takes Steve and his team a very long time to make changes to this application and release it to production. Steve has been talking to vendors, consultants and peers about agile software development techniques and DevOps, but Steve and his team have assessed that his application does not lend itself to these approaches.

See, the system Steve looks after is called a monolith, which is a very confusing title since it was developed using the latest modular software development approaches when it was created, in the latter half of the 1990s and early 2000s. Steve and his team are constantly advised that they need to decompose their monolith into microservices. However, this advice only confuses Steve and his team even more, since the system was designed using a service oriented architecture (SOA) with the intent of ensuring that it was easier to maintain and scale.

Steve’s team fell into a very common trap associated with many SOA-based designs: The services were designed with deep dependencies, they share a common database and they are very coarse-grained. Indeed, the collections service, which is one of eight services in the system, handles every potential operation a collections agent might ever need. Modifying any operation within the collections service means the entire collections service must go through regression testing and production must enact a very cumbersome deployment strategy that requires a five-hour maintenance window.

Needless to say, Steve cannot keep up with the requested changes from the business with these constraints. Steve’s advisers have given him some good advice on organizing to develop faster, but he would only be applying Band-Aids faster. However, once the constraints identified above had been pointed out to Steve and his team, they better understood the challenge ahead of them.

Steve and team are going to have to redefine the data models so they support service independence, then extract subsets of functionality out of the large, intertwined services to deploy as within a well-defined business context—following the “do one thing well” mantra. They then can start to extricate the complex business logic trapped inside the enterprise service bus so it can be incorporated into the services at the edge.

Steve and his team realize that continuing to enhance the monolithic code base avoids the challenges of redevelopment but reduces the value of that codebase to the business. While there will be a cost of untangling the monolith, the cost of not doing so is lack of competitiveness, lack of agility, long wait times for new features, possible opportunity loss and continued investment in technical debt. Steve’s job will be to present the justification for this to the business based on these risks.

When the task is completed, Steve and team will be able to deploy new services and modify existing services within weeks. The costs related to development and testing will drop significantly and the finer-grained services will allow developers to address business needs more quickly.

We’ll check in with Steve and his team as they embark on this journey and deal with the challenges ahead.

— JP Morgenthal

November 20, 2017
Welcome to Microservices Journal

Welcome to Microservices Journal, a community dedicated to the design of microservices-based applications and related technologies. The introduction of microservices into the digital application landscape enables a new level of agility for businesses that adopt this model of design. As with any architectural construct, there are no absolutes; hence, no wrong or right way to build microservices-based applications, however, there are some well-accepted tenets, patterns and anti-patterns associated with this design approach. Microservices Journal is being established as a site where practitioners can share their points-of-view and their experiences.

Following in line with emerging agile development approaches, we are introducing Microservices Journal as a minimal viable product and will iterate rapidly to add new features and content. As with any community, we will evolve and thrive with contributions. These contributions can come in the form of blogs, articles, ebooks, podcasts and videos. Soon, we will add discussion forums, deliver webinars and hold live Q&A sessions with renowned experts in the field.

In addition to the website, we are also creating Microservices Day events to be held regionally. Our first event will be in Austin on November 16, 2017 as part of the Innotech Austin event.

Like any community, we’re only as good as our contributors and contributions, and we look forward to your contributions.

If you wish to contribute, please email editor@microservicesjournal.com

— JP Morgenthal

October 16, 2017

Microservices Days coming to a city near you

Microservices is rapidly becoming a design paradigm for developing software. While there are many benefits to microservices designs, it is more complex than traditional application design. For this reason we are formulating a new community around microservices architecture. This community will explore design issues and the intersection with Mobile/IoT, Containers, Cloud, DevOps and APIs.

Microservices Journal is a newly-launched online community where practitioners can share their experiences and learn from others.

Microservices Days are a series of regionally delivered one-day seminars led by practitioners.

Who should attend: CIO/CTO, Application Development Management, Software Engineers, DevSecOps, and IT Operations.

For our 11/16 program we are lucky to have some strong industry experienced speakers come speak:
• Michael Cote – Pivotal
• JP Morgenthal – DXC.technology
• Deepak Kulkarni – Charles Schwab & Co
• Irakli Nadarishvilli and Mark Donovan – CapitalOne
• Ryan Bagnulo – Guarden.io

This is a great opportunity to come hear from early adopters and learn from their experiences.

Register Now: http://www.innotechconferences.com/austin/about-2/microservices-day/

Microservices Days, Austin – Agenda

Time	Presentation
8:00am	Opening Keynote The Future of Work: Mind-shift from Robots vs Humans to Robots Empowering Humans
9:30am – 10:15am	The Death of Enterprise Architecture: Defeating the DevOps, Microservices, and Cloud-Native Assassins
10:20am – 11:05am	The Decomposition Dilemma
11:10am – 12:00pm	12 Factors for the Enterprise
12:00pm – 1:00pm	Lunch and Expo
1:00pm – 1:45pm	Implementing Microservices at Capital One
2:30pm – 3:20pm	TBA
3:30pm – 4:20pm	Microservices Maturity Matrix, Token Authentication & API Gateway Mediation

Register Now: http://www.innotechconferences.com/austin/about-2/microservices-day/

— JP Morgenthal

October 16, 2017

SOA vs Microservices

Some say microservices architecture is proof that SOA is still alive. I contend that microservices architecture replaces SOA due to deficiencies in how SOA has been implemented as well as the original intent.

In the mid-1990s there was a surge of investment for implementing enterprise applications to replace the aging homegrown legacy applications. These new applications managed complex activities around accounting, supply chain, human resource and manufacturing. They also were easy to integrate with other facets of the business, including existing legacy applications that were not going away and new and emerging web applications for customer and internal use.

Out of necessity, there was a rise in the number of point-to-point integrations occurring. Over time, the web of point-to-point integrations became so complex to manage that even small changes resulted in major failures. To correct this problem, a category of software emerged to make integration easier by simplifying the control plane that focused on cataloging the endpoints, fostering connections and marshalling the data. These control planes were identified as enterprise application integration (EAI). Early EAI products did not inspect or act on the data.

Somewhere along the line, vendors became aware that there was another underlying issue: The old system and the new system used different vocabularies and message structures. Initially, the consumer of the EAI tool output did the translation itself, but then vendors saw an opportunity to increase the capabilities of the EAI tooling to also become a data plane. So, then, the EAI tooling would also handle mapping and translation.

Now, if your source systems never change, the value of an intelligent middleware that acted as a control and data plane seems both logical and of great value. However, when the source systems change, middleware comprised of dynamic routing and mappings from the source to the target becomes a considerable limitation. Any change to the source requires comprehensive regression testing on the entire system, and you must hope you have an example of every permutation of every message that may ever be delivered as input to ensure the change does not introduce future problems.

Having been an integration consultant during this era, I can attest to weekly failures due to message combinations that no one had ever conceived of during testing.

So, what does any of this have to do with SOA and microservices?

The above approach I just described evolved with the products that acted as a control and data plane morphing into the enterprise service bus (ESB). The ESB became the cornerstone of the SOA strategy and together they formed the strategy for modernizing legacy systems. Hence, SOA, as it was ultimately delivered, ended up being an expensive mapping exercise from legacy data structures and messages to a newly appointed business language, also called the canonical representation. In some cases, these new SOA services would derive these new business data structures by querying and massaging data from multiple legacy systems, while in other cases, new business logic would be made available via the bus expanding the ecosystem of available capabilities.

With SOA, the canonical message became the product of innovation and, hence, ultimately its downfall. Perhaps unfairly so, but, SOA never fostered continued modernization of the legacy systems into re-platformed and redeveloped services. It was about exposing the systems of record, both data and process, for inclusion into business processes, facilitating minimal changes to the legacy systems all handled through intelligent pipes.

While digital transformation may be a popular concept that every business is now chasing, SOA’s failure demonstrates that simply exposing the existing systems was not enough to become the agile business we strive to be. Indeed, the systems of record that are brittle, burdened with technical debt and developed in a monolithic fashion were already impairing businesses’ ability to compete long before Uber and Airbnb came into existence. The appearance of these businesses just increased the urgency to address the legacy issue.

Enter microservices architectures. First, I’ll reiterate my favorite soapbox topic: Putting a legacy app inside a Docker container does not in and of itself define a microservice. A microservice follows specific tenets of design. One of these tenets—and one that is very relevant to this discussion—is smart endpoints and dumb pipes. I find this design principle very interesting, considering what I’ve stated above about SOA. For me, it’s clearly a rebellion against SOA strategies.

Also, I believe microservices approaches legacy modernization from a slightly different perspective. Microservices architectures by nature focus less on tooling (they’re polyglots by design) and more on the contextual bindings to business domains. Additionally, as someone eloquently stated on Twitter (which I cannot find now and so will paraphrase here), a microservice should be designed to be easily deleted without impacting the operations of the system. I believe the statement illustrates that a well-designed microservice isn’t intertwined with other services nor does it share intimacy of design.

Some will debate—present company included—that this is what SOA was supposed to deliver. Unfortunately, this was not how it got implemented. So, this time we reduce the scope. Instead of a SOA service that represents the entire business domain, we deliver smaller, more well-focused entities representing subsets of the business domain. Subsequently, these services are designed to operate more independently, allowing them to better align with greater use of agile techniques.

In microservices, architecture agility is the product of innovation. Ultimately, the improvements in speed, cost and quality are why microservices will succeed where SOA has failed.

— JP Morgenthal

October 16, 2017
The Big DevOps Question: How Do We Get Started?
Every conversation I have with clients regarding DevOps eventually reaches a point where the client wants to know my recommendations for getting started with a DevOps program. Some have already embarked on a DevOps program, but want me to critique their approach against what I see others doing. Others simply don’t have a clue how to make the necessary changes in their organization.

Ultimately, I sense in all these conversations a lack of confidence that their intended outcome is achievable and that they’re on the right path.

The first questions I ask clients are, “What are you looking to accomplish, and what are your business drivers?” Interestingly, 90 percent of the time clients are unclear why they want to embark on this journey. Sure, DevOps makes sense academically, but if you don’t have a clear understanding of your expectations for improvement most likely you’ll end up with more questions.

Sometimes it helps if I give prompts:
- Are you looking to improve quality in your production releases?
- Do you need to release updates and patches into production more quickly?
- What metrics are you currently looking at, and what makes you believe these are not optimal for your business?
This last question is really important. Quite often data represents invalid perspectives. Other industries’ and services’ metric of how often they deploy to production or how long it takes execute a release cycle might not be right for your business or industry. I hope medical device and airplane equipment manufacturers take a lot longer to bring something to production and test it more robustly than Netflix does its website.

In an earlier post I wrote that the need for DevOps is effectively the admittance of a problem; to wit, the cure is continuous delivery. Hence, I often tell clients that a very realistic place to start is not with the actual work, but with the communications methods and pathways that will affect implementing continuous delivery within the organization.

So many clients want to start their DevOps journey in application development and talk about what they’ve done with continuous build. That’s fantastic, but it has more to do with Agile software development than it does DevOps. If all that work creates bottlenecks in testing, you’ve accomplished very little. Now you must fix the flow to balance out development and testing. Trying to achieve continuous development through serial analysis and correction will take a long time and show little progress along the way—and often leads to abandonment.

However, if all the leaders of all the various areas involved in the continuous delivery cycle discuss how their efforts are impacted by what the other leaders’ groups do, everyone can see the bottlenecks and constraints and they can prioritize changes. Management often fosters much of the cultural impairment, as these changes can impact everything from workload to bonuses. Getting everyone involved in working together to eliminate the bottlenecks often buys credibility for the program and minimizes some of the individual adoption issues.

Don’t jump into the deep end of the DevOps pool just because everyone else is. Before embarking on your DevOps journey, do a deep analysis of your organization’s goals and baseline your current levels of performance. Discuss what needs to change based on real business goals and drivers (e.g., it takes us too long to issue regulatory changes to the system). Identify bottlenecks and constraints that exist in the current environment today and put together a plan to eliminate as many as possible.
January 18, 2016
Leveraging DevOps to Expose The Entire IT Iceberg

On September 16th I attended and spoke at a CIO Magazine event in Chicago. These are some of the best events to attend as the audience is typically CIO or senior IT leadership. One of the speakers on the day, Martha Heller, Author of “The CIO Paradox”, presenting on new competencies for the modern CIO, presented a very eye-opening perspective on limitations moving IT forward into a digital age.

Anyone who has worked in IT for any length of time, or even watched the British comedy show, “The IT Crowd,” probably has some concept of what it means to work in IT. But never have I seen it so explicitly spelled out from an executive level as Mrs. Heller does in her talks relating the brutal reality as expressed by one of her clients. I present the excerpt here from her talk:

“Do not under any circumstances expect applause or a raise when you have dismantled the iceberg. Since no one in the business sees the iceberg and no one understands the iceberg is real and that it is ours they cannot understand what used to be under the water looks and behaves completely differently. Business leaders will understand cost, headcount, throughput and all of the new capabilities that run on top of iceberg. These are the likely places for applause. Expecting recognition and compensation for what you do below the water line is not advised; do not fight that battle. Praise for what happens under the water line only comes from industry peers.”

Clearly the iceberg is IT itself and the waterline represents the separation between what the business can see and touch, e.g. user interfaces, accessibility of data, application performance and IT support, and all the stuff that enables the aforementioned that sits below the water line. If advice by counsel, such as Mrs. Heller, to CIOs is not to expect recognition or compensation for making what happens below the water line work optimally, then it will be very difficult for CIOs to engage and keep personnel that excel in this capacity.

To be clear, I’m not saying that Mrs. Heller is telling her clients not to pay attention. She is merely reiterating the paradox that all CIOs must confront as they inherit and deal with the ever growing mess below the water line caused by acquisitions, aging systems, disruptive technologies, aging labor pools leaving the work force, and workers going to companies where they receive better pay and recognition by because they focus on providing below the water line management services, etc.

The industry has been telling CIOs for at least three years now that they need to pay attention to what is occurring with DevOps. Yet, many still view DevOps as a deep below the water line activity focused on development and system administration. What they are ignoring is that DevOps may very well be the answer to this paradox that Mrs. Heller puts forth. Leveraging DevOps is not only advisable, but necessary for these leaders to succeed.

DevOps has the ability to help lower the water line with the business. As Mrs. Heller reiterates, the business understands costs, headcount, throughput and all the new capabilities that run on top of the iceberg. We have concrete evidence that companies that adopt DevOps philosophies and institute continuous delivery capabilities are able to respond to business changes more rapidly and are more profitable than businesses that are hindered in these areas. DevOps typically increases the level of transparency between IT and the business giving them greater input into direction and providing them a wider perspective for how what is below the water line today impacts the very attributes the business cares about.

Increased automation associated with continuous delivery opens up greater opportunity for self-service, which again moves more of what is below the water line today to above the water line. Self-service provides more touch points for the business to directly control their own agenda and direction leveraging IT resources. Indeed, there are additional ways that DevOps can help to lower the water line that for readability I will not go into here.

Mrs. Heller’s point that attempting to battle the business to gain recognition for what happens below the water line is a losing battle that will tear at the credibility of the CIO that chooses to engage in this battle. However, if CIOs cannot hire and retain the best talent, their ability to stop the iceberg from growing below the water line will continue to diminish. Their only hope is get the water line to drop and expose more of the iceberg. The more of the iceberg that gets exposed the more opportunity the CIO has to obtain recognition and compensation for his/her team.

So, Mr./Mrs. CIO, is DevOps important to you now?

September 28, 2015
DevOps and Bi-Modal IT
Can a business have two competing approaches to delivery and operations of their information infrastructure and be successful? If you follow Gartner recommendations, it would seem that bi-modal IT is the way to go. Bi-modal IT has been explained many different ways, but, ultimately, comes down a failure in leadership; allowing a subset of IT to continue to operate in an non-optimal way because it’s too expensive and difficult to transform. This results in formation of a secondary subset of IT leveraging emerging trends that have demonstrated results to deliver modern information infrastructure faster with reliability.

If we examine from the 50,000 foot view, bi-modal IT actually makes a lot of sense. Disruptions are happening everywhere and businesses need to respond quickly. For many of these businesses IT can barely keep the core systems afloat let alone bear the burden for handling creative exploitation of massive data growth, enhanced customer experience and transformation to new compute platforms. A nimble team with the right skills can facilitate forward movement in these areas with little disruption to the core business. Moreover, they can establish a framework and approach that enables the core business to adopt when ready.

The issue is when we move down to 5,000 feet, this strategic viewpoint becomes a series of tactical actions that have the potential to spell long term disaster for the business. For example, the data that feeds many of the new digital applications is actually housed in systems being managed by the “traditional IT” group, which means that the “fast” team now is reliant upon the “traditional” team to enable access. Needless to say this is fraught with many hurdles; everything from heel-dragging due to animosity to lack of skills to lack of slack time needed to make the changes to support. While the stick might help with heel-dragging there’s little it can do to fix the lack of skills or slack time issue.

Technical and operational debts are finally coming home to roost for many of these businesses. Lack of attention to automation, documentation, training and staffing are now creating a Sophie’s Choice for many IT and business executives.
- Where do we invest?
- Do we spend time and money making what we have work better before addressing the needs to compete in a digital economy?
- If we do this, how far behind will we fall?
- If we follow a bi-modal IT strategy, do we risk losing critical resources needed to support our existing systems?
For years we’ve been telling the business that downsizing and budget cuts for IT is a mistake. The reduction in headcount and budget continued to squeeze existing resources to take on more responsibilities. As we now know through DevOps research the larger the quantity of work-in-progress being managed the more likely the increase in defects during the build process and the higher the likelihood of outages in production.

If there is a possibility to escape this conundrum, it most certainly is not bi-modal IT. Leveraging a nimble team to meet the needs of digital economy is fine, but they must work cooperatively with the traditional IT teams. Prioritization of backlog and increased transparency with the business is critical for moving forward. The business has to accept they can’t do everything they want that’s new and keep all that’s running today. Specialization must be reduced through standardization to reduce complexity and bottlenecks. Increased automation efforts will reduce time spent on manual operations and afford time to support other key efforts. Visualization and communications of work-in-progress across teams will minimize conflict and foster faster prioritization of critical efforts. Finally, moving to fewer changes with increased number of releases will enable the business greater opportunity to prioritize efforts.

In summary, one could say that the fix for IT isn’t going bi-modal, but adopting continuous delivery and lean IT approaches.
May 8, 2015
DevOps is a Symptom, Not a Diagnosis, and Continuous Delivery is the Cure

You wake up and you have a sore throat and nasal congestion. Is it the common cold? Perhaps the flu? Is it bacterial? Viral? Any good physician will tell you that diagnosing disease purely from symptoms is a fool’s errand (or maybe malpractice). A diagnosis identifies disease which leads to prescribing a treatment plan for eliminating or placing in suspension the symptoms. In a similar vein, DevOps is not a diagnosis either. DevOps is merely the symptom(s).

Initially, DevOps was the recognition that development and operations were very often not aligned to a common goal of satisfying the business customer. Over time DevOps became representative of other symptoms of IT dysfunction, such as configuration drift, poor release management controls, lack of testing, high numbers of defects during development, and a culture of finger pointing when systems failed. Again, these are just more symptoms of the same diagnosis, which is application delivery dysfunction.

This realization that DevOps is a symptom has been a long journey for me dating back to July of 2011. At first, I viewed DevOps as the modern day version of the Haymarket Riots. It was a way for the overburdened and under-appreciated efforts of the IT workers responsible for delivering and maintaining today’s systems to take control and bring some stability to their jobs. From there it transformed into the movement focused on dealing with the technical and organizational issues that impact application delivery.

Today, DevOps seems to be a catch-all for so many facets of application delivery that it’s quickly becoming a useless term. When a sales executive calls me and says a client is interested in DevOps, I respond by asking for more details regarding their interest areas, such as continuous integration, release management, continuous testing, etc. To help them understand the comprehensive nature of their request, I put together a checklist of items for them to review with the client.

Perhaps, even more frustrating than the uselessness of the term DevOps in deciphering need, are the conversations with people that say, “we’re doing DevOps,” but refuse to drill down into what they mean. It’s very reminiscent of conversations where businesses state, “we’re doing cloud already.” However, if we look at DevOps as a symptom, and you say you’re doing DevOps, I’d look at you and say, “I bet you are.” After all, many businesses are struggling with application delivery. And as your IT doctor, the real fun of giving you a checkup and identifying the causes of your application delivery dysfunction diagnosis can then begin.

Ultimately, I came to the realization, DevOps doesn’t solve anything (or for some it solves everything, either way its lacking focus) , which led me to the realization that DevOps is the canary in the coal mine highlighting the need for change in the way we build, release and operate applications.

So, if DevOps represents the symptoms and application delivery dysfunction is the diagnosis, what is the treatment plan? Continuous Delivery. Continuous delivery is to the application delivery pipeline what enterprise architecture is to all systems in the business. It’s a holistic approach to enabling the business to leverage its IT assets to operate better and innovate faster. Achieving maturity in continuous delivery doesn’t mean you’ve fixed every issue limiting application delivery, but it does imply that business has taken an honest look at it’s bottlenecks and constraints and are removing them when possible thereby constantly improving.

Interestingly, if you accept this premise, the connection to lean manufacturing often associated with DevOps, and illustrated by The Phoenix Project, makes perfect sense. That is, as we seek to improve the continuous delivery pipeline we can use tools and practices from lean manufacturing, such as value stream mapping and Kanban designed for this type of problem. Following on our analogy, lean tools and practices become the pharmaceuticals supporting our treatment plan (continuous delivery) eliminating our ailment (DevOps).

February 23, 2015