For more than two years, I have participated in a number of panel discussions at Boston DevOps, our local DevOps meetup. Panels have been put together from several disciplines to address issues through the successful adoption of DevOps and reaching optimized use of this current methodology in software development.
As relevant as the DevOps challenge is for every technology organization, it remains elusive. Many groups simply are not cracking the code and in some cases simply do not know how to effectively start the change. Check out SMART DevOps as a way to get started.
A recent panel in which I participated was set up as an informal question-and-answer session, including questions from attendees viewing a live stream of the event. The introduction by one of the panel members laid out the complexity of issues required to optimize DevOps in an organization. It was clear from the audience questions, as well as those from the live stream, that there was discontent among representatives of all three groups who have been combined into a DevOps function–issues related to implementation and ongoing results. As I prepped for the panel, I came upon a Gartner Group article that had an extremely interesting quote in it: “The DevOps trend goes way beyond implementation and technology management and instead necessitates a deeper focus on how to effect positive organizational change. The DevOps philosophy therefore centers on people, process, technology and information. With respect to culture, DevOps seeks to change the dynamics in which operations and development teams interact.”
One of the questions from the live stream was: “If DevOps is so great, why is everyone so unhappy?” Another question from the panel moderator very clearly and succinctly asked about the kinds of cultural change happening in organizations, separate from new tools and new practices. The commentary from some technology-oriented panel members precipitated the rapid disintegration of audience discussion into a heated debate about tools and technology.
Observing that rapid switch away from very “soft” questions about culture to what tools will and will not support aspects of DevOps was enlightening and frightening at the same time. It was very apparent to me that this major shift in how information systems organizations were structured and managed was a throwaway in terms of organizational change. The Nike phrase “Just Do It” comes to mind.
The issues were obvious to me. In no way had most organizations viewed the combination of three separate, virtually autonomous departments (one of which is adjacent to customers) each with a different culture, as an organizational change. And, by the way, a change to be accomplished while maintaining a 24-hour idea-to-delivery cycle, with each new idea coming within seconds of the last. The departments in question were not filled to the brim with sales and marketing extroverts. Rather, they were groups and individuals who were used to looking at the problem in front of them and solving it without having to resort to communicating with others. Like its close relative Agile, DevOps relies heavily on communication and conversation to be successful.
The situation is a wake-up call to technology leaders and managers. If you are working in an organization that has embraced DevOps with the goal of providing customers with what they want almost immediately, check in with all of your technology teams. They may be floundering in what they did not perceive as an organizational change. If you are a technologist trying to implement DevOps successfully and suspect that there are underlying organizational issues, there are several things you can do to be effective and help your organization realize a successful implementation. Start the conversations that need to happen to have optimized DevOps, learn some of the principles of collaboration and consensus, and draw upon the expertise of outside resources who can facilitate you and your teams through the organizational complexities of a transition—“a philosophy, a cultural shift that merges operations with development and demands a linked toolchain of technologies to facilitate collaborative change.”
On March 15, I gave four keynote speeches on DevOps in Toulouse, France. As the main speaker of Econocom Club MEP 2018, an annual think tank summit organized by the European business-to-business digital services provider, I shared the day with 15 firms in industries as prestigious as financial services, energy, aerospace, aviation, and telecommunications and heard all about their implementation projects.
The lessons learned from this year’s summit are troubling, they tell a story the vast majority of DevOps vendors aren’t prepared to hear: They miss their client expectations. There’s a severe disconnect between what’s expected and what’s delivered.
Participant visions and approaches didn’t surprise me; they simply confirmed my point that DevOps is misunderstood. By stressing the implementation of Continuous Delivery (CD) toolchains—selling technology is more lucrative than transforming a business—IT service providers aren’t helping.
What do these lessons tell, exactly? if you’re an open-minded leader who understands that a new IT service delivery paradigm more focused on business concerns is needed, this is for you.
The Untold Frustration
In his opening speech, facilitator Fabrice Berne was inspired to ask participants while they introduced themselves to share the purpose and challenges of their DevOps implementation project.
What I heard didn’t surprise me; it was a mix of fascination and frustration—participants were at the same time excited to work on a DevOps project and frustrated because they weren’t getting the promised business benefits.
“The reason is, you all tackled your DevOps project the wrong way,” I said. “You started by implementing the CD toolchain, thinking tools on their own will deliver the expected business benefits. That illusion is widespread in the IT community!”
So, What is DevOps?
As my claim turned the atmosphere icy, I added, “DevOps can’t be narrowed to these tools—not even to agile practices and automation. It’s more than that. DevOps seeks to make the business competitive, wealthy and healthy by continuously delivering high-quality software.” And concluding my introduction, I made it clear: “The fundamental question is, How is business value created? Many experts, particularly techies, struggle to answer it.
“Let me say this straight: Most DevOps projects fail because certain experts erroneously spread the idea that the most important piece of DevOps is the CD infrastructure,” I said, adding, “The fact of the matter is these experts and the IT solution vendors they represent are creating confusion; through massive marketing hypes they force CIOs to address DevOps through the distorted lenses of the so-called DevOps tools.”
The Inconsistent Disparate Visions
The following chart results from a survey we conducted at ITaaSNow last year. It involved 2,681 businesses that were unhappy with their cloud and DevOps migration projects; we wanted to show how poor DevOps approaches lead businesses to failure:
Figure 1 – Disparate and erroneous visions that cause implementation failure
“As you can see, there’s no homogeneous perspective right now,” I told the audience. “It varies according to the business culture, objective and strategy or the DevOps architect’s understanding, experience and expertise. In most cases, these customized visions are incomplete and generate no business value at all.”
I picked two failure causes in the chart to demonstrate my point.
Starting with the notion that, “automation is the core of DevOps,” I argued, “DevOps primarily seeks to help generate business value. Generating business value is about identifying market and customer expectations, translating expectations into profitable service ideas, developing, deploying, promoting, selling them and continuously improving customer experience.
“Where did techies get the idea that to make a business profitable the only thing to do is to implement Git, Jenkins, Cucumber and Chef into a CD toolchain?” I asked, somewhat ironically.
Picking a second cause, “DevOps strategy should only stress software development activity,” I continued. “Thinking that isolatedly software development activities will generate business benefits without involving marketing, sales, testing and operations is a pure illusion. Once again, generating business value is a complex process that covers identifying market and customer expectations, translating expectations into profitable service ideas, developing, deploying, promoting, selling them, and continuously improving customer experience.
“Where did solution vendors get the belief that, software development alone is enough to generate business value?,” I asked. “That’s another myth they nurture!”
I then warned: “Make no mistake about it: All the visions represented on the chart are either incomplete or wrong. They do not reflect what DevOps is. You must acknowledge once and for all thatit is a business transformation effort, not only an IT capability implementation. Period.”
The Contrast with the Founding Fathers’ Vision
“Who better than its founders can provide the right vision and definition of DevOps?” I asked.
In his article, Adam reminded us, “Becoming better at building and delivering software is the stake. It’s not an optional piece of their strategy: It’s the future of how their customers want to work with them. Software is table stakes for survival.”
He added, “But the real answer lies in looking at what John, Paul, and Jesse were doing in 2009—not in the specific technical choices they made, but the style in which they worked, the essence of what they believed made them high functioning and successful. … Then the challenge begins: how to apply this new style to businesses trying to become better at building and delivering software.
“DevOps is about bringing together all the people you need to build and run your business effectively, and empowering them to move as quickly as possible toward their goals. Tools matter. Make no mistake: Trying to change the way you work without changing the mechanisms by which you do that work is a futile exercise in excruciating failure.”
He concluded, “Fundamentally, it is about taking the behaviors and beliefs that draw us together as people, combining them with a deep understanding of our customers’ needs, and using that knowledge to ship better products to our customers.”
With Respect to Techies, DevOps is Primarily a Business Operational Model
“If you paid attention to what Adam Jacob reported about Jesse, John and Paul’s vision, you probably noticed that it emphasizes two fundamental elements of high-performing businesses: the well-known IT infrastructure and the unknown operational model,” I told the audience.
Moving to the next slide, I provided evidences to my point, “As the picture shows, DevOps, according to its founding fathers, isn’t the CD platform, automation toolchain or the platform-as-a-service (PaaS) infrastructure solution vendors and certain consultants would like you to believe.”
Figure 2 – The Founding Fathers’ Vision
The slide seemed to clear up a doubt about what the audience had been hearing. I confirmed, “With all due respect to my fellow techies, as I’ve been implementing it, DevOps transcends software development issues; it’s a business operational model supporting your company’s competitiveness.”
Figure 3 – DevOps as the Business Value Chain
I told the audience I chose to represent it as a two-component system “because I wanted to make sure IT professionals and business leaders understand that implementing DevOps is primarily a business transformation effort for optimizing the organization’s operational model and IT infrastructure.” I added, “The operational model is the subset of interactions, staff and skills, processes and practices, values and beliefs, and tools that’s repeatedly mobilized across your organization to generate the expected business benefits. It’s where all its value propositions and competitive advantages—organizational agility, flexibility and velocity, innovation culture, more like it—are created. As (fellow speaker) Stephanie Carton puts it, the operational model is where DevOps culture is implemented.”
Anticipating the audience’s reaction to that point, I said, “It doesn’t mean CD toolchains are useless or doesn’t bring value, without them we’re dead. The point here is, business benefits result from an optimized operational model enabled by a well-engineered CD toolchain.”
I’m not the first one to go against the techie perspective of DevOps, I warned. “Ernest Mueller in his fantastic, “DevOps: It’s Not Chef and Puppet,” raised the red flag.” His message is clear: “If someone has a systems problem, and you say that the answer to that problem is Chef or Puppet, you understand neither the problem nor the tools.”
The Near Future of DevOps
In the first part of this article, I wanted to share with you the clarifications I made and more importantly, re-establish the truth on messages as erroneous as “DevOps means CD toolchain and Agile practices,” and “CD infrastructure or software development on its own creates business value.” They’re misleading CIOs and are taking hundreds of projects straight into the wall!
Figure 4 – DevOps Big Picture
In part 2 of this article, next month, I’ll tell you what we debated about the near future:
Why Agile and Lean practices will boost business competitiveness?
What is ITaaSNow DevOps Capability Maturity Model (CMM)? How does it enable continuous business performance improvement?
What is Artificial Intelligence for Operations (AIOps)? Why will it make DevOps the cornerstone of businesses’ digital strategy?
Stay tuned!
The Key Takeaways
This what I wanted to tell you about DevOps as we discussed it. The key takeaways are:
DevOps isn’t narrowed to agile practices and CD toolchains. Don’t fall into the trap of certain solution vendors who tell you that Git, Jenkins, Cucumber, Chef, Containers and microservices will boost your business performance. It’s false; things aren’t that simplistic.
DevOps combines an optimized operational model and a CD platform to bring together marketing, sales, post sales, software development, testing and operations into an application delivery capability that boosts your organization’s competitiveness.
Starting the CD toolchain implementation without optimizing the operational model is the surest way to take your DevOps implementation project into the wall.
DevOps implementation is primarily a business transformation effort, not just an IT capability deployment effort.
What matters isn’t applications and digital services delivery at the speed of light, but the timely delivery of added-value applications and digital services to the right market segments and business users.
According to Econocom’s feedback, most participants are reviewing their DevOps implementation strategy; they will make their operational model optimization the priority. Mission accomplished!
Making the transition to DevOps requires as much a shift in cultural mindset as it does new platforms for building, deploying and managing applications. Between the two, however, it’s often the extent of the cultural challenges needed to be overcome that gets most underestimated.
Michael Elder, a IBM Distinguished Engineer, said one of the most important DevOps philosophies that organizations must embrace is that No Code is Sacred (NCIS). NCIS fosters an experiment-driven culture that encourages self-editing and promotes a “don’t be afraid to fail” mentality. This approach allows teams to form questions, quickly implement means to gather data, try alternatives and incorporate real user behavior and feedback, he said.
Manage the Scrum (Meeting)
Other cultural issues that need to be addressed include leadership and how daily scrum meetings are managed.
An overzealous build champion can wind up adding technologies to the continuous integration/continuous deployment (CI/CD) platform that don’t get adopted simply because the team as a group did not agree to their inclusion, said Elder.
Daily scrum meetings don’t necessarily need to take a full hour. Non-critical, time-consuming status updates should be eliminated from the agenda. Daily scrum is more about problem-solving and looping the wider team in on critical decisions than talking about updates most team members can already read about for themselves, Elder noted. The goal of the meeting is not necessarily to inform team members of events. They should be expected to know what’s occurred. The real goal of the meeting is to decide what to do next.
Mindsets Matter
Elder said DevOps teams be cognizant of Conway’s Law, which says that organizations that design systems are constrained to produce designs that are copies of the communication structures of these organizations. If the goal is to develop a more agile application environment by employing microservices, Elder said, then the organization should create smaller, more independent teams to construct and manage them. However, that doesn’t mean every monolithic application needs to turned into a set of discrete microservices.
Elder also advised organizations to embrace Design Thinking principles that encourage DevOps teams to focus on what needs to be achieved instead of enabling processes. The focus ultimately needs to be on the experience of the users who are being asked to employ an application, he said.
The Tools Issue
The biggest cultural challenge, however, still may be to tooling. Organizations trying to embrace DevOps clearly need new tools. But acquiring new tools won’t drive meaningful change. Too many organizations buy tools in the hopes that processes will eventually be transformed. The decision to acquire new tools needs to be closely connected with the transformation of processes, said Elder. That said some tools, such as an enterprise instance of Github, are core to encouraging the sharing and reviewing of code within a DevOps process, he added.
Of course, Elder noted the most critical cultural change as far as tooling is concerned is keeping an open mind. Developers and IT operations teams tend to get overly attached to specific programming languages and platforms. A true DevOps culture, above all else, encourages experimentation.
I spend my days discussing DevOps and digital transformations with companies of all sizes, from startups to some of the world’s largest enterprises. I’ve found that many teams are still at a very early stage in modernizing their technology, culture and practices. However, I’m surprised at the number of the organizations that don’t believe in change because they “have been doing just fine up until now.”
This is a recipe for failure in the long term. Their competitors will likely be working in loosely coupled teams with shared responsibilities and increased collaboration. They will have adopted continuous integration, continuous deployments/delivery and automation, to name a few processes. In time, these competitors will begin to consume the laggards’ market share. To put it into perspective, you might still be talking about your next deployment coming up in six weeks time, while they are talking about deploying into production 12 times today alone.
With their increased agility, stability and reliability, these more successful teams will be faster. They will not be afraid of failure and will have built fault-tolerant systems that can rapidly and automatically repair. These self-healing, well-tested systems and processes will free time for developers and other team members to focus on developing newer and better applications in response to customer feedback—all while the your team might continue “doing fine.”
We aren’t just talking about buzzwords here; the data shows that those who are playing catchup are also potentially causing themselves more harm than good. Puppet’s “2017 State of DevOps Report” shows that the gap between higher- and lower-performing teams has narrowed as throughput has increased, meaning lower-performing teams are working faster, shipping more code to production and responding to changes quicker than they were the previous year. This is great in principle, but in 2017, change failure rate had also increased, showing that low performers were actually more likely to fail when making changes than they were in 2016. The data shows that low performers had a 3x change failure rate in 2016, which increased to 5x in the 2017 Puppet study. From a business standpoint, this means that those higher-performing teams are able to deliver more value to their customers by responding faster and more reliably to feedback and market demands.
Companies should certainly be paying attention to what their competitors are doing, no matter what area or sector in which they operate. However, they also must not lose sight of the importance of quality, security or feedback during this journey. If you know you are at a much earlier stage in your transformation, focus on getting it right. There will always be setbacks and challenges, but as the old saying goes: This is a marathon, not a sprint.
DevOps methodologies, when applied effectively, will make your organization more successful.
DevOps principles are built on an effective, continuous cycle of improvements—key factors that help drive digital transformation for companies large and small. Inserting this agile mindset into culture and operations, however, can be challenging, as the approach sometimes conflicts with the more the methodical ways of IT. So how can people, process and technology be brought together for business success in this new paradigm?
People and Process
Digital transformation first must be defined. It’s a ubiquitous app or service that can be consumed from anywhere on any device at any time. Digital transformation aims to reduce the consumption barrier, which enables businesses to profit from every possible engagement while leveraging data-driven models.
A DevOps culture supporting such digital transformation is about agility, availability and scalability with continuous integration and continuous delivery of services. These strategies require new technology skillsets, which begs the age-old question: build or buy?
This question leads to more questions, which IT organizations struggle to answer. Should they “buy” new talent and build the capabilities with the existing team? Should they buy new talent and implode the existing team? Or, should they build up the existing team and enable them to bridge the gap from traditional ops into a DevOps culture?
Organizations often overlook one of the most important considerations. Putting top engineers with topnotch development skills together does not always produce a successful team. DevOps calls for collaboration and cooperation, so organizations must pay attention to soft skills. People collaborating across traditional boundaries and silos are at the center of digital transformation. Progress only happens when the organization is moving forward as a whole with the same end goal and execution in mind.
Digital transformation is a function of speed to market, and that’s where traditional IT and DevOps processes clash. IT follows the doctrine of trust but verify, a methodical process filled with rigor and discipline. On the other hand, DevOps’s modus operandi is continuous integration and continuous delivery while failing fast if needed, trying anything and everything to iterate and get as many data points as possible, as fast as possible.
Blending these cultures and processes is a matter of stepping back and realizing they are actually more similar than different. Striking a process balance between IT Ops and DevOps empowers organizations to deal with the volume, variety, and velocity of change involved in digital transformation.
Normalize the Right Technology
Making the right technology choices means finding the right signal in the noise of all the possibilities. From programming languages to the automation engine, infrastructure services (virtual machines, cloud instances, containers and microservices, serverless) and more, when creating a DevOps culture, people will gravitate to solutions that are biased toward what they know best.
The number of choices turns into an endless buffet with a tendency to overindulge, which creates bloat and starts accruing tech debt. Shouldering too much technology and being half-baked into “best-in-breed” solutions are not enough.
Disruptive technologies surpass the bounds of “good enough.” The wrong technology choice will put an organization behind in a world that’s disrupted constantly. The organization may get disrupted into nonexistence. Choose the technology that will best enable normal consumption of services while you scale services, become more agile, and expand availability. That’s what digital transformation done right accomplishes.
Customer-centric
A transformative DevOps culture keeps the customer as the focal point. Some organizations view this as “the customer is always right.” That’s one way to look at it, but sometimes the customer doesn’t know what “right” is. It’s up to the organizations to demonstrate good and great.
That means being less feature-centric and more solutions-centric—actually solving the customers’ pain points. Don’t get stuck in the “easy” to implement. Don’t assume your use case is the right use case without being certain of how customers would leverage and utilize the end results of the transformation. Remember, the goal is to reduce the barriers to consume for consumers.
One of those barriers is having a bad experience with an application or a service. Users want it now and are willing to pay if the service is excellent, always available as needed, and easy. We’ve seen this with rideshare applications, Netflix and Amazon.
Yet, the SolarWinds Digital Experience Monitoring Survey shows that 40 percent of respondents are suffering digital experience issues that negatively affect their businesses. They also report a lack of visibility into customer experience. Forty-four percent say end users are first to discover an issue, and 25 percent of those surveyed say they are the last to know.
Best Practices
Digital experience monitoring (DEM) tools help solve these problems. For example, when implemented as part of a rigor and discipline approach, the survey shows that 59 percent received fewer support tickets, 44 percent improved brand image, and 33 percent gained a better ability to retain customers. Among those who reported that DEM has driven increased revenue, 15 percent of revenue increase can be attributed to DEM.
Observability and Control
At the same time, a certain level of observability and control must be integrated into a successful DevOps strategy for digital transformation efforts. Observability is a measure of how well internal states of a system can be inferred from knowledge of its external outputs.
Depending on who you ask, observability can mean logging for one IT professional, metrics for another, or tracing for a DevOps engineer. Despite these shades of grey, the reality is that application stacks are evolving with so much velocity, variety and volume that such that a combination of all three (metrics, logs, and traces) are needed to build a viable observability protocol.
To achieve this level of observability—and, as a result, control—you must have a powerful integrated full-stack monitoring platform that enables analysis of key performance metrics, events and logs,and application code traces. This approach to monitoring as a discipline enables developers, DevOps engineers and IT professionals to better manage, optimize and troubleshoot the full stack at every layer—from the infrastructure services and application to the end-user experience—from a single dashboard.
With observability and controllability enabled, you can successfully and continually integrate and deliver the applications and services enabled by your digital transformation.
The last half of 2017 was pretty amazing. I wrote a brief post about the Inclusive Collaboration Summit I attended in London last July. Sallyann Freudenberg and Katherine Kirk started this campaign to raise awareness around neuro-diversity in tech. With respect toward the silence experiments I kept my original post very short. It was great to participate in the silence experiment in London and to help Sal and Katherine with their workshop in Orlando after I returned to the United States, but that’s not the whole story. I offered to facilitate a Wardley mapping workshop while in London in an effort to help the Inclusive Collaboration Campaign form a strategy.
Coincidentally, enough I also ended up helping Simon Wardley organize the first Map Camp in London last year. While I didn’t get a talk accepted into Lean Agile Scotland and wasn’t expected to speak at Map Camp either, I decided to make the trip back to the UK in October to attend both events as a volunteer. It’s great to get a chance to speak at conferences but I think it’s important to occasionally show up in another capacity as well. I’ve even gone so far as to buy a ticket and show up at an event as a garden variety attendee.
Both of these events were the same week in October, with Map Camp right in the middle of Lean Agile Scotland. In what one might consider a lapse of judgment, I decided to spend that week traveling between Edinburgh and London on planes, trains and automobiles. Due to a speaker cancellation at Map Camp, I was able to give a short presentation on the mapping approach I used for the workshop at the Inclusive Collaboration Summit earlier in 2017. I also helped Sal with a workshop on sensory perception in Edinburgh. Simon was running a mapping workshop on the last day of Lean Agile Scotland as well, so we had a bit of a race from England to Scotland. He left on a sleeper train right after Map Camp while I took a plane out the next morning. Simon ended up beating me there and I owe him yet another cup of tea. Still, I somehow managed to make all my flights and appointments while I was in the UK, but I didn’t have much time to take in any of the scenery while I was out of town.
At some point I accounted for this and booked a seat in the quiet coach on a train back to London from Edinburgh. I knew I’d be exhausted and would need some down time to decompile before heading back home. As the train left the station, I crossed off my last checkpoint for the trip and tried to relax a bit before flying back to the United States. As with any journey, this last checkpoint was an appropriate time to pull out the maps from the ICC workshop in London to see how we were doing and what might be next.
Diversity and inclusion are a common conversation in tech these days. They remain a priority at Map Camp as well, which is why we had a pool of diversity tickets reserved and did our best to include as many folks from different walks of life at the event as we could. Diversity and inclusion can and should be improved continuously. If you can’t take my word for it, then maybe this compilation of G.I. Joe public service announcements will convince you. Like Gung Ho says, “Don’t judge people until you give them a chance!”
Simon got right to planning the next Map Camp as soon as he left London for Edinburgh. I wanted to jump right back in to Map Camp, too, but hosting an Inclusive Collaboration conference was a stretch goal that came out of our mapping session in July, and I had devopsdays Atlanta 2018 to think about as well. The least complicated thing I could do was make Inclusive Collaboration the theme for devopsdays 2018, so that’s exactly what I did. I was geeking out about trains at the time, so of course devopsdays Atlanta will have a train theme this year as well.
The train theme is more relevant than you might think at first glance. Trains, like any technology we have, are part of a larger socio-technical system. If your system doesn’t have any people in it I’d suggest taking a step back and trying to look at the bigger picture. Of course, you can look at just the train from a purely mechanical perspective, but cutting out the societal impact of expanding travel options and connecting people, not to mention the social impact the construction and operation of railroads had on so many communities, really understates the global impact of the technology. I looked for songs about trains to listen to on the way back from Edinburgh and was amazed at not only how many there were, but how many different decades, genres and cultures were represented in the list. I found this list of about 1,000 train songs, but even the article admits the set is incomplete and the total number of songs about locomotives is likely uncountable. The fact that we’re still writing songs about trains today really goes to show the permanence of the impact railroads have had on our human race.
The DevOps community is adamant that DevOps isn’t about the tools. We’ve also heard time and time again that containers will not fix our broken culture. I had a long chat with Simon while in Edinburgh over his concerns that the community is about to make the same mistake again with serverless technology. No, serverless isn’t going to fix our broken culture, either. Similar to trains, it’s time to zoom out on the technical details so we can bring the social impact of modern computing into our frame of reference.
This isn’t news, but like the title of the post says, knowing is only half the battle. We have to actually get off of our asses and try to do something about it if we want anything to change. I’m glad to have Sallyann Freudenberg traveling from England to Atlanta this April to deliver a keynote. I’m glad Jamie Knight and his friends, Lion and Mike Southgate, will be in town from London to spend some time with us in Atlanta, too. I’m excited to hear what Atlanta’s own Kim Crayton, a technologist with a background in special education, will bring to this conversation as well.
I don’t have any silver bullet answers for the social issues we’re facing in tech today and I doubt we’ll get things sorted in Atlanta this April, but I do know we’ve got some very well-qualified individuals showing up and there will be a lot to learn over the course of two days. Cloud computing, CI/CD practices and the typical tech content seem like known, solved problems to me by comparison, but we’ll be sure to have some folks talking about these topics as well. It’s not a question of social or technical, but rather, How do we do a better job of building and operating socio-technical systems?
In addition to Map Camp 2018 this October, there may be an Inclusive Collaboration Summit in London later this year. The details on both of those events are still getting sorted out. What I do know for sure is you can join us in Atlanta this April and I sincerely hope you can make it. Early bird tickets are still available and our CFP is open until March 1. Please join us.
There is a common misconception that Agile and DevOps cannot work together, which does a disservice to everyone in the software development and delivery field. In fact, both approaches can significantly add to the Scrum pillars of transparency, inspection and adaption to better enable product teams to develop, deliver and sustain complex products. Both Agile and DevOps embody great ideas and enthusiasm that can add value to how teams work, ensuring the success of their organization in a world where business is both driven and disrupted by software.
For businesses to take full advantage of the opportunities presented by Agile and DevOps, both communities—the Agilists and DevOps professionals—need to rectify the misunderstandings between the two approaches that amplify the differences between them. They must understand why these myths permeate and learn how to overcome these barriers so Agile and DevOps can finally play nice in the sandbox.
Agile and DevOps: The Misunderstandings
Both Agile and DevOps pursue the desire to deliver value to customers in a more effective way, but from different starting points—Agile with the developer, and DevOps from operations. Their emphasis also varies, leading people to believe that they are different things: Agile emphasizes team interactions, culture and values, while DevOps emphasizes delivery pipelines and flow.
These apparent differences are further amplified by misunderstandings that do nothing but to pit the two approaches against each other:
Some Agilists think all process is bad, while DevOps is steeped in a history of stability through process.
DevOps uses tools and automation to increase speed, while some Agilists view tools as unavoidable at best and a distraction at worst.
DevOps uses holistic systems to solve problems, while Agile uses people to drive change and discover it through work.
It is disheartening that these misunderstandings exist because Agile and DevOps share more in common than most realize. For instance, both include faster delivery cycles, smaller increments or releases and use feedback to improve and remove waste and impediments. Both approaches can also influence a cultural shift in an organization that promotes small changes that are easily manageable so they lead to bigger and better business outcomes over time.
If they are more alike than dissimilar, then what exactly is holding these two methodologies back from getting along?
Politics, Organizational Silos Widen the Gap
Organizational changes, such as new hires and managerial moves, etc., attract strong personalities who get recognized for driving change and who get rewarded when those changes produce results. When their success is tied to the success of either their Agile or DevOps program, competing political goals are deemed a threat and lead to confusion:
Different tool strategies: How many times do you hear, “The Agile teams use JIRA, but the DevOps teams use Service Now”? These tools and the vendors selling them focus on a change initiative to drive purchases. This makes it even more difficult to integrate Agile and DevOps, as their data and processes are hidden within different tools.
Politics rather than value to make decisions: The saying, “It is not what you know, it is who you know,” is not just true in national capitals, but in any organization. Politics is the reality of most organizations and that means that the associated initiatives not only come with baggage of the work, but also with the politics of who is driving them.
Increased division and organizational conflict: Ironically, the DevOps and Agile movements call for unification and collaboration between different groups. However, when practiced, they can sometimes lead to even deeper gaps within traditional groups due to their different terminology and ideas.
Different Cultures
Agilists consider process to be “emergent” in response to the needs of the situation. This means teams are empowered to make decisions about how they work via a light framework (such as Scrum) and then add any other process they think are needed to achieve more value and faster feedback.
DevOps traditionally takes a different tack, with a focus on systems thinking with flow models and value streams.
Despite the culture differences, Agile and DevOps need consistent language and concepts to work together, or else communication breaks down and collaboration suffers. This leads to alienated stakeholders, increased fear and defensive behavior, poor product and service quality, lack of transparency and visibility and, more painstakingly, never getting anything done.
Scrum Breaks Down the Barriers to Agile and DevOps
Delivering value to customers through software requires elements from both Agile and DevOps coming together in a comprehensive process that is fluid and enables the teams to work in a dynamic manner. Through Scrum that empowers cross-functional, business-oriented teams to deliver software continuously, work flows seamlessly between all the right stakeholders, in real time.
However, simply having cross-functional teams in your organization isn’t enough to make sure all involved are working in a holistic way to deliver customer value. Organizations need to make the right decisions about who should be in or out of a team, since different technologies require specialist skills and infrastructure to support it.
A cross-functional team must have all the competencies needed to accomplish the work without depending on others who are not part of the team, as described in the Scrum Guide, to deliver a working product.
As technology, market and environmental complexities and their interactions have rapidly increased, Scrum’s utility in dealing with complexity is proven daily—Scrum teams are highly flexible and adaptive so that they can deliver products of the highest possible value. When misunderstandings are put aside, and people learn to communicate with each other, cross-functional teams comprised of Agilists and DevOps with all the right skills can deliver done software, faster for your organization.
A transformation can’t come from simply changing some tools or tweaking some antiquated processes to create improved collaboration. This is ultimately setting yourself up to fail. The magic comes from creating a universal change. You want to just improve your DevOps teams? Why not improve your entire organization? Be dramatic—organizational opposed to departmental. It sounds long and hard, and it will be. But ultimately, many companies must disrupt themselves to survive.
Among many other benefits, DevOps breaks down the walls between development and operation teams. But what about the other walls? We (almost) got rid of cubicles and separate offices a while ago. Now, the walls between the wider teams must also come down to become a modern organization.
Change is OK. It’s necessary, it’s powerful and it’s now. Change is about stepping out of our comfort zone and working on new and efficient ways to collaborate, all while bringing new processes and modern technologies to compliment the needs.
“Our only security is our ability to change.” — John Lilly
Take risks or die not trying. In a world of change, we can take a few calculated risks along the way. The status quo has changed and we are continually driving collaboration and communication. When the people get together ideas are born. Let’s give those ideas a shot.
“It is not necessary to change, survival is not mandatory” — Edward Deming
Agilityacross the business. Allocate guilds within the teams to continue to develop and foster this new change of culture. Focus on connecting one small team to another to start, rather than conquering the world, and then roll out to more and more teams over time.
“Success today requires the agility and drive to constantly rethink, reinvigorate, react and reinvent” – Bill Gates
Hire the right mindset and not the right tools. The winning hires are the people who join your organization not just to implement some fancy new orchestration tool and automate a few processes, but those who come in with the “multiple hats” mindset. In DevOps, job titles mean less. An implementation and automation engineer is more than just an implementation and automation engineer. The quicker you understand that, the better your DevOps hires and transformation will be. Here’s a great article from my friend Conor Delanbanque that offers more insight on scaling teams.
“If you don’t understand how to run an efficient operation, new machinery will just give you new problems of operation and maintenance. The sure way to increase productivity is to better administrate man and machine.” — Edward Deming
Constant feedback is necessary. As part of a successful transformation, feedback to the business must be considered as important as the feedback during all phases of continuous integration, testing and deployment. I have seen many successful transformations in which the business side has been actively involved to ensure feedback was effectively passed along to other teams and departments within the organization.
“I think it’s very important to have a feedback loop, where you’re constantly thinking about what you’ve done and how you could be doing it better.” – Elon Musk
Ongoing industry disruptions are forcing businesses to change their operational model; that’s the only way to avoid and survive digital bankruptcies.
In North America, Sears Canada filed for bankruptcy, while Toys R Us and Weight Watchers were on the verge of it. In Europe, particularly, in France, banks including Societe Generale and leading retailers including Carrefour are anticipating it through ambitious and innovative digital transformation programs.
The cheap access to innovative technologies enabled by cloud services providers including Amazon Web Services (AWS) has caused a proliferation of innovative tech startups and technology companies, making competition tougher for established businesses.
The innovative digital products and services they deliver are slowly but surely substituting conventional ones. As an example, Weight Watchers’ weight-loss programs did not resist innovative wearable devices, such as the Fitbit Tracker, that helped millions around the world lose weight. Trackers measure data such as the number of steps walked, heart rate, quality of sleep, steps climbed and many other personal metrics involved in people’s health.
Digital products and services are becoming customer consumption standards, accelerarated innovation life cycles are now the competitive advantage to have and DevOps in its entirety is the capability companies must equip themselves with to make substantial digital profits.
In 1985, Harvard’s Strategy professor Michael Porter unveiled to the business world in his best-selling book, “Competitive Advantage,” the concept of value chain. It ‘s been revolutionizing business practices for more than 30 years, and inspired approaches as varied as Michael Hammer’s Process Reengineering, Toyota’s Lean Production, Total Quality Management (TQM) and even Software Capability Maturity Model (CMM).
The value chain’s principle is the notion that in every single company there’s a subset of interactions, staff and skills, processes and practices and tools and infrastructure that repeatedly generates expected benefits. That subset is what’s meant by the value chain; to succeed, businesses must continuously optimize it.
Today, software development is the foundation of digital products and services; it’s becoming the fundamental industrial activity. By providing the technological, organizational, and operational framework that will enable digital profits, DevOps is replacing the value chain as Porter designed it. It’s the competitive instrument businesses need to make big gains.
Michael Porter’s Value Chain: Purpose, Foundations and Principles
The value chain, through focusing companies on their added-value activities and resources, has been revolutionizing businesses’ ability to compete for 30+ years and made several thousands business leaders successful around the globe.
Figure 1 – The Original Porter’s Value Chain
What you must know is, the underneath principle is, only a subset of the company’s activities—Primary activities and Support activities—can repeatedly generate expected benefits. That subset is what’s meant by value chain.
Let’s discuss them!
Creating Value: The Primary Activities
The objective of the primary activities is to create value. They include seven activity groups:
Inbound Logistics
Operations
Outbound Logistics
Marketing and and Sales
Services
Inbound Logistics performs supply chain functions. It’s a group of key supply chain processes and associated experts, tools and infrastructure. The mission is to make sure the company gets on time the best material to manufacture top products and services.
Operations refers to manufacturing functions. It puts together the company’s top experts, engineers and product managers to design, develop and test products and services. They take advantage of key processes and industry best practices to make the company’s products and services as innovative and profitable as possible.
Outbound Logistics performs distributions functions. It’s a subset of the company’s top distribution partner relationships management (PRM) experts mobilized across the company using top distribution processes, practices and tools to make sure the company’s productions and services are properly distributed across its partner networks and make a profit.
Marketing and Sales, as the names suggest, perform marketing and sales activities. It is a group of the company’s top marketers and salespersons mobilized through top marketing and sales processes and practices to make sure the business make profits.
Services refers to post sales activities. It’s a group of the company’s best customer relationships management (CRM) experts and associated top processes, practices and tools mobilized to make sure customers have the best experience with the company’s productions and services.
Making Value Creation Sustainable: Support Activities
The primary activities do not generate value on their own; they need levers. Their purpose is to be the value chain’s support activities, making sure the company generates sustainable value. They include four activity groups:
Firm Infrastructure
Human Resources
Technology Development
Procurement
Let’s how they enable value creation activities!
Firm Infrastructure is a group of high-level executive experts appointed to guarantee the company’s infrastructure profitability. Such executive experts include chief information officers (CIOs), chief technology officers (CTOs) and chief enterprise architects (CEAs), and they rely on top processes, practices and tools such as ITIL, IT Services Capability Maturity Model (CMM) and TOGAF to guarantee the company’s infrastructure profitability.
Human Resources is a group of the company’s selected recruiting and human resource management experts and the associated top processes, practices and tools. Its mission is to not only make sure the company has the top experts it needs, but also get from them their maximum contribution.
Technology Development relates to research and development (R & D) activities. It’s an elite of experts dedicated to monitoring markets trends and developing product ad service prototypes likely to create value for the company. They rely on extremely sophisticated product and service engineering processes, practices and infrastructure.
Procurement relates to a selected group of the company’s investment decision experts. Relying on top financial processes, practices and tools, this group’s mission is to guarantee the maximum return on the company’s investments.
Why DevOps is the New Value Chain
Contrary to what many thought leaders, experts and solution vendors pretend, DevOps is unknown; it can’t be narrowed to the complex toolchains IT vendors have been selling to CIOs. Things aren’t that simplistic.
DevOps According to Chef’s Adam Jacob’s Perspective
Adam Jacob, co-founder and CTO of Chef, wrote a memorable, article about DevOps: “The Secret of DevOps: It’s Always Been About People, Not Technology.” Relying on his experience working with the inventors of DevOps—Jesse Robins, John Allspaw, Paul Hammond—he provides an unprecedented and accurate perspective of DevOps.
He reminds this, “Becoming better at building and delivering software is the stake. It’s not an optional piece of their strategy: It’s the future of how their customers want to work with them. Software is table stakes for survival.”
He adds, “But the real answer lies in looking at what John, Paul, and Jesse were doing in 2009—not in the specific technical choices they made, but the style in which they worked, the essence of what they believed made them high functioning and successful. … Then the challenge begins: how to apply this new style to businesses trying to become better at building and delivering software.”
He proceeds, “DevOps is about bringing together all the people you need to build and run your business effectively, and empowering them to move as quickly as possible towards their goals. Tools matter. Make no mistake, trying to change the way you work without changing the mechanisms by which you do that work is a futile exercise in excruciating failure.”
Then concludes, “Fundamentally DevOps is about taking the behaviors and beliefs that draw us together as people, combining them with a deep understanding of our customers’ needs, and using that knowledge to ship better products to our customers.”
DevOps’ Fundamental Building Blocks
What’s interesting in Adam’s rhetoric is, two groups of key words and sentences help to bring out two fundamental components of DevOps.
People, customers, work, work style, beliefs, bringing together people, empowering people are part of the first group. It relates to what experienced and qualified business transformation experts refers to as operational model.
Technology and tools—and make no mistake, tools matter—don’t the change the way you work without changing the mechanic underneath are all part of the second group. It relates to what experienced and qualified business transformation experts refer to as a technology platform, known by DevOps experts as continuous delivery (CD) platform.
The following picture is a DevOps model derived from Adam’s perspective. It makes easy its understanding:
Figure 2 – DevOps as the New Enterprise Value Chain
As illustrated, DevOps can be modeled as two-components business capability including:
The operational model
The CD platform referred to in this article as DevOps automation toolchain
Let’s see how these two components are making DevOps, the powerful business capability CIOs have been expecting!
The DevOps Automation Toolchain
The automation toolchain or CD platform is what many either refers to as or confuse with DevOps. It conceptually structures software deployment in four steps:
Source
Build
Test
Deploy
Each stages uses one or more products as illustrated in the following table:
Figure 3 – Automation Toolchain Features
The Agile and Lean Operational Model
The operational model is the poor relation of DevOps; it’s totally either unknown or ignored by 98 percent of business leaders and their CIO. It’s where agility, flexibility, responsiveness and even innovation culture are implemented. It organizes the application development effort in a four-step circular life cycle:
Plan & Measure
Develop & Test
Release & Deploy
Monitor & Improve
Each stage mobilizes specific staff and skills, uses specific processes and practices, tools and infrastructure and, more importantly, specific Agile and Lean practices. The following table summarizes each stage:
Figure 4 – DevOps Agile and Lean Operational Model Features
Why DevOps is The New Enterprise Value Chain
One of the problems with the value chain as Michael Porter thought it is, it has a compartmental vision of the business with two categories of activity divided each in several processes and associated staff, skills and tools.
This perspective nurtures organizational silos and prevent cross-functional collaboration. It slows down problem-solving and decision-making and inhibits innovation culture.
The value chain as Michael Porter thought it didn’t help Sears Canada to avoid digital bankruptcy, it didn’t prevent Toys R Us and Weight Watchers get on the verge of it.
Where DevOps brings value and fills the gaps of today’s value chain is, it not only provides a business capability that’s easy to deploy but also an organizational, operational and technological framework embedding the Agile and Lean practices, as well as the automation toolchains companies need to survive their industry disruptions and make substantial digital profits.
Key Takeaways
Industry disruptions are a reality and properly transform the business is the only solution. Sears Canada, Weight Watchers and recently Toys R Us learned it the hard way.
In 2018 make DevOps transformation your priority investment decision, I’m not talking about DevOps’ automation toolchain but DevOps a business capability which includes the Agile and Lean operational model and the automation toolchain.
I first heard about Conway’s Law from reading the “DevOps Handbook” (an excellent read, by the way, if you want to dive head-first into DevOps). I was intrigued by how this idea came from slightly left field, yet seemed to have big implications for businesses on their journey to increased efficiency.
In this blog, I will share my understanding of what Conway’s Law is and how it is relevant for your business, whether you are in the middle of a DevOps transformation or just taking your first steps.
What is Conway’s Law?
In 1967, Dr Melvin Conway submitted a paper to Harvard Business Review. It stated the following observation: “Organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.”
Conway’s Law explicitly mentions the system design and communication structure. However, this idea can also be extended to include team structure, which is usually aligned either to systems structure or communication channels.
When it comes to delivering streamlined and efficient change, Conway’s Law suggests that we should align all these areas to the overall business model. This is a concept that also sits at the heart of DevOps, and is a cornerstone for improving efficiency and overcoming delivery bottlenecks.
The Importance of Alignment
Alignment between these four areas is critical for becoming truly agile and thriving in today’s online and digital world. Without it we will hampered by heavyweight handoffs, conflicting team goals, and resistance to change.
The larger the organization and the more siloed the teams, the more pronounced the constraint described by Conway’s Law is. However, organizations with more nimble, collaborative teams are free to design their systems in the most efficient way, not restricted by inefficient wasteful processes.
Dealing with Misaligned Team Structures
In IT, with its constant flow of new projects and requirements, we are always improving our systems in response to changing strategies and directions from the business. So why shouldn’t we have the same mindset when thinking about our team structures?
If our team structures aren’t evolving to meet the current needs, then we are inadvertently constraining ourselves to only deliver solutions that fit in with our existing team structures. This makes innovation difficult. Similarly, we need to mold our communication and collaboration structures into the right shape for our systems, based on business goals. This will, in turn, define how we organize our teams.
With knowledge and understanding of Conway’s Law, we can actively design systems, team structures and communication channels to avoid the pitfalls of misalignment.
Overcoming DevOps ROI Barriers
DevOps is an incredibly powerful approach to IT. But for DevOps to provide value to your organization, you need to address team silos and collaboration gaps. If you cannot align collaboration channels with your business needs, then no amount of automation or DevOps change will help your business to achieve agility.
For some businesses, team structures may need to change dramatically. For others, it may just be a case of increasing collaboration between specific teams, or creating ways to improve the efficiency of handoffs and reduce wastage in processes.
Aligning Teams to Business Goals
Overall business success isn’t siloed, and team/department success shouldn’t be either. Sharing KPIs and dashboards is one important way of ensuring that teams stay aligned to business goals by improving staff mindset and reducing distraction.
So what do fully aligned teams look like? In businesses delivering IT change, teams will be aligned to products or market sectors. This means each team will be as autonomous as possible, without cumbersome handoffs.
Often aligned teams will incorporate all speciality skills (such as information security, infrastructure and DBA) within their own team. However, it is becoming more common to solve the alignment problem by improving the efficiency of interactions between teams through self-service. For example, self-service environment provisioning enables delivery teams to function independently throughout the delivery life cycle without any environment requests or delays.
The Art of Collaboration
We are currently seeing a big shift in culture across many organizations, powered by the fast uptake of new collaboration systems. Tools such as MS Teams, Slack and HipChat remove barriers between teams and improve communication, making it easier to align collaboration channels with your business goals.
However, just using a shared collaboration tool does not guarantee streamlined business delivery. While it may be quick and easy to create a new channel or chat group, creating a culture where collaboration evolves alongside business needs is much more difficult.
The key to a collaborative culture is trust. Everyone needs to know that the entire team is working toward the same end goal, with shared responsibility for the results.
Closing Thoughts
When staff are in heavily siloed teams, they can be easily diverted away from the business goals. Staff might also resist change as they haven’t got the business view. But when teams, systems, and collaboration channels are fully aligned to the business goals, all employees will understand their impact on the business—and measure their success against the business success.
I hope this insight into DevOps from the perspective of Conway’s Law was useful, and you have some ideas to foster greater collaboration and efficiency in your business.
