— ROELBOB
Category: Blogs
-
First We Fixed Dev, But Let’s Not Forget About Ops
Since its beginning in 2008, DevOps has gone from an underground rebellion by developers to a mainstream movement with representatives in every industry and at every scale. You can see the change just by looking at how people attending DevOps events dress: the black T-shirts are still there, but these days there are lots of suits and ties as well. Another hallmark of this success is companies trying to fit DevOps into their existing conceptual frameworks—hence, the plethora of “ITIL and DevOps” papers out there.
Despite all this success, though, the DevOps movement is still young. Of course, things move fast in IT, but we aren’t talking about a technology or a technique that can be adopted rapidly and spread very widely. Committing to DevOps is a major cultural shift for organizations, and that takes time. Some organizations have been doing DevOps for five years and have fully re-engineered their processes around this new approach, but most are still only at the beginning of their DevOps journey. As William Gibson said, “The future is here; it’s just not evenly distributed.”
First Things First: Speed Up Releases
The early phase of DevOps adoption focuses on activities around releasing code to production, because that was the bottleneck that the DevOps movement was originally created to break. While it still may be too soon to declare victory, it’s safe to say the improvement in the release process is enormous. Today, any new project that does not propose to implement DevOps principles would have to have a very good reason and explanation.
Notice that qualification about new projects. It’s easy to adopt DevOps methodologies and expectations in a greenfield situation; however, it is much harder in a brownfield project, where previous cultural norms have been expressed in tooling and processes that are hard to change in-flight. The good thing is that over time, the old way of doing things will die off naturally, as code bases get re-engineered, ported and replaced. Eventually, every project will be a DevOps project.
Are We Running So Fast That Others Can’t Keep Up?
DevOps, however, is not just about release and deployment. It also covers operation of what has been deployed. It’s right there in the name, after all! Coming from an operations background, though, there can be a feeling that DevOps is not for “people like me.” Some simplistic interpretations of DevOps focus on the sometimes adversarial relationship between Dev and Ops, forgetting that good Ops people are just as incentivized as Devs to get Devs more involved in Ops.
One of the difficulties that arise when engineers try to “do the DevOps” without involving Ops is that Ops are not ready to join in. When Dev are doing non-agile releases on an 18-month cycle, Ops have a certain set of assumptions about what a release represents for them. Doing these sorts of monolithic releases is a major pain for Ops for a while, but there will be plenty of time to get ready for it, and other required activities can be scheduled around that release.
One of those important dependencies is preparing the monitoring and management of the code that is being deployed. What needs to be instrumented, where do those feeds need to go, what needs to be in place to make sense of those feeds when they arrive, who needs to jump when one of the feeds goes red, and so on. There is a lot going on in the NOC that Devs are not always aware of.
If Devs suddenly show up with a more Agile cadence of a release every couple of weeks, Ops are simply not going to be able to adapt their existing tools and process on the fly to that new cadence. When you have months of lead time, it’s feasible to write and modify extensive sets of rules and filters to make sense of monitoring data, filter out the cruft, and route the remainder correctly. When your lead time on a release is measured in days or hours—not so much.
Devs should remember that “infrastructure as code” also includes debugging that code. Adopting release automation, containerization or whatever will be the next cool tool will not solve the problem on its own. Bridget Kromhout has a wonderful articulation of this mechanism:
Containers will not fix your broken culture. Microservices won’t prevent your two-pizza teams from needing to have conversations with one another over that pizza. No amount of industrial-strength job scheduling makes your organization immune to Conway’s Law, which states that “organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.
Leaving aside the communications barriers that all too often still exist between Dev and Ops, one of the constraints that Ops teams have to deal with is that Ops is segmented into many different teams, each of which deals only with one area or one technology. These days, though, problems tend to have symptoms that cross these domain boundaries, and so operations teams have to communicate with each other.
The historical ways of doing this are not very efficient, relying on ticket ping-pong, escalations, war-rooms and good old blame-storming. What do you think happens if you suddenly accelerate a system like that? The answer is that it just torques itself into scrap.
Completing the DevOps Culture Shift
The good news is that others have been where you are now, and there are both tools and techniques that can help you operate what you have developed. In common with the rest of the DevOps movement, it’s as much a cultural shift as a technological one. Ops teams need to start thinking of operations in a new way.
One aspect of that change is the ongoing shift to immutable architecture, also known as puppies vs. cattle. Another is to enable efficient communication across support domains. Today, like in the parable of the blind men and the elephant, support teams often get hold of different bits of the problem, misidentifying them because they only have partial information, and therefore cannot truly resolve the problem. Nobody really has a complete view of the whole elephant, except perhaps for some savant who can be reached only after many escalations—and really has better things he or she should be doing.
If Ops teams can work side by side with Dev, on the other hand, there is a virtuous cycle that becomes possible, with efficient operations removing even more friction from the development process. Communications are the key, whether that means machine-to-machine feeds, human-to-human conversation or machine-to-human interaction. Organizations that get this right reap enormous rewards in both Dev and Ops.
Dominic Wellington serves as Moogsoft’s Chief Evangelist
-
Funding Failure by Intent
Why is it that when we purchase a lotto ticket, every part of our brain tells us we won’t win, yet some portion of it is able to inspire hope where there shouldn’t be any? Could it be the same part of our brain that inspires hope in the face of nearly certain failure has been nurtured by the software development projects we have been exposed to over the years?
It would appear that the ability to prognosticate has been the foundation of nearly every waterfall project that had a timeline of three years or longer. It is this myth of “certainty” that agile methods are designed to eliminate. In our new approach, we simply break down and digest the scope of work until it is small enough for our risk tolerance, and easy enough to implement in our abbreviated time horizons. We try, we evaluate and we try again.
But no matter how low we believe the risk to be (and generally the reward we expect from it), risk does remain. And now instead of a three-year project plan well-equipped with milestones, corrective actions and projected costs, agile gives us only the current sprint, based loosely on our historical velocity and a literal “addiction” to continue this process going forward without any real regard for a predictable end-point or total project cost. Continuous delivery infers continuous innovation (emphasis on the word “continuous”). And if done right, why would a business ever choose to end innovation that works?
Imagine the analogy of Mozart sitting down to compose a quick advertising ditty that your company could play as background music on its website. Mozart begins with a few bars, and notices a theme that is worthy of far more exploration … so he does. He continues exploring this until an epic symphony emerges. This was not the plan and does not meet the original requirements. This could be considered scope-creep in the waterfall world, and thus a failure, as no one stares blankly at a website for the 30 minutes it takes to hear an entire Mozart epic … or do they?
Every audience grasps the concept of background music we pay little attention to but subconsciously understand it improves our experience. But nearly every audience also realizes the beauty of a Mozart epic, and would gladly buy CDs, MP3s or concert tickets to hear one of those over and over again, relishing every note. The question this analogy begs is simply this: Would Mozart’s employer—more specifically, his development manager—have been free to fund this “failure” or not? What happens when a project changes direction significantly away from the first vision we had of it? Does DevOps enable us to pursue this radical change in direction, or are our legacy financial systems and methods binding us to a waterfall delivery and reporting structure—and, in so doing, limiting what we will ever be able to accomplish through radical innovation?
The beauty of agile construction and DevOps delivery is that we may deliver “unexpected” innovation. Nearly every time a user “sees” what we have delivered to meet their need, they immediately think of “x” more things we could do to make it even better. This may result in taking a project in a radically different direction within the confines of a single financial year. To further enable our ability to do this, our legacy financial funding and reporting systems must evolve to match this demand. Our new risk profile must include the ability to fund failure on purpose, understanding that failure itself is part of the maturation process of innovation, not the death-knell of the company.
A more honest approach to funding agile/DevOps efforts would be to determine the annual investment the organization wants to make in certain capabilities, and innovations it expects to see in each thematic area. Overarching themes are funded on an annualized basis; however, great autonomy is granted and expected within the themes to ensure delivery is achieved. Pushing financial responsibility down in the organization to as low a level as possible has the effect of pairing the team leaders with the costs of what they do. Spending too much time on delivery of any given feature or function may cause the team to re-evaluate the approach and propose a better solution. This is more unlikely when budget responsibility is managed several levels above the team doing the work—in that instance, the budget resembles more of an infinite resource they need not be concerned with, until it is too late.
When proponents of DevOps claim it is changing the culture of a company, they are not limiting this view only to IT. Tertiary services in a company, such as the supporting financial budgeting systems, hiring systems and even contract management systems, may require significant change to match the pace of delivery and innovation DevOps delivers. As an enterprise looks to implement DevOps, it should look broadly at what else may be required to fully enable the pace of innovation it is capable of.
To continue the conversation, feel free to contact me.
-
Webinar: How major enterprises are bringing their database Continuous Delivery up-to-date
Leading organizations are increasingly adopting continuous delivery, yet they often find it challenging to support database release automation, while ensuring database updates are delivered rapidly and securely.
These challenges have led a number of fortune 500 companies to database downtime caused by out-of-process updates, code overrides and other database glitches.
Attend this webinar and discover how some of the leading organizations addressed these challenges through understanding:
- The different requirements of application-code & database-code from a source control point-of-view
- Why their database was often left behind
- How they were able to use baseline aware analysis to overcome database updates challenges
- How they automated their database Continuous Delivery, from development to build to deploy
Recorded 3/22/16
Recording
Slides
[seoslides embed_id=”21adc07df1b9″ script_src=”https://devopsy.kinsta.cloud/embed-script/14078/14134/” overview_src=”https://devopsy.kinsta.cloud/slides/14078/” site_src=”https://devopsy.kinsta.cloud” site_title=”devopsy.kinsta.cloud” title=” ” /] -
DevOps Days returns to London
On April 19, after a break of nearly two and a half years, the DevOps Days conference is returning to London. Expect great things: All DevOps Days events are extremely interactive, making use of the Open Space format, and this year’s event includes a world-class set of presenters ready to share their experiences and guidance.
The emphasis of the conference is on DevOps in the Enterprise, with specific reference to the financial services industry. That said, the event is open to any delegate, and participation is encouraged for enterprises that may be finding DevOps to be a struggle.
The conference kicks off with a talk from Joanne Molesky a global expert in improving IT at large enterprises and co-author of, “Lean Enterprise: How High Performance Organizations Innovated at Scale.” Molesky’s talk, “We Are Great at DevOps but the Enterprise is Failing,” will include legacy management practices in finance, IT governance and procurement, and how they inhibit our ability to extend the benefits. She will offer up many real-world examples challenge us to apply Lean thinking and principles to break down these management barricades and deliver even more value to our customers.
Kris Saxton, organizer of the London Infracoders meetup group, is slated to present a discussion with the provocative title, “Bimodal IT and other Snake Oil.” Saxton will draw on experience from five projects (four within large retail or investment banks and one with a large re-insurer) to challenge the idea that reforming legacy estates while continuing to innovate shouldn’t be attempted. Saxton hopes to convince you that this thinking is a flawed oversimplification which, when adopted, results in lost opportunities, delays crucial decisions and ultimately cripples innovation.
Expert in everything from platform-as-a-service to people and culture to Perl—with the proud claim that modern Macs still ship with some of his Perl code, Casey West plans to discuss platform applications. He will play close attention to the operations, and, in particular, the operability end of DevOps. In fact, he won’t just be talking about it: Instead using 100 percent open source software, he will be performing a demo on stage in what he promises to be an “interactive live explosion” of a modern platform that will highlight quite how accessible in 2016 they are now.
Day two kicks off with Gene Kim, award-winning CTO, researcher and author of, “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.” With all his immense dedication to DevOps operations and IT security, when Gene offers to share, “My Top Five Lessons Learned: DevOps In Large, Complex Organizations,” you know it’s going to be extremely valuable. In Gene’s own words: “I can’t wait to share with the DevOpsDays community what I’ve learned, share the problem road map that’s been verbalized, and some of the things we’ve been working on to help people overcome those problems!”
Gareth Rushgrove is perhaps most well-known for writing the Devops Weekly newsletter, which is now more than 5 years old, but his talents don’t stop there. He is also a senior software engineer at Puppet Labs; a former infrastructure, operations and information security specialist for the UK government; and an established conference speaker. In his talk, “Rate of Change, (Un)opinionated Platforms and Devops Coevolution,” he will share insights on how the adoption of practices such as DevOps, microservices and tools such as platform-as-a-service are interconnected. He will explore the power of co-evolution, which makes cause and effect bi-directional, making it possible to leverage what you do and how you do it to improve your organization as a whole. Be prepared!
Philippe Guenet is an expert in DevOps, in particular transforming banks toward continuous delivery. With his talk, “From Lean in Automative to Lean IT,” Guenet plans to draw heavily from what other industries learned in the 20th century dating back to Henry Ford and, of course, Toyota. He will revisit fundamental Lean concepts and principles applicable to IT and explore how they can be applied to improve organizations. The topics will include Total Quality Management, organization optimization, simplification and improved time-to-market.
This is of course all just a taster. Slated also are talks from Thiago Almeida (Microsoft), Michael Coté (Pivotal), Jeromy Carriere (Google), Clair Agutter, Benjamin Wooton, Oliver Wood and John Clapham. In addition a panel is scheduled that will include at least one chief operations and technology officer. Last but not least, on top of all of the great interaction in breaks and Open Spaces and great presenters, we promise a great social event the evening of April 19, including a few surprises.
The last few tickets are available here, so get one before they are gone!
See you there!
About the Author/Mark Rendell
Mark is a DevOps and continuous delivery specialist at Accenture with 11+ years of experience in development, configuration management, infrastructure as code, continuous delivery, environment management, build, deployment and testing automation as well as tooling. Currently, he runs a team specializing in implementing continuous delivery for large-scale, transformation deliveries across multiple industries. He is also a proud contributor to the DevOps Platform.Blog: https://markosrendell.wordpress.com/
Twitter: @markosrendell
-
Continuous Improvement: The Key DevOps Distinguisher
DevOps has become a well-accepted and growing component for delivery success, shown in the State of DevOps Report among other respected publications. But even as it becomes more ubiquitous, it’s naive to think every organization will cease having delivery concerns. Implementations will differ widely, and adherence is essentially self-reported. So what is it, then, that separates those organizations that struggle with effective software delivery for years from those that consistently shoot further past their competitors?
The fuzzier nature of the culture and relationships aspects of DevOps obstruct the potential for an ISO- or CMMI-style certification—which is likely a good thing, though it also opens the door for DevOps to be simply a re-branding exercise without any teeth. As a guideline, the CAMS model is still an excellent overview of key DevOps principles. It lays the groundwork for measurement, feedback and sharing that set the stage for improvement, yet stops before making continuous improvement a top-level concern.
The rate of continuous improvement is the metaphorical equivalent of compounding interest. It takes just a simple illustration to show how starting further behind, but focusing on continuous improvement, can take you past those that have more initial resources.
Prioritizing DevOps for Continuous Improvement
DevOps with a focus on Kaizen or continuous improvement can catapult a company’s ability to effectively deliver software. Unfortunately, it falls squarely in the Important/Not Urgent quadrant in Stephen Covey’s model, where good ideas and good intentions are continuously frozen. An inability to keep momentum and traction in this state affects many companies.
To reach our goal of continuous improvement in the DevOps implementation in a company, first we must examine how to successfully drive internal initiatives in general. As a default behavior, companies commonly relegate themselves to just making it through quarter to quarter, delivering mostly incremental customer features. More ambitious initiatives draw lots of excited talk in executive suites, yet often stall and get delayed in actual implementation. Why is that an all-too-common occurrence?
There are three main hurdles:
- Inconsistency of shared goals and priorities across all levels of the organization: There’s often a gap somewhere around either middle management or line management where initiatives can tend to slip. It doesn’t hurt to over-communicate and ensure it goes across all levels, especially when important changes are occurring.
- Lack of individual accountability and empowerment: If communication is adequate, do individuals know not just the goals but also what they can and should be doing to reach them? This aspect is in principle tackled by official systems such as management by objectives (MBO) or objectives and key results (OKR). You can find success or failure in many systems, so rather than debating the merits of various people management systems, make sure to focus on the key goals and actions chosen.
- Do people actually have the budget of time and potentially funding to take action? Without time for action, there is only a nebulous intention—and perhaps a slight feeling of guilt at not having made progress.
Simply identifying the above obstacles and addressing their existence can be eye-opening and likely will cause common-sense measures to emerge to align the organization. After the initial quick wins, there certainly will be more difficult steps to follow that are specific to each organization. The end result is certainly worthwhile.
Unblocking the organizational obstacles to DevOps implementation, not as a one-time project but an evolving program of continuous improvement, blazes the path for long-term success.
As a further thought, imagine you also accelerate the rate of continuous improvement so it’s not a constant. An amazing example of the power of acceleration is the notion of a solar sail: Accelerating its velocity just 1 millimeter per second seems imperceptible, yet after one day, it’s moving 195 mph, and after 12 days, it’s up to 2,300 mph. “Astronomical” is an apt description of the delivery speed of some famous tech startups. That’s the power of continuous improvement!
-
3 Reasons Why Automation is Critical
We’re hearing a lot about the “outcome economy”—how businesses must be results-oriented to meet evolving customer demand and gain greater market share. Today’s marketplace is connected, always on and increasingly competitive. Companies are adopting what we call an “as-a-service” approach to achieve better outcomes fast, consuming and leveraging leading-edge technologies such as cloud and automation.
As IT systems grow exponentially, and cloud solutions proliferate, non-automated, manual systems increasingly are becoming a major business liability. Today’s systems are simply becoming too big and complex to run completely manually, and working without automation is largely unsustainable for many enterprises across all industries.
Automation involves a set of tools, processes and insights that allows IT environments to self-modify and adjust, and some enterprises have started using intelligent automation to drive a new, more productive relationship between people and machines.
For example, IT automation is often used to auto-scale and load-balance large fleets of servers, manage global content distribution based on geographic demand, enable self-healing of IT systems and manage security mostly with limited ongoing manual intervention.
Moreover, automation enables the ability to adapt and improve the service experience without manual intervention. However, while these tools offer new strengths and capabilities, they are meant to complement and enhance human skills.
Effective automation depends on adequate insights collected from all the systems relevant to the service experience and business outcome you’re trying to augment. Insights from data is necessary to create opens paths to automated predictions and ultimately using machine learning, or artificial intelligence, as part of a full scope the as-a-service construct.
Specific insights known as telemetry allows signals to be harvested and interpreted so automation can better adjust production systems to maintain a healthy business. The insight gathered from such analytics allows automation to validate and compose modification rules. For example, sensors that detect a supply chain issue could automatically reroute or fine-tune related functions, such as dispatch or logistics, to solve or generate a workaround for the issue. The business flow can adapt and realign automatically with the ultimate goal of improving the customer experience.
Automation Creates High Resiliency
Two common business outcomes that depend on efficient automation are highly resilient systems and experimentation platforms.
Highly resilient systems include automation that can detect, avoid, heal and remediate any deviations from normal, healthy business function. To detect deviations, automation capabilities need to understand what the “steady state” of the system is and what constitutes the “health” of the system under varying conditions. For each detected deviation from an established steady state, a specific automation is triggered that attempts to return the system back to the steady state.
The best way to determine if resiliency automation works effectively is through a process known as “fault injection.” Highly resilient systems run under constant fire drills in which operations insert faults into the system while developers continuously build responding resiliency automation.
Automation also can provide a higher degree of experimentation and increase agility, two key attributes of as-a-service economy. Automatically provisioning a component such as a virtual machine, for example, is only a piece of the puzzle since automation is most valuable when it contributes to improving a customer experience or delivering a business outcome.
A platform that’s constantly testing, experimenting and developing allows companies to try new ideas in production quickly without fear of failure or outage. When confidence in system resiliency is high, it allows businesses to test new things directly in production (A/B testing). If an experiment fails, there is no harm done as automation returns the system to steady state. If an experiment succeeds, it is quickly absorbed into the production itself.
A fast, efficient experimentation platform enables businesses to react faster to failures and successes—and pivot accordingly without excess wasted resources. For example, a retail company might change a shopping basket feature for 1 percent of its customers. With constant measurement and instrumentation, the company can automatically derive insights, determine if the change is effective and create a chain of automated reactions. If, say, the demand spikes for a new offering based on a limited customer pilot, the system can reset stocking levels ahead of geographic or further customer segment rollout. This ability increases a company’s agility and adaptability, improving the customer experience and delivering on the most important factors determining success in today’s as-a-service business environment.
Moving forward
In this fast-paced marketplace, it is becoming clear that automation is the next critical step in IT maturity—building trust into IT systems and enabling innovation of new capabilities.
However exciting, impactful and resourceful automation is, it is important to note that automation will not replace people. Machines and artificial intelligence will be the newest recruits to the workforce, bringing new skills to help people do new jobs, reinventing possibilities across the business spectrum. The companies that succeed with automation take a “people-first” approach, adapting their enterprise’s organization, culture, skill and experience to use automated technology.
Although automation becomes critical to stability when systems grow to global scale, it also provides great benefits to smaller systems. Every modern enterprise should start thinking about an automation strategy. Companies that aren’t experimenting with automation are going to be outpaced quickly by the competitors that are quick to analyzing data, yielding insights that make their processes more agile and unchains their systems and infrastructure from undue age-old complexities.
About the Authors
Marshall Wells, Managing Director, Hybrid CloudMarshall Wells is a managing director at Accenture, focusing on Accenture’s Hybrid Cloud platform. He holds an MBA in Finance from theUniversity of California, Berkeley and a BS in Chemical Engineering from Colorado School of Mines.
Miha Kralj, Managing Director, IT Strategy and Cloud Computing Transformation Miha Kralj is managing director at Accenture, where he is responsible for Accenture’s IT Strategy and Cloud Computing Transformation platforms. In this role, Kralj helps to define core cloud strategies that help clients transform traditional IT, creates and delivers IT transformation programs, and identifies the right cloud disposition for various workloads.
Prior to assuming his current position in 2015, Kralj served as principal architect at Amazon Web Services, leading the domain of IT Transformation and modern IT delivery. Before that, he was senior technology director at Microsoft, where he designed Enterprise Architecture offerings through assessments, planning and value realization.
-
The DevOps Equation: Agility + Empathy = Quality
I had the chance to catch up with Jeff Sussna (@JeffSussna), founder and principal of Ingineering.IT, ahead of his keynote address on continuous design, scheduled for DevOps Days Atlanta, April 26-27. He discussed the importance of designing for service, responding to the unexpected, and the importance of building empathy across teams.
Derek Weeks (@weekstweets): Today I’m really happy that we have Jeff Sussna joining us for the latest in this series. Jeff, why don’t you introduce yourself?
Sussna: Sure. Thanks for having me. I’m an independent consultant. I’ve been around, since rocks were young or the 80s, whichever is older. My particular background is that I have built systems and led organizations across the entire development, QA and operations spectrum. So, I’m a little unique in that respect.
My current focus is on helping IT organizations and digital businesses really bring together their ops and design thinking so they can not only go faster and not only be more resilient, but provide more customer value at the same time.
Weeks: Let’s talk about that a little. How is continuous design—a subject you have written a lot about—coming into play? How can it help us be more resilient.
Sussna: Just to explain a little about what we mean by continuous design. It’s basically bringing together the entire organization and a unifying design in operations. It’s taking what we are doing in DevOps and expanding it to include product, design and support. It lets the whole organization really have an agile conversation with a market and with its customers.
The issue that we face is that we live in a service world now. The point is no longer delivering products like cars coming off an assembly line. It’s much more about responding to our customers and having relationships with them. This is happening in a world that is more and more complex while less and less is in your control. We can’t figure everything out in advance. We have to have the ability to respond to the unexpected. We have to be able to do that at every level.
“This is happening in a world that is more and more complex, while less and less is in your control.”
I will give you a couple of examples of things that are perfect illustrations of this:
I recently wished a friend of mine happy birthday. The reason I did that was that because there was a notice on my calendar that said it was her birthday. It was wrong. It wasn’t her birthday. I had no idea where that notice came from. Did it come from Skype? Did it come from LinkedIn? Did it come from email? I had no idea.
We face that in IT, too. Systems are becoming more complex. We develop microservices. We use public and private clouds at multiple levels. We’re no longer in a situation where we can control things. We have to be able to respond and respond at all levels.
When we make design and operations unified and continuous, we are able to say, “Where did we think we were going? Where would we need to go? and what do we do next?”
What I talk about in my book, in my workshops, and all my work is that this needs to happen at every level. It’s not just a matter of designing for failure inside the data center or inside the infrastructure. We also have to do it at other levels.
Here is another example:
I was trying to get paid recently by a large enterprise client in Europe. They were trying to wire money internationally and my bank wasn’t really set up for it and they were having problems. They just kind of gave up. I kept getting these messages saying, “Sorry, we tried paying you and it didn’t work.” That was the end of it.
I started to get frustrated and my client within the enterprise started to get frustrated to the point where they sent a scathing email to our AP department which they CC’d me on, saying, “Why can’t we take the lead to get his guy paid?”
That’s an approach that is based on resilience in the sense of assuming that something can go wrong and figuring out what he can do to fix it when it can go wrong. As opposed to the traditional approach where we’ve been trained to push certain buttons and take certain actions. If those actions don’t work the way they are supposed to, well, I guess that is someone else’s problem.
https://www.youtube.com/watch?v=ISSMHmwIMtA
Weeks: This brings to light some recent conversations I was having with Paula Thrasher at CSRA. She was talking about developing systems and how to make them more resilient. She is using pictures as part of her conversations with people to help visual their work. What work steps are there? What work is flowing? What is communicating with what?
Are you taking similar approaches with continuous design in how you work with clients to pursue it?
Sussna: Well, I try to introduce them to the principles and I also try to introduce them to a set of practices which actually comes out of Mark Burgess‘s work on promise theory. It is really a conceptual model for how we think about success and failure and how do we think about service. For example, Amazon helps you get everything you need in your life from shaving cream to batteries to snow tires to books as quickly and easily as possible. In order to keep that promise they make a whole variety of other promises. Some of which filter all the way down to a promise to be able to go over and make a change to production as quickly as possible.
Once you take that mindset. The interesting thing about a promise and the reason we use that word is because it is very intuitive. We all know what a promise is. We all make promises to each other every day. On the other hand, when we think about it in terms of service, IT, or whatever it may seem a little odd. The thing about promises is that on the one hand we are making commitments to do things and on the other hand promises sometimes get broken. That is actually good because it forces us to think. If I promised to pay my vendor. What do i do if I’m not succeeding in paying my vendor?
The thing about promises is that on the one hand we are making commitments to do things and on the other hand promises sometimes get broken. That is actually good because it forces us to think.
It’s sort of flipping our mind. Again, we are doing this in DevOps world where you start thinking about things like mean time to repair (MTTR) instead of mean time between failure (MTBF). We shift from how do we create a situation to how do we create a situation from where nothing will ever break—which becomes less and less possible. When then need to shift to a situation where we can respond more and more quickly.
We go to executives and say, “We don’t want to worry about how often stuff breaks anymore,” and they freak out. We have to teach them and we have to teach ourselves that we can’t prevent things from breaking. Instead, let’s be able to fix them so quickly that maybe no one notices. My focus is on really helping people shift their mindset from thinking about ourselves to thinking about who we are helping. And from thinking about what we are doing to thinking about how well it succeeds and how well it overcomes problems.
We have to teach them and we have to teach ourselves that we can’t prevent things from breaking. Instead let’s be able to fix them so quickly that maybe no one notices.
Weeks: You talked about delivering on promises as quickly as possible. How does continuous design account for doing things as quickly as possible but making sure the quality is there at the same time?
Sussna: In two ways—one way is that when you do things in small increments you are able to find problems more quickly. This is classic; it comes out of lean. It comes out of integration. It’s pretty basic stuff for our community now that it’s much easier to find and fix a bug five minutes after you change a code than five weeks after you change the code.
When you put things like Linux, agile, DevOps, and design all together, they all focus on incremental feedback loops where you discover the gap between what you think you’re accomplishing and what you’re actually accomplishing very quickly. Secondly, when you take this all to the way to the marketplace and you make it part of the relationship with your customer, you can reduce a tremendous amount of waste.
The way to think about continuous design is: it’s all about steering. Think about a boat and you’re trying to sail across a bay. You can’t just point the boat at the other side and say go because the winds are changing and you’re going to be able to respond to things that you can’t predict. If you sail for 20 minutes then discover you had pointed in the wrong direction, you’re going to be way off-course. You’re then going to generate a tremendous amount of energy getting back on course.
What continuous design says is just an extension of continuous integration and continuous delivery taken all the way though the business. When we steer continuously we steer efficiently.
What continuous design is really just an extension of continuous integration and continuous delivery taken all the way though the business. When we steer continuously we steer efficiently.
Weeks: I know you’re going to be keynoting DevOps Days Atlanta coming up in April. Chris Corriere, one of the guys on the organizing committee, tells me that you’re leading a full day workshop on continuous design before that. Maybe you could tell some of the people that are attending DevOps Days Atlanta why they should be there a day earlier for your workshop and what things you’re going to address during that time.
Sussna: To some degree what I am going to address is everything we have been talking about for the last 10 minutes or so—relative to DevOps. I think the key thing is, and we need to think about it. We need to think about more than just going fast. One of the big misunderstandings that I see with DevOps is this notion that it is primarily about delivering application changes to production faster. That’s only half of it. We really need to think about from the perspective of providing service.
One of the big misunderstandings that I see with DevOps is this notion that it is primarily about delivering application changes to production faster. That’s only half of it. We really need to think about from the perspective of providing service.
You know what we are talking about is essentially developers and operations people providing better service to each other. As a result, they can deliver better service to the business.
Where we talk about microservices. You know that word service is in there for a reason. If all you do is you kind of shatter your organization and shatter your architecture into a bunch of little pieces, you still have to figure out how those pieces come back together into something coherent. The answer to that is taking a really deep service-oriented approach. That is very much what my workshop is about. What does that word even mean and how do we do it?
Weeks: You mentioned thinking about interactions as delivering better service to one another within the organization. How does empathy come into your thinking around DevOps and Continuous Design?
Sussna: I’d like to say agility plus empathy equals quality. What I mean by that, in particularly empathy, I think it’s important to take a minute and clarify. When we talk about empathy we are not talking about wallowing in someone else’s pain. We’re not even necessarily talking about things on an emotion or feeling level. It’s very simply the ability to see things as if from an other’s perspective. I will give you a prime example of where that comes into play and how it’s important and how it helps.
Agility + Empathy = Quality
When I work with clients, I tend to look at their process using their organization from the product beginning to the support end. Often times I see support getting the short end of the stick. I see places where they get phone calls from customers with features that they don’t even know exist yet. They sometimes get talked down to by development or product or even operations. They say things like, “Well you’re just supposed to support the stuff. You’re the young people. You’re the less experienced people.”
When we have the ability to see things from their perspective, we can think about how support is actually experiencing our delivery process. If we don’t think about them when we go through the process of delivering new features, they will be lost. Them being lost will show when customers call for help. That will significantly affect customer satisfaction and then even beyond.
The reason we need to empathize with support is so we can empathize with the customer. Often times when we build things and we don’t think about the notion that our customers might have trouble with it. On the one hand we have the illusion that if we design it and we build it really well, they won’t have trouble. We can’t perfectly predict who they are, what they need or what they will encounter. There’s an inevitability to the idea that they are going to need some amount of help.
Weeks: I think that is probably the best description of empathy within a DevOps environment that I have heard anyone talk about. Jeff it has been a great conversation. Thank you very much. I really appreciate it and we look forward to seeing you at DevOps Days Atlanta coming up.
Sussna: Thanks. It has been great talking to you and I am really looking forward to the conference. It will be fun.
If you want to learn more about continuous design from Jeff, be sure to join us at DevOps Days Atlanta. I will also be at the event giving an Ignite Talk, titled, “Real Heroes Draw Pictures,” focusing on lessons learned from the 31 DevOps and continuous delivery reference architectures I assembled.
