Author: Helen Beal

  • Assessment of DevOps Capabilities: 5 Dimensions Explained

    Assessment of DevOps Capabilities: 5 Dimensions Explained

    In order for organizations and teams to move toward DevOps maturity, they need to understand their DevOps capabilities. An Assessment of DevOps Capabilities determines a baseline DevOps state, measures and accelerates continuous improvement.

    When we designed our Assessment of DevOps Capabilities (ADOC), the ambassador cohort brought a ton of ideas to the table around the high-level topics based on their experience performing this work for years. We talked about how important it was to cover the three aspects we have classically referred to in our industry; people over process over tools and many referred to the CALMS (culture, automation, lean, measurement and sharing) model widely known in the DevOps world and originally coined by Willis, Edwards and Humble.

    Our chief of research, Eveline Oehrlich, who led the 40-strong ambassador cohort who crowdsourced the model along with myself, had been conducting research for our Upskilling Report for several years. We looked at the models she’d been using there to drive comprehensive coverage of the different dimensions of DevOps. They aligned with the other models we were considering, resonated with the team and we all agreed that they gave us the best chance of ensuring the organizations that would use the assessment considered all the elements of these complex and sophisticated ways of working we call DevOps.

    The 5 Dimensions of the Assessment of DevOps Capabilities (ADOC)

    1. Human aspects
    2. Process and frameworks
    3. Functional composition
    4. Intelligent automation
    5. Technology ecosystem

    Our next job was to define the topics in each dimension and then to define Likert statements for each of the topics. These statements allow for data to be collected that provides teams with insights into where their capabilities are strong and should be shared, and where we recommend they experiment with improvements. Let’s look at each dimension in more detail:

    1. Human Aspects Dimension

    At DevOps Institute, we’re all about the human aspects. We know that culture is the biggest challenge for most organizations on a DevOps journey. Helping individuals and teams to understand and change behavior is a core purpose of ours.

    In this dimension, we address psychological safety and a culture of trust. We explore how leadership behaves, using transformational leadership as a model. We ask questions that explore how the team feels the organization is performing in terms of alignment, accountability, autonomy, empowerment and if and how they feel supported. We look at dynamic learning and constructive collaboration.

    Some topics focus on The Three Ways; we consider flow through the lens of customer focus and value stream thinking. We ask about how feedback is generated, received and reviewed in retrospectives. We explore how innovation and experimentation happens. There are some statements that help the teams learn about organizational design and team topologies. And we also include happiness and joy at work, and diversity and inclusion topics.

    2. Process and Frameworks Dimension

    Many years ago, I borrowed our esteemed leader Jayne’s phrase, “DevOps is the happy, polygamous marriage of ITSM, lean and agile.” Naturally, these frameworks are in this dimension – specifically, how they connect with DevOps activities.

    We also look at how the teams are experiencing scaling their activities and whether it’s working for them. We want teams to understand how to achieve high performance in continuous compliance, so we help them explore how they incorporate governance, risk and compliance (GRC) in all that they do.

    We take another look at value stream ways of working from a process perspective, and how well teams are transitioning from project to product. We look at a couple of frameworks very closely related to DevOps; site reliability engineering (SRE) and DevSecOps.

    We explore how the teams are leveraging systems and design thinking. We gather data on which elements of immersive learning, Holacracy and Humanocracy are helping them achieve their DevOps goals.

    3. Functional Composition Dimension

    In this dimension, we are looking at all the steps from ideation to value realization.

    We start with looking at how the teams collaborate and then where and how ideas are conceived, nurtured and refined through portfolio, backlog and product management and ownership.

    We look at change, whether the teams are using lower-level change advisory/approval board (CAB) patterns, and how far they have progressed towards peer-review. We explore with them how loosely coupled their architecture has transitioned, and how they build, develop and integrate. We ask them to rate their capabilities around test and validation, and deploy and release.

    Other capabilities we explore include operations and support, security, infosec and cybersecurity – helping them understand the patterns that create the safest environments to reduce business risk. We look at data lakes and how they manage data generally; its statefulness has always caused problems with DevOps at speed; for remediation and test data in particular. And we help them understand how to build in reliability and recovery patterns to counteract the risks associated with high-velocity working.

    4. Intelligent Automation Dimension

    This dimension is all about the DevOps toolchain; from artifact management and source control through CI/CD and into live.

    We also look at how tools are used to create a unified backlog, and how service and support happens post-release. We explore environment management and how well the teams are achieving continuous compliance and what this means for customer delight.

    We consider some of the key tool categories that help obtain feedback for improving customer experience; observability and monitoring and tie it all together by exploring how the team is using value stream management to gain visibility, traceability and insights into the end-to-end flow through the DevOps toolchain.

    5. Technology Ecosystem

    Our final dimension is designed to help teams understand how the infrastructure that surrounds their product, service or value stream influences their ability to work better, faster, safer and happier. This means understanding their capabilities for cloud, elasticity, serverless, containers, microservices and APIs.

    They rate their attitude toward open source, and how inner source does or doesn’t help them work better. It requires investigation into how they construct their route-to-live and how they architect the toolchain within these contexts.

    We look at security elements, such as secrets management, and some of the technology concepts that are currently emerging and that promise to drive us further forward; robotic process automation (RPA), blockchain and virtual and augmented reality.

    What to Expect From the Assessment of DevOps Capabilities

    This five-dimensional model allows teams and organizations to assess their current state on their DevOps journey. The data collected allows teams to self-discover improvements and accelerate progress, with the guidance of a qualified ADOC consulting partner. Organizations can compare team capabilities across their enterprise, pinpoint high levels of capability and turn local discoveries into global improvements. ADOC is a tool to measure and accelerate a DevOps journey.

    ADOC is now available as an exclusive benefit to Premium and Enterprise Members. Learn more about the assessments at: https://devopsinstitute.com/devops-assessment/

  • Why is Site Reliability Engineering Important?

    Why is Site Reliability Engineering Important?

    Site reliability engineering (SRE) is one of the fastest-growing enterprise roles and set of operational practices for managing services at scale.

    During the Site Reliability Engineering SKILup Day June 18, I asked a group of the DevOps Institute Ambassadors why SRE was important to them.

    Below you will find some insightful responses, tips and calls to action. If you missed SRE SKILup Day or wish to rewatch any of the sessions on-demand, visit the conference landing page and access the videos for free: https://devopsinstitute.com/site-reliability-engineering-skilup-day/.

    Happy Reading!

    Anshul Lalit
    DevOps Institute Ambassador from Kristiansand, Norway

    “The term SRE surely has been introduced by Google, but directly or indirectly several companies have been doing stuff related to SRE for a long time, though I must say that Google gave it a new direction after coining the term ‘SRE.’ I have a clear view on SRE as I believe it walks hand-in-hand with DevOps. All your infrastructure, operations, monitoring, performance, scalability and reliability factors are accounted for in a nice, lean and automated system (preferably); however this is not enough. Culture is an important aspect driving the SRE aspects, along with business needs. As the norm ‘to each, his own’ goes, SRE is no different. It is easy to get inspired from pioneer companies, but it’s impossible to copy their culture and means to replicate the success, especially with your ‘anti-patterns’ and ‘traditional’ remedial baggage. Do you have similar infrastructure and business needs as the company showcasing brilliant success with SRE? No. Can it help you? Absolutely. The key factor here is to recognize what is important to your success blueprint after understanding the fundamentals of it and find your own success factors considering your cultural needs. Your strategy and culture need to walk together, just like your guiding (strategy) and driving (culture) factors.”

     

    Maciej Jarosz
    DevOps Institute Ambassador from Warsaw, Poland

    “With all of those methodologies and frameworks out there, one can only ponder: ‘Which one should I choose?’ Well, all of those frameworks and methodologies provide something of value, as long as you have an opportunity to actually test some of those concepts in your own local environment. Consider SRE for example: It worked for Google, sure. If anyone doubts it there are at least three books on the topic with the materials, proof and philosophy behind SRE. Even if your company is not the scale of Google you still should take a look at best practices that worked in 2017, 2018, 2019 and that are still working now. Considering the crazy times that we live in right now and the plethora of various free materials, conferences, blog posts, research whitepapers and videos on the topic, what are we waiting for? The knowledge is right here, ripe for the taking. Advancing the humanity of DevOps is just a matter of experimentation, trust and continuous improvement. And with events such as the SKILup Day: Site Reliability Engineering where you can network, exchange field stories, brag about your ‘battle scars’ and participate in some splendid speeches, it’s only easier to jump into the fascinating world of SRE discipline.

     

    Helen Beal
    DevOps Institute Ambassador from Chichester, England

    “I might be a bit biased since I’ve been focused on DevOps for nearly a decade as a consultant, coach and learning facilitator, but for me there are a number of systems of thought that contribute to the DevOps body of knowledge. I stole (with permission!) a phrase I heard from DevOps Institute’s CEO, Jayne Groll, many years ago that DevOps is the ‘harmonious, polygamous marriage of ITSM, lean and agile’ and I’ve also presented at several events over years around the correlations between DevOps and Holacracy. At an event in Munich a few years back, John Willis and I started a conversation about a DevOps Superpattern as he also added his ‘three-legged stool’ of The Theory of Constraints, Safety Culture and Learning Organizations’. For me, SRE is another valuable system of thought that we can learn from and use to make practical changes to the way teams and organizations work to drive higher organizational performance.”

     

    Mark Peters
    DevOps Institute Ambassador from Texas, USA

    “I recently saw a social media image for a shirt which broke out SRE as ‘Simply Restart Everything.’ While not entirely true, the concept of a site reliability engineer is important as another way to link Ops and Dev in the DevOps construct. SREs formalize a position whose job is to connect, to link folks together, and to streamline processes. Increasing reliability means to increase simplicity across multiple workflows. The bigger your organization, the more simple the process needs to be. Unfortunately, many organizations increase complexity, reduce transparency, and create wasted effort as they add more individuals. SREs are important in trying to understand processes from end to end and emphasizing outcomes rather than any particular process stage.”

     

    Sean Davis
    DevOps Institute Ambassador from Georgia, USA

    “Site reliability engineering bridges the gaps between platform design, development and operational execution by providing new perspectives on system reliability. When you link software engineering and systems engineering mindsets with operational engineering, you streamline the support of a product-focused on business outcomes not ticket output. It can be a valuable gambit for the traditional approach that pays off big for companies. By trading off specializations in focusing on operational burden, you gain a larger view of the platform and reduce the efforts required to support the product and spend more time automating and innovating to move towards self-healing and autoscaling products instead of manual, reactionary troubleshooting.”

     

    MIke D. Kail
    DevOps Institute Ambassador from California, USA

    “Instead of the word reliability, I prefer to use and approach it as resiliency, which then includes approaches and contributing members of the chaos engineering group that are hyper-focused on continually increasing the ability to recover from any and all failures, which we all know happen in both software and the underlying hardware. Resiliency also should resonate with the core Development teams as they should strive to continuously improve and collaborate with the SRE teams.”

     

    James Wasson
    DevOps Institute Ambassador from Oklahoma, USA

    “We create pipelines, version control, and infrastructure as code like breathing now. That means we are done, right? The beauty of SRE is that it takes us right to where the value is made. It requires us to go back to the beginning and ask questions about decisions we need to make regarding a system. Site reliability engineers help us remove “resulting” from our processes. With the help of these principles, we begin to make near-real-time decisions based on the impacts to our customers, rather than the results of the past. We spend so much time creating amazing features, determining the velocity of our teams, and implementing the newest framework from that great book, but one of the most helpful questions we get from site reliability engineering is ‘How fast is too fast?’ If our organizations can answer this question early, we are able to make more informed decisions that create extended and meaningful value to our customers.“

     

    Biswajit Mohapatra
    DevOps Institute Ambassador from Pune, India

    “The ever-changing digital and cloud landscape brings in an unprecedented need for collaboration, transparency, resiliency, stability, performance, reliability and correctness. SRE sets measurable engineering goals mapping to service level objectives (SLOs) and enables monitoring and tracking of these quality of service (QoS) parameters. SRE is a set of practices focused on reducing silos by shared ownership, planning for failures using error budgets, small-batch changes with focus on stability, automation of manual tasks and introducing a culture of measurement, monitoring and tracking. The fundamental goal of SRE is aimed at depicting a prescriptive approach to plan, build, implement, measure and achieve DevOps objectives with a focus on reliability and automation at every opportunity. SRE covers several critical areas of cloud platform architecture such as orchestrated automation, responsive operation, optimized performance, just-in-time scaling, modernized environment and predictive event management. It’s time to look at operations through the lens of software engineering practices. Leveraging SRE to design, build, operate and enhance software systems is critical to building the culture of risk-managed and reliable digital footprint for enterprises.”

     

    Mitesh Soni
    DevOps Institute Ambassador from Mumbai, India

    “What vs How
    Interface vs Class
    Abstract Class vs Class
    DevOps vs SRE

    Similarity and difference lie in their positions. Ryan Foland has given a wonderful example to describe What and How: ‘What’ begins at the surface level, and usually describes things in a general sense while ‘How’ describes the process of something or the underlying steps and actions that are done to get results. I find this example a perfect fit for the relationship between DevOps and site reliability engineering (SRE). Both DevOps and SRE have created buzz and confusion and will continue to do so. DevOps is cultural change to transform existing practices with people, processes, and tools for a better quality outcome with speed. SRE is a concrete implementation of DevOps practices or principles. I would like to leave one question here. Let me form the base first: Agile and DevOps complement each other. Agile is like a train compartment and DevOps is like an engine. Is SRE like a train track?”

     

    Marc Hornbeek
    DevOps Institute Ambassador from Puerto Vallarta, Mexico

    “Site reliability engineering (SRE) compliments DevOps by measuring and achieving reliability of applications and services working on production and DevOps infrastructure in a prescribed manner using error budgets, team relationships brokered by an error budget, Ops-as-code, and the use of reliability to control deployments.  SRE emphasizes the following objectives that are important to DevOps engineering:

    • Assure availability of applications and services.
    • Assure error rate and latency meet Service Level Agreements (SLAs).
    • Enable large-scale systems while controlling risk.
    • Create operational cost savings by automation and policies.
    • Improve skills of developers and SREs.
    • Resolve conflicts between Dev and SRE.
    • Improve capacity planning and provisioning.”

    Suma Puligella
    DevOps Institute Ambassador from Virginia, USA

    “SRE encompasses both software engineering and IT operations and bridges the gap between them. It ensures that the organization’s systems are scalable, stable, predictable and automated. SRE and DevOps seem like two sides of the same coin with a unified goal of enhancing the release cycle without any compromises. However, there are some differences between SRE and DevOps:

    • SRE covers architectural and design patterns along with reliability implementation techniques such as reactive programming. Whereas DevOps is a set of practices that combines software development (Dev) and IT Operations (Ops). DevOps also aims at continuous software delivery with high quality, reliability and shared ownership among software stakeholders.
    • DevOps raises the risks for the development team(s) to solve. Whereas SRE’s approach is to find problems and solve some of them themselves. SRE teams set service-level agreements (SLA) that define how reliable the systems need to be to end users. Their main goal is to improve performance and operational efficiency.
    • DevOps is all about the “What” needs to be done, SRE talks about “How” this can be done.

    The bottom line is that it’s all about the data. You need information in order to understand how to measure success and failure and how to gain continuous reliability across the applications.”

     

    Tiffany Jachja
    DevOps Institute Ambassador from Maryland, USA

    “Organizations are finding the best practices for delivering faster and meeting customer demands. Site reliability engineering is about building confidence that our systems work as expected. SRE defines the methods, concerns, and responsibilities needed to maintain or increase the reliability of a system. It improves our ability not only to deliver a product to end customers but also to do so in a sustainable way as an organization. Many workloads can be costly, complicated, or time-consuming for different reasons. These practices provide a foundation for the root analysis of our systems, at the end of the day we want to be able to set key goals, measure systems, and ensure a product or service fulfills its objectives. This is important for anyone with a stake in software development and delivery.”

     

    Uchit Vyas
    DevOps Institute Ambassador from Singapore

    “Site reliability engineering is a discipline that seeks to create reliable IT systems by managing the rate and quality of change, ensuring clients protect themselves from any break in service and avoid being subject to fines and penalties.

    As per my experience, SRE practices are mainly based around the following elements:

    • SLIs, SLOs and SLAs.
    • Monitoring.
    • Toil Definition.
    • Error Budgets.
    • Test Automation.
    • Capacity planning and measuring risk factors.

    SRE implements a lot of the same practices as DevOps, but from an operations perspective focuses predominantly on operational stability and resilience of the production estate.”

     

    Zuzana Homolyova
    DevOps Institute Ambassador from Slovakia

    “SRE works as a kind of connection bridge between development and operations as it cares about every process from the source code to deployment. It really focuses on how to bring scalable and reliable software systems into the real infrastructure, way, application, performance and security. To describe the difference between DevOps and SRE I would say: ‘DevOps asks what needs to be done but SRE asks how it can be done.’

    What is really important is that DevOps and SRE do overlap each other. They share goals to understand how to measure success or failure and how to gain continuous reliability across every application.”

     

    Craig Pearson
    DevOps Institute Ambassador from Newcastle, England

    “I’ve walked the line between DevOps and SRE in organizations and witnessed first-hand both good and bad practices. While there are a lot of SRE success stories, I’d like to focus on the occasional use of SRE as a justification to rebuild the walls around operations/production that DevOps strived for so long to take down. While adopting  SLOs and focusing effort on toil reduction are obviously beneficial, taking a defensive position over the production environment is not. The trigger for this is often an issue impacting production—sometimes major, sometimes minor—or the perception that more audit rigor is needed in this new continuous delivery world. Whatever the trigger, sometimes SRE is proposed as the answer (R stands for reliability, right?) SRE has to shift-left and spread more of the wisdom of production to teams delivering value-adding products and services to that production environment. Deploying value rapidly is crucial. Doing this in a reliable manner is key. Both are valid reasons for introducing site reliability engineering—the future. However, keeping the wisdom of production in a separate (silo?) SRE team and withholding access to production is a backward step.  A touch of back to the future maybe …”

     

    Nadeem Augustine
    DevOps Institute Ambassador from Cape Town, South Africa

    “Having been exposed to DevOps and SRE, I do see a difference between the two roles. I often find the DevOps way of working focuses on how to bring the issues into development and a way to process it between teams. This creates a collaborative way of working in the organization. That is a pivotal part in rolling out DevOps. DevOps is the ‘what‘ that needs to be done in order to bridge the gap between development. Having played both roles as a ‘developer’ and ‘operations,’ you now see how SRE plays a deeper role in assisting the organization in achieving the end goal. As an SRE you focus on the ‘how‘ to resolve issues end to end with the quickest most stable resolution. Having resolved the issues you create a more reliable and repeatable process.”

     

    Daniel Oh
    DevOps Institute Ambassador and CNCF Ambassador from Boston, USA

    “Who would be the most valuable resource to stabilize the cloud platform built on an immutable infrastructure (i.e. Kubernetes) for the Ops team? Here is the evidence that many recruiters continue to seek the roles that are named site reliability engineers (SRE) but also more IT Ops folks want to have SRE responsibilities in their careers.“

     

    Hugo Lourenço
    DevOps Institute Ambassador from Lisbon, Portugal

    “How long does it take to find the product you need? How long does it take to get the product delivered? Does the product/service require training or expert assistance? Do you need other calls +, products +, services +, to make this product / service work? Does the product require external maintenance and you will be accountable to remember the provider when it’s needed? To get support assistance or any other contact with the provider consumes your valuable time. Your time? All these reasons were the reasons why I embraced agile, lean and automation = DevOps, DevOps is the only weapon in a digital era. I’m passionate about building products and services where consumers will pay to keep your company to exist and deliver such amazing products/services!”

     

    Felipe Duenas
    DevOps Institute Ambassador from Quito, Ecuador

    “SRE is more than buzzwords and trends. It is far more about the people and process than the technology that underlies them. While technology certainly can (and likely will) adapt to the concept as it matures and more teams adopt it, you don’t need new tools to align your development and operations organizations around the principles of Site Reliability Engineering. SRE ’embodies the philosophies of DevOps with a greater focus on measuring and achieving reliability through engineering and operations work.’ SREs also ensure that everyone in the company agrees on how to measure reliability, and what to do when availability falls out of specification. This includes contributors at every level, from developers, through team managers and all the way up to VPs and executives.”

     

    Suresh GP
    DevOps Institute Ambassador from Singapore

    “Site reliability engineering is redefining the value of infrastructure operations and its relevance in the wake of DevOps toolchain and deployment pipeline. After a long time, we see that this area of discipline is creating a profound impact on the holistic value chain by building world-class resilience and reliability of products and services with an end-end view. It also focuses on creating a sense of shared ownership and experimentation in challenging the status quo and delivering streamlined customer experience. I see a lot of synergy with Agile, DevOps, Lean, ITSM with SRE that compliments the dimension brilliantly. Last but not the least, cultural change is of phenomenal importance to make this transition happen and be sustainable. The principles and practices are so sound that the legacy environments can still take advantage of it and contribute to velocity increase and improved reliability.”

     

    BMK
    DevOps Institute Ambassador and CNCF Ambassador from Wellington, New Zealand

    “Enterprises who run or want to run a cloud application at scale, need to adapt to SRE practices and disciplines. Merely stating a 99.99% SLA does not help, if you care about your customers and your customers’ experiences with your systems then, yes, we need to level-up our game, we need to understand the nitty-gritty of reliability of our systems, operations and more importantly understand the limitations and constraints within which we need to operate. Some useful pointers: 1) Caution: Do not simply rename your ‘system admins’ to ‘SRE’  2) Upskill your ops folks to code 3) Upskill your dev folks to learn and know more about infrastructure and operations.”

     

    Aparna Balasundar
    DevOps Institute Ambassador, India

    “If DevOps is the extension of agile to build continuous delivery pipelines, SRE is a natural extension of DevOps to build resilient, repeatable, reliable, scalable production systems. Shift left to include the SRE as part of the architecture design, build, deploy favoring long term viable systems. With the cultural mind shift, if SREs have to provide safe and reliable systems, developers have to dedicate building such systems. These joint working partnerships will benefit the overall business goal.”

     

    Simone Jo Moore
    DevOps Institute Ambassador from Occitanie, France

    “Site Reliability Engineering has as many touchpoints across the service value stream as any other practice. I challenge you to map your interactions across your enterprise so the impacts of SRE to the value outcome are clearly visible. Those companies that treat these touchpoints as part of their beating heart competencies versus peripheral consideration, will continue to leap forward. The socio aspect is important here. Technology is a catalyst—people’s interaction with the tech has shifted, is shifting and continuing to shift. Security is embedded; there is no point at which practices (human and automated) do not exist in SRE enterprise-wide. The resources we apply depends on how big and complicated the system is.

    It goes beyond technical feedback to being an advantageous feedback loop for improvement across business objectives with engagement and financial benefit. Think of SRE as a movie, yes I’m a sci-fi fan but think about movies like “Mission to Mars” or “The Martian” and even “Ad Astra.” In each of these, reliability also includes our trust in something being resilient. Even when things don’t go as expected, there are ways in which the system can be adapted to solve problems. Resiliency is about humans rather than technology, but think about the progression of AI. We want AI to become resilient in not about coming back to a baseline but move forward to a new baseline of its own accord. Its point of stasis is never the same yet reliable based on the quality of the engineering. Our point of stasis is never the same either, and the cycle continues.”

  • DevOps Days London: The Human Factor

    DevOpsDaysLondon16logo

    Day 2 of DevOpsDays London 2016 kicked off with a presentation from the legendary Gene Kim. The theme that dominated today’s proceedings was the “human factor,” (fitting, as we finished Day 1 with #humanops) nicely started by Gene’s quote from Nathan Shimek: “We need DevOps to make our work humane.”

    The second day delved beyond what makes people happy, posing questions regarding what it is to be human, and why automation is not everything when it comes to DevOps. As Gene said, “DevOps transcends the technology stack we are working with.”

    DevOps aims to supplant pain with joy and help people find autonomy, mastery, purpose and deep satisfaction in their working lives in technology. Werner Vogel, CTO at Amazon, is often quoted as saying: “You build it, you run it.” But Gene took this further as he explained how a developer had described his most satisfying working experience as coding something and deploying it and monitoring it and fixing it, too. He also touched on the power of shifting left, using Test Driven Development—and its demand that we write the test script in before we even start to code—as an example. That’s building quality in. That’s cross-skilling and cooperation.

    It resonated with the audience when Gene said there was one single question that, when answered, tells us everything we want to know about an organization’s DevOps maturity: “To what degree do you fear deployments?”

    We came back to this question later in Jeromy Carriere‘s talk about his experiences of scaling at Google. Going the gym for the first time is painful—we have to build muscle memory, he said. We need to practice these processes until they are like breathing.

    Gene made a book recommendation:  “The High Velocity Edge,” by Steven J Spear. When we do DevOps we are aiming to transform into a dynamic learning organization. We should not expect but be required to talk about problems. No fear, no blame. Everyone in an organization should be trained to run a blameless post-mortem—or a healthy retrospective, if you prefer. As Claire Agutter put it: “Improving daily work is more important than doing daily work.”

    Gene then outlined the DevOps Enterprise Summit London event coming up at the end of June and the extraordinary stories of experiences people share about their DevOps journeys during DOES events. He said: “Often the transformation leader was putting themselves in personal jeopardy.”

    I can testify to that. It takes courage, optimism, conviction and tremendous amounts of energy to be a change agent.

    The human theme continued into the panel discussion as the COO of Technology at Barclays explained how its command and control structure, waterfall-ruled development and volume of changes resulted in total gridlock. He said that the big challenge is trying to re-engineer the organization to behave differently: “Changing HOW we do our work is the hardest thing.”

    Changing human behavior is very challenging indeed; as Joanne said, measurements have to change and we have to focus on customer value. Gareth Rushgrove also majored on the human angle in his talk on DevOps and microservices. He said: “Microservices are as much about people as technology,” and, “Which comes first: the software or the team?”

    Essentially, DevOps and microservices help us to make lots of small bets—spreadbetting, if you like—spreading risk and accelerating feedback. And we need small teams to be able to do microservices and move away from monolithic applications.

    One particular Ignite talk stood out from the rest, and won the hearts and minds of the audience: @coldclimate from Server Density. Through a personal story of a back injury brought on by prioritizing work over looking after himself, the #humanops message: “You are not immutable, you are not highly available!” struck home hard.

    So there we have it. My key takeaway from DevOpsDays London 2016? DevOps is less about technology and automation and more about happy, healthy humans.

  • DevOpsDays London 2016: Making Happy

    DevOpsDaysLondon16logo

    We haven’t had a DevOpsDays event in London since 2013. The audience size was about the same at this year’s event, held April 19-20, as I remember from last time, but about 85 percent of the audience were attending a DevOpsDays event for the first time. It was great to see so many new faces, and I guess the high proportion of first-time visitors could be a sign of the maturing and proliferation of DevOps thinking.

    I’m delivering a webcast April 26 titled, “DevOps for Architects,” so on the way to the event I read this book in preparation. At Ranger4 we ask people a lot: “Are you having a good day?” or, “How do you know you are having a good day?” Something has always discomfited me slightly about this, in the same way that “What does good look like?” and, “Just good enough,” trouble me. Can’t we aim higher? What does “great” look like? In the book, the authors talk about “happy days,” and this makes me happy. And this was the theme I took away from the first day at DevOpsDays London 2016—DevOps makes happy.

    Here are a few reasons how:

    First up on the stage was Bridget Kromhout, a DevOpsDays organizer I last saw speak at the DevOpsDays five-year anniversary in Ghent in 2014. Her talk was titled, “Cloudy with a Chance of DevOps”—a succinct introduction to DevOps as it looks today with some key takeways:

    • “Automation is not everything.” Most people get this now, but we meet a lot of people who started with automation and realized they’d skipped to the end. Start with culture.
    • “DevOps is practiced, not purchased.” You might be surprised to find us supporting this, since we make a living out of selling DevOps. But we don’t. We believe that the best way to help organizations transform using DevOps principles is to partner with them—to coach, motivate, guide, challenge and help them move through their journey. This is what we do.
    • “It is not necessary to change. Survival is not mandatory.” This is a quote from the “Godfather of Lean,” Deming.

    Bridget was followed by Joanne Molesky, the co-author of “Lean Enterprise,” a frequently cited must-read for DevOps enthusiasts.

    Key messages from Joanne’s talk were her assertion that technology is no longer the bottleneck, particularly now that conventional business models are under attack. Joanne also cited quantum computing as the key disruptor she sees coming through in the near future. It was particularly interesting to think about what she said about Tesla disrupting not because the company has designed a better, more affordable electric car, but because it is changing the game in terms of how individuals will access the power grid.

    Following on from Bridget’s recognition that not all companies are any where near delivering on the velocity promise of DevOps and the subsequent happiness, key barriers Joanne identified include slow decision-making processes and conflicting goals embedded into many enterprises that have been around for a significant amount of time. She described “process madness” and how most enterprises “plan, do” and forget the “check, act” part of the PDCA cycle. She explained how many organizations have created walls of “no’s”—effectively, “Culture says no”; a sentiment shared by many individuals we spoke with after her session.

    A popular quote that was tweeted multiple times from Joanne was: “You only fail when you fail to learn.”

    Speaking with organizations that are moving to agile and DevOps, many of them talk about their difficulties moving from project-centric thinking to product-centric thinking, a point reiterated by Joanne. She recommended the three horizons framework as a useful tool to plan and enact change while managing the current state. She also had some very useful advice for organizations trying to tackle regulatory requirements: “Understand the intent.” Organizations have a habit of misconceiving and conflating what the regulations say.

    “The best code is that which you don’t write”—a bit like the advice to writers: “kill your darlings.” This is about being lean, planning properly and removing extraneous effort.

    Close to our own hearts, Joanne also emphasized the importance of measuring the things that matter. Similarly, she pointed out that organizations that have a classic “command and control” structure have no hope of being able to do DevOps. When organizations perform cultural change, they must amplify their feedback loops at all levels. By using principles like holacracy, organizations can understand who the sensors are in their business and learn to hear ideas, how to make the process madness sane and let ideas germinate and blossom.

    I frequently quote John Willis:

    “DevOps finally proves how IT can be a strategic advantage that allows a business to beat the pants off the competition. This is the moment we’ve all been waiting for.”

    For far too long IT has been seen as the guys in the dungeon, (ever seen the IT Crowd?) but as John said, these are exciting times for us in IT. As Joanne put it, we used to say IT supports the business, then we said IT is aligned to the business, now we say IT is integrated to the business or, even better IT is the business. This should make us all happy.

    Kris Saxton followed Joanne with a talk we were anticipating highly: “Bi-Modal IT and other snake oil.” We’ve been challenging this term recently, too, so I was happy to find more support that this is an oversimplification of the situation organizations are in and is counterproductive, undermining the type of culture we are trying to create by creating more silos, more “them and us-es.” Put succinctly, Kris said: “Bi-modal IT is poison.”

    As Jez Humble recently said:

    “Gartner’s model rests on a false assumption that is still pervasive in our industry: that we must trade off responsiveness against reliability… This assumption is wrong.”

    Kris said there is a “happy place” where we can have both responsiveness and reliability, velocity and quality, speed and stability. DevOps helps us do this by shifting left, having multi-disciplinary teams, a people-focused culture working to small, scalable scope—and more.

    The rest of day one was filled with slots from the many sponsors of the event, a demo from Pivotal, Ignites and Open Spaces. The last three thoughts I’ll leave you with though were:

    1. Sponsor ServerDensity used its pitch to highlight health and well-being in IT; #hugops. The company is setting up a MeetUp group in London for HumanOps, emphasizing that DevOps is not just automation and it’s here to make humans happy
    2. Microsoft’s Thiago Almeida shared stats on how DevOps initiatives at Microsoft have positively influenced employees’ work/life scores
    3. John Clapham spoke on how he set fire to his car and what this taught him about the currency of empathy and how this can be used to improve life at work

    Check back for coverage of Day 2 of DevOps London.

  • 6 Common Causes of Conflict DevOps Sets Out to Solve

    6 Common Causes of Conflict DevOps Sets Out to Solve

    It’s a very exciting time to work in Information Technology – as John Willis* put it:

    “DevOps finally proves how IT can be a strategic advantage that allows a business to beat the pants off the competition. This is the moment we’ve all been waiting for.”

    I work with organizations that are palpably excited that the software and applications they are creating deliver groundbreaking, mission-critical capabilities to their end users and plan to outsource all their ‘generic’ IT – the services that don’t offer anything unique to their customers’ experience. The challenge though, is that these organizations have a lot of habits and history to deal with; while the business puts pressure on IT to develop and deliver more innovation faster, the increasing criticality of these applications and the evolved complexity and fragility of the systems they run on results in lots of risk and lots of pain when things don’t go according to plan. Here’s how we see this often playing out in today’s enterprise IT marketplace.

    Herewith, 6 of the most common causes of conflict DevOps sets out to solve that we hear when we’re talking to our customers:

    1. I need a server… now

    This is without a doubt the most common complaint we hear: that operations teams, or those responsible for making infrastructure available to development are blockers, too slow and don’t understand the needs of the business. What we see actually happening is demands for immediate drops of physical and virtual servers from teams who not only have literally just heard of this requirement coming down the line but have a pretty full on schedule of planned work too. Oh, and more than likely unexpected system failures and defects are generating a fair amount of unplanned work too having a direct knock on effect to their ability to respond to these demands and their planned work.

    Recommendations:

    • Culturally something has to happen to ensure everyone knows that they are on the same team and it feels like it. Try involving operations in the development cycles earlier so they have visibility of what’s coming down the line and are bought into the overall goals of the project
    • Make it easier to deliver new infrastructure by automating it, creating or using cloud and virtualization infrastructure
    • Aim for an on demand deployment capability
    • Analyze your teams’ planned/unplanned work proportions and try and figure out ways to eliminate the problems that distract from the goals trying to be achieved – this might be better testing or better monitoring of outages

    2. I want access to the production systems

    This one usually comes about after the developers have reached a tipping point in their frustration in not getting their needs/demands met in what they would consider a timely fashion. Sometimes it indicates that an organisation doesn’t appreciate or hasn’t implemented workable controls in the operations teams. DevOps doesn’t mean throwing all process out of the window and putting live environments at risk – it’s by no means mutually exclusive with methodologies like ITIL but it is about individuals collaborating better to achieve a shared desired outcome.

    Recommendations:

    • Read this article: “DevOps Does Not Equal Developers Managing Production”(link)
    • Involve development in the definition of your change control processes and set up of any systems
    • Set some Service Level Agreements as a team

    3. Something is wrong…whose fault is it?

    One thing DevOps is very clear it’s not about is blame. The “S” in CAMS* is for “Sharing” – that’s sharing ideas, goals, workloads and responsibility – it’s about being innovative, taking calculated risks, protecting against failure and BEING NICE. That means no finger-pointing and no swearing at each other.

    Recommendations:

    • Watch this video from AppDynamics: “The Real DevOps of Silicon Valley”
    • Read more about running a good, healthy retrospective – and practice it. Here’s an example from the UK Government Service Design Manual
    • Consider implementing tooling that will enable you to pre-empt failure or fail smartly. And that will help you identify the root cause fast

    4. So many alerts they are meaningless

    We spend a good proportion of our time working with companies to implement systems for smart failure or that can pre-empt failure as mentioned above. This is one third of our devops utopia triumvirate. The kind of systems that can deliver this capability are the new breed of Application Performance Management solutions like appdynamics. One of the challenges we often find though is that enterprises already have monitoring systems. Lots of them. Monitoring lots of different things. All producing a multitude of alerts. This is when we see alert fatigue. This is not something I believe is, or have indeed witnessed to be, relevant only to the healthcare industry. When someone receives a lot of alerts, of which a very large proportion are meaningless, it becomes highly likely that the critical ones are missed.

    Recommendations:

    • Review and consolidate your existing systems
    • Take some time to really understand what alerts matter and adjust your policies according – and consider this an ongoing task to tweak your systems as your situations change
    • Some DevOps proponents recommend alerting development on system outage immediately – I think this could work, if development are bought into the process and not thinking they are simply being blamed (see 4 above) – sharing the load is definitely a good idea, as long as the process makes overall responsibility clear and forces resolution

    5. I don’t have time to save time

    We hear this one a lot, particularly when we start to talk about business cases for automation tools. The two things driving DevOps (1: the increased rate of innovation and 2: the fragility of today’s evolution of IT infrastructure) together result in very high workloads and a LOT of unplanned work. Throw a recession and budgets that have been squeezed hard year on year on year and what you get is a lot of late nights and a lot of missed targets. DevOps is designed to streamline the throughput of work at a cultural, interaction and application based level – diffusing the pressure building up in many IT departments today.

    Recommendations:

    • Consider ringfencing some budget so invite in an external consultancy to perform an audit or assessment of your current DevOps state – you may feel they won’t tell you anything you don’t already know, but you will have someone documenting your position for you and giving you justification to take the next step in your transformation
    •  Make time – find a senior decision maker who will listen and mandate a project where you can prove something – for example the savings made by automating a piece of a process

    6. Our hero is a bottleneck…

    This one is particularly common when we are talking about release and deployment automation. And if you’ve read “The Phoenix Project” * you’ll recognize Brent as being one of these. What tends to happen is this: one day an engineer decides that building servers is a tedious, manual process and that they could write a quick script to make that bit a bit quicker. The next day, they write another one. And another one. Until, over a period of months or even years a whole bunch of scripts have been written automating lots of parts of the process – I like to call this a ‘script monster’. The problem is that even though the script monster works pretty well most of the time, when it fails it’s virtually impossible for anyone other than its creator to troubleshoot. So when there’s an outage at 3am, you have to call ‘Brent/the creator/Frankenstein’ and he breezes in and saves the day. But what if he isn’t there one day?

    Recommendations:

    • Create an environment embracing autonomy, mastery and purpose – where ideas are heard and acted upon. Often these ‘script monsters’ wouldn’t have been born if Frankenstein had felt he had alternatives to explore
    • Understand motivations: yours and your colleagues’. Some people are particularly motivated by certainty at work and if they have made themselves indispensable by creating something like a script monster, they may feel threatened if someone proposes to take it away from them
    • These heroes are often genuinely some of the most talented members of the team, so if they are resistant to change, talk to them about exciting new projects they can be involved with – make them the master of a shiny new system that will multiply the frequency of releases ten – one-hundred fold by allowing others to push them through an automated process – deployment on demand

    In Summary

    DevOps isn’t voodoo, a silver bullet or automatic. But it is a drive to encourage a different, positive and innovative way of thinking to address a common set of issues affecting the experience of living in, working with and using IT in the world we live in today. Be DevOpstastic.

    What other causes of conflict have you experienced that DevOps can resolve?

  • 10 Top Tips for DevOps Cultural Change

    10 Top Tips for DevOps Cultural Change

    DevOps is commonly recognised to be primarily about creating a culture of effective and seamless collaboration but this is often the hardest piece to enact when it involves changing an existing culture (as opposed to creating a culture from scratch in a startup type environment and having the luxury of hiring individuals with certain mindsets and defining working practices from day one). So how can you change culture? Here are our top ten tips:

    1. Learn to Trust

    It sounds a little cheesy I know, but at the core of what DevOps is trying to fix is the conflict that has come about from years, decades even, of development and operations teams being segregated from one another and not being enabled to communicate, let alone collaborate, effectively. This situation has been exarcebated by the pressures to innovate and push rapid change battling with the need to stabilise fragile systems – and created what James Turnbull described as a ‘toxic’ working environment.

    So the very first thing that needs to happen is a unified agreement and understanding that we are all on the same team.

    DevOpstastic Tip ~=>
    Take some time to review your individual and team goals and ensure none are in conflict. Then create a single goal that brings everyone together. Get everyone to agree it. Fire anyone that can’t. Kind of joking – but kind of not.

    A change in culture will demand changes in the way that you do your daily work – habits will need to be broken.

    DevOpstastic Tip ~=>
    Shift left – arrange your teams around projects not linear tasks and involve operations staff who will be responsible for your application in production much earlier in the process. Being involved sooner will help them plan for when the code is ready and feel they also have some skin in the game.

    2. Understand Motivations

    Conflict often occurs when people don’t understand each other – I’m sure we’ve heard the phrase: “They have a personality clash” countless times over the years. Humans and our brains, personalities, backgrounds and characters come in a whole heap of flavours and this is a good thing since, as my mother always says: “It would be boring if we were all the same.” However, sometimes at work, under pressure, it’s difficult to take the time to understand why someone is reacting or thinking differently to you and the immediate response is to fight or denounce it; not just unpleasant but downright unhealthy.

    So if we accept we are all on the same team, how can we all get to know each other better and grease the wheels of collaboration? I know I’ve been subjected to countless team-bonding exercises that I’ve found quite excruciating over the years so I am applauding the new wave of neuroscience that allows us to be a little more exact about what’s going on in our heads than just try and decide whether I am a green or red personality type.

    Our particular favourite at the moment is SCARF from The Brain at Work – the premise is that the same reward circuit in our brains that we use for survival and defence is also used for our motivations at work. So if I am motivated by Autonomy (the ‘A’ in SCARF) and someone at work attacks my sense of autonomy in the workplace (by, for example, micro-managing me) I will have the same neuro-response as if they made a direct attack on my life – including the release of the related neurochemicals, in this case, cortisol, the stress hormone. Similarly, if something happens to reinforce my sense of autonomy (for instance, an event I created has an excellent turn out and feedback) I feel great and get a nice rush of dopamine.

    DevOpstastic Tip ~=>
    Have a SCARF workshop with your team(s) and ask everyone to complete this self-assessment first, share the results and discuss. We don’t always agree with the results of this assessment tool but finds that only adds fuel to the fire of debate.

    My key takeaway here is that we should try to understand ourselves first, and then those around us (a bit more ‘fluffy’, but this is about mindfulness). Having this level of insight into ourselves and our colleagues means that when conflict occurs we have a better chance of taking a breath and understanding what’s really going on, remembering we share that common goal and working closer together, collaborating to reach it.

    We’ve more recently also starting working with unconscious bias – we’d love to hear from you if you have any experiences in this area.

    DevOpstastic Tip ~=>
    If you want to know if you have unconscious bias you can test yourself using Harvard’s online Project Implicit testing tool.

    3. Eliminate Blame

    Let’s start by watching one of my favourite DevOps videos – here you are:

    [youtube http://www.youtube.com/watch?v=2PjVuTzA2lk?wmode=transparent]

    If we’ve taken on board tips one and two and are now in a ‘circle of trust’ we should be able to avoid this kind of finger-pointing blame game. There is no place for blame in a DevOps culture – it is a pointless waste of emotion. We need to understand what went wrong in a retrospective but the most important thing is how to make sure it doesn’t happen again.

    DevOpstastic Tip ~=>
    Try running a blame free post mortem/retrospective next time you have an outage. Here are some retrospective plans you might want to road-test.

    4. Embrace Smart Failure

    In order for businesses to thrive, survive, even, in today’s uber-competitive markets, innovating at speed is an imperative. When we are assessing organisations’ DevOps cultural maturity, one of the features we measure is the organisations’ capability to ‘dance around the edge of failure’. Innovating at hitherto unseen velocity introduces a whole host of risk that must be mitigated. Systems need to be created that can warn and pre-empt failure and capabilities need to be implemented to do things like instantly redeploy a last known good state or working version of an application in the event of a failure. In no way are we suggesting that failure should be celebrated.

    In addition to creating an safe environment, where failures are quickly and easily fixed, you can also prepare for failure by spending some time prioritising features and architecting applications in a composable manner – for example, your business transactions are probably more important than your social feeds. At a recent workshop we were told a story of how the French railways online ticketing application was down for three days due to a problem with its analytics – if the components weren’t so tightly coupled, the users wouldn’t have suffered as the reservations system could have been operating standalone.

    DevOpstastic Tip ~=>
    Study Netflix’s Chaos Monkey approach and identify a way to test failures safely in your infrastructure.

    5. Focus on Bottlenecks and Flow

    If you’ve read The Phoenix Project you’ll know all about The Theory of Constraints – our favourite summary of the concept is that “Any improvement not made at the constraint is an illusion”. Work hard as a team to identify the real constraints – and watch out for that common pitfall as described via the Brent character in the book that those people who often look like heroes are, in fact, your bottleneck. Try to understand the real value to your business of each improvement and prioritise them accordingly.

    DevOpstastic Tip ~=>
    Try playing ‘Featureban’ – exercise #7 in our DevOps LiftOff Workshop

    Don’t be afraid to rethink your process or find a new pair of eyes to see the things you can’t because you are too close. Check out this video to see the improvements motor racing have made to their process over the last fifty odd years – check out the resource utilization, perfect process and automation in 2013:

    [youtube http://www.youtube.com/watch?v=RRy_73ivcms?wmode=transparent]

    6. Eliminate Unplanned Work

    Another major topic from The Phoenix Project and a universal complaint we hear from organisations when we start working with them: “How do I find the time to save time?”. People know their processes can be improved and there’s a good chance automation could exploit advantages even further but if the culture is to keep doing things that are breaking and broken it’s tough to put aside the hours outside of the firefighting to make DevOps happen.

    It’s nigh on impossible to tell a business they can have innovation or support but not both but it’s sensible to get a handle on the projects running and the resources available and the volume of unplanned work coming through the pipe. Some lower priority projects may need to go on hold while the DevOps project has higher priority – this will probably need to be mandated from management and have a solid case behind it that all subsequent projects will be accelerated as a result once the foundation has been laid.

    DevOpstastic Tip ~=>
    Read The Phoenix Project – what on earth are you waiting for?!

    7. Be Continuous

    There a lot of these out there – we’ve seen continuously:

    • integrate code
    • test
    • integrate
    • deploy
    • deliver
    • fund

    The idea is to always have software in a shippable state and have a DevOps toolchain capable of deploying on demand – have your software development lifecycle like a manufacturing pipeline.

    Our favourite here though is ‘continuously fund’ – as Forrester say:

    “Organizations that deliver at rapid release cycles create a stable basis for funding a continuous stream of innovation. They use customer feedback and business priorities to continuously prioritize the capabilities that will result in the greatest business value.”

    DevOpstastic Tip ~=>
    Try setting a pre-approved innovation budget and short, iterative approval cycles rather than annual, large project driven budgets.

    8. Form Dedicated, Cross-Functional Teams

    Lots of people ask us about creating DevOps teams, something we think it mostly counterproductive as it has a tendency to create another silo. However, there are challenges here as it’s impossible to make everyone a master of everything, particularly in the case of more complex technologies. Simpler technologies can have the advantage of bringing people closer to business.

    We recently held a workshop with Jonny Wooldridge, the CTO at The Cambridge Satchel Company where we discussed the DevOps team he built and managed during his time at Marks and Spencer. Jonny’s team acted more like a ‘tiger’ team, where key resources could be drafted into project teams as needed to evangelise the DevOps way and help cross-skill through the project lifecycle.

    DevOpstastic Tip ~=>
    When you are considering bringing on a new technology, consider the time to mastery – the shorter this is, the better chance your team has for cross skilling for it.

    The key is to get everyone thinking in a DevOps manner and cross-skilling and obtaining mastery as time and resources allow – but arrange your resources around projects, rather than their core skillset.

    9. Love Transparency

    The adage says that knowledge is power – but this isn’t always the case. Sometimes you are better off sharing your knowledge – not only may you gain respect and credibility, you may find that the team around you are able to then work faster to achieve your shared goal – and encourage others to share their knowledge with you too. We can’t possibly all know everything but if we keep things to ourselves, we are often stymying the performance of our team and therefore ultimately ourselves.

    DevOpstastic Tip ~=>
    Write a blog post about an area of your work you are passionate and enthusiastic about. Share it with your team. Share it with the world. Repeat.

    Knowledge hoggers can often be bottlenecks – for instance, we often see cases where an individual has had responsibility for a deployment process, become bored of repeating it manually and written scripts to speed it up – and more scripts and more until they have a ‘script-monster’. Now they are the only person that can possibly understand this part of the process and when anything goes wrong they’re the only one that can fix it – what a hero! If a person is motivated by Certainty (back to SCARF) they’ll certainly enjoy the sense of security and indispensability it will give them – but it’s not so helpful for the rest of the organisation as now they are a serious bottleneck.

    Transparency and visibility are features of DevOps cultures and, going back to tip 2, Understanding Motivations, those of us motivated by fairness will feel happier in a culture of openess and equality. Secrets can be very counterproductive and alienating.

    10. Build Autonomy, Mastery and Purpose

    Changing your culture will probably mean you need to revisit the way you target and reward the people in your business and review the goals you have set, ensuring they are not in conflict and there is a clear, single goal everyone can focus on. The psychology of rewards may surprise you – have a look at this Ted Talk from Dan Pink.

    You need to be able to create a working environment where people feel they have autonomy, mastery and purpose – these are the ultimate rewards. Take a look at this blog for an exploration of DevOps goal setting and rewards in more detail.

    DevOpstastic Tip ~=>Have you heard of The Illusion of Control? If not, spend a little time researching it and imagining how it could affect your outputs at work.

    What’s your top tip for making DevOps cultural change happen?