devopsy.kinsta.cloud

Author: Bob Davis

Building a Platform for DevOps Evolution, Part Two
In the introduction to this series, we discussed the concept of change and how it applies to DevOps environments and release managers.

In a world of constant changes in our surroundings and user demands, DevOps has to maintain hundreds of moving pieces across large organizations of people who, by nature, are adverse to change. This requires release managers to delegate power, transition from a centralized commander to an enabler, understand different value streams and constantly experiment.

In this section, we’ll cover some of the ways and types of experiments DevOps organizations can conduct to improve their release management procedures and meet their objectives using DevOps principles.

Value Stream Mapping

Value stream mapping is a series of steps used by organizations to provide products or services their customers want or need. In the traditional sense, the release and deployment process can seem tedious, especially if teams must communicate with a release management team to schedule release slots on a calendar.

Despite its advantages, value stream mapping has a few drawbacks. It requires a lot of people and it can be difficult to coordinate and sparsely repeated.

Value stream management platforms can extract the four key DevOps metrics for teams to inspect and adjust in real-time. However, there are some shortcomings with using the four metrics:
- Lead time is characterized as time from code commit to live; however, in value stream management, we also consider the time it takes for an idea or customer request to reach value realization.
- Deployment frequency becomes interesting when teams deploy sporadically but are still improving, but when teams attain the ability to release on demand, other metrics become more helpful.
As a result, teams must alter metrics over time to match team ability. This is why it’s critical to empower value stream teams to discover and measure their own metrics. Another factor is that people are more likely to change when doing it for themselves rather than having it done to them.

Lessen Batch Size

Reduced batch size lowers risk and reduces the time an activity waits in a queue. When changes are minimal, they may be implemented quickly and with less danger of causing disruption, thereby making the distinction between the phases less significant.

While some key metrics may be accessed through specific tools such as backlog or release automation, without a value stream management platform, it’s more difficult to gain insights into overall queueing time.

The value stream map is automated when all aspects of the DevOps toolchain are combined, resulting in a unified system from idea through the realization that allows teams to understand their cycle time and value flow efficiency and observe how changes improve work.

Evolving Architecture From Monolith to Microservices

Most organizations have monolithic, tightly coupled applications because that was the best software design approach available in the past. However, as the industry evolves, release managers become increasingly conscious of the limitations of this architectural approach:
- Large batches of work are required to build, test and deploy.
- Dependencies must be managed, including people and systems.
- Dependencies cause fragility.
- Fragility causes bureaucracy (e.g., CABs).
- Bureaucracy and handoffs hinder flow.
Rearchitecting applications to microservices, like changing human behaviors in a system, does not happen overnight, and there are frequent bottlenecks along the way. This is often a “change or die” conversation. If the application does not become more capable of change, market disruptors will provide new experiences to consumers, and customers will jump ship for these competitors.

A value stream management platform can reveal where restrictions, delays, and dependencies occur and how they might be reduced. As a tenet of DevOps says, “Don’t manage dependencies; break them.”

For many companies, however, the road to systemic autonomy, or loosely coupled systems, is lengthy with many incremental steps of improvement. This method is recommended by the strangler pattern: chipping away at the monolith one bit of functionality at a time.

During the application architecture change, we need to support value stream teams, so they don’t waste time on unexpected work caused by unknown dependencies. We also need them to see and appreciate the improvements they’ve made with the support of a value stream management platform.

Part of the release manager’s job is to identify value stream success stories and share them with the other value streams so that local improvements can become global ones.

Transfer the Release Management Role to the Value Stream Team

When teams work independently on small changes, they can release them as soon as they are ready. However, moving from one state to another takes time to transition to that stage of the journey.

In a traditional method of working, the release process is likely to be coordinated by a release manager or a team of release managers. The release manager role can be transferred to the value stream team, with the team providing access to systems that provide visibility into current releases to the release manager.

Transparency into operations across value streams provides traditional release managers with great comfort during this transition, relieving them of the administrative strain. It allows individuals to learn new skills and refine the meaning and purpose they derive from their work.

By taking an end-to-end and cross-value stream approach to change and improvement and ensuring blame becomes accountability, release managers can be key to the success of these improvements. Platforms for value stream management give the data needed to support the conversations that will lead to this cultural and behavioral shift.

In this series’s third and final part, we will cover more types of experiments and DevOps best practices for achieving change and driving results.
September 22, 2022
Building a Platform for DevOps Evolution, Part One

Winston Churchill once said, “To improve is to change; to be perfect is to change often.” While nearly everyone will confess that change is necessary to grow and succeed, we all know it’s much harder, especially when human behavior is involved. Professionals who work in DevOps environments know this all too well.

Adoption of DevOps principles and behaviors should be viewed as a gradual process. Many organizations make false starts or attempt to implement a change only to see it fail. Sometimes this is due to changes in leadership, market conditions or technology, but then organizations catch their second wind, build momentum and reaccelerate their evolution.

DevOps can Change

With a shared vision, mutual goals and hard work, DevOps can change to keep up with user demands. DevOps professionals will always confront two potent barriers: Cultural and process debt and technical debt.

Cultural and process debt occurs when leaders refuse to delegate authority, distrust their teams and lack a value stream approach to work. Technical debt develops when monolithic systems are tightly coupled, automation is delayed for testing, environments and version control and there is no access to cloud technologies.

Although these obstacles are challenging and complicated, many organizations still meet them head-on. Evolution depends on a habit of making improvements every day through the use of an improvement kata such as PDCA (plan, do, check, act), which involves long-term strategies and objectives and continual experiments between the current and next target states.

An evolving DevOps approach maintains moving pieces across a large organization of hundreds, thousands or even tens of thousands of people who cannot change overnight. Many lack the will and the cognitive capacity to embrace and enact change.

This environment requires release managers to delegate power and transition from centralized commander to an enabler, understanding different value streams have varied skills and requirements. Mark Peters, a cybersecurity specialist and DevOps leader with years of military operations experience, outlined his top release management traits in the context of DevOps ways of working as automated processes, distributed accountability and accelerated delivery.

PDCA Improvement Kata

DevOps proposes that we adopt an improvement kata in our day-to-day efforts as an improvement habit, consisting of four steps that are repeated:

1. Set the long-term vision
2. Understand the current state
3. Identify the next target state
4. PDCA between the states

Experimentation is at the heart of the PDCA cycle. The most practical experiments are empirical, data-driven and date-specific. Using a value stream management platform to give data to support the experiments leads to data-driven discussions on real and measurable success.

In the next part of this series, we will begin exploring different experiments DevOps organizations can conduct to drive change and improve their release management procedures.

August 16, 2022
DevOps Transition: The Cultural Commitments and Challenges

Enterprises can achieve certain levels of success by filling a need in the market, hiring skilled teams and utilizing cutting-edge tools and technologies. However, to set themselves apart and chart a course for long-term success, they need to address the values, beliefs and behaviors that make up organizational culture as carefully as they address business decisions. It’s culture that ultimately informs the decisions made about how a company evolves its concepts from idea to value, so when you want to make a change and grow, that’s where you need to start.

One area where culture is particularly important is the introduction of new practices and processes applied to software engineering and the role that a DevOps strategy plays within organizations. DevOps encourages a mindset of communication, collaboration, integration and automation among software developers and IT operations, so it’s more than just a new set of tools; it’s a cultural shift. This change doesn’t happen immediately or automatically, it requires deliberate steps and actions to be taken, and calculated decisions to be made.

The shift also requires commitment. A recent Gartner report, “New Insights into Success with Agile in Digital Transformation,” found that organizations must be prepared for a commitment of three or more years to ensure success when moving to a more agile development process. Teams with less than a year of experience with a new development process show a success rate of 34 percent, while that rate jumps to 64 percent with between one and three years of experience. After three years of agile experience, the success rate skyrockets to 81 percent.

New Technologies Accelerate Momentum, not Create It

Implicit in the decision to move to DevOps is the desire to develop faster. To do that, organizations often begin the process by saying to themselves, “We need to automate things.” They buy a variety of testing automation tools, production delivery tools and anything they think will make them more agile. However, if they don’t take steps to prepare the people in the organization, things can get worse, not better, because the new elements can create isolated teams centered around “campfires” where separate teams use the various new automation tools individually.

When teams are undergoing transformation, we often see them go through a roller coaster of up-and-down results. This was illustrated in the DORA report, “Accelerate: State of DevOps 2018: Strategies for a New Economy.” When the transformation begins, teams quickly identify wins and automation helps lower performers improve, creating an uptick in overall performance. However, as the process moves along, automation increases test requirements, which are still dealt with manually, and a pile of technical debt blocks progress. At the same time, the mounting technical debt and increased complexities cause additional manual controls and layers of process around changes that, more often than not, slows work. Teams generally don’t see an upswing in results until putting in relentless efforts toward improvement, leveraging expertise and learning from their environments, which finally results in a jump in productivity.

This is a moment when organizations need to lead with their brains, not their pocketbooks, and stay committed to the process. To minimize the risk of failure, they must address the people and the process with the same rigor and commitment they address the technology. In fact, evolving the culture and process associated with how people work is the biggest challenge in adopting DevOps. People resist change and no amount of technology makes change easier.

Any initiative that begins with just buying software or technology to make the organization agile is a mistake. The best way for a business of any size to launch a DevOps initiative is to get people comfortable with the vernacular and the process before changing anything. I’ve seen organizations go so far as to change the name of existing processes just to get teams thinking differently before the real change is made.

Putting in Tools Before a Plan is Like the Cart Before the Horse

After seeding the terminology and idea of change, the next step has to be tactically thinking through how the organization will evolve. It’s often thought that tools will make an immediate impact, but not enough planning is done to integrate the tools before being taken out of the box. Basically, they end up like a bunch of kindergartners at recess: Everyone has their own toy to play with but they do it alone and no one works together.

Before putting in new tools and changing processes, there needs to be understanding around where the gaps and inefficiencies are so KPIs and metrics can be mapped back to tell where progress is being made. You can’t manage what you can’t see, so you have to develop visibility into the process to understand where those inefficiencies currently are. When that is achieved, remaining constantly vigilant to understand the constraints that exist while establishing new processes and using the new DevOps tools will allow you to continuously improve.

It needs to be known that the reward is worth the risk. Making the move to DevOps can’t be thought of as a decision or a choice, it’s an imperative for organizations that want to remain competitive. Every organization needs to become a software company and to do that they need to move to DevOps and make continuous improvement part of their DNA. Organizations can’t fear the move, they just need to be diligent with the transition, monitoring and follow up when adopting DevOps.

— Bob Davis

November 27, 2018
The Struggles DevOps Administrators are Up Against

DevOps has quickly gone from being a niche concept to a widespread practice that is gaining traction rapidly in both large and small organizations. As it becomes a key part of many development teams, administrators must work to integrate DevOps techniques, tools and practices into a wide range of teams, from operations to development, support and management.

The seamless completion of this implementation, together with the overall change in methodology and culture from top to bottom, provides a smoother workflow. The issue, however, is that admins are being asked to do more with less. DevOps administrators from large companies are struggling to keep pace, partly due to the newness of the practices, and in part out of a need to provide a competitive advantage for the business. Additionally, they find themselves in the middle of many different cultural pressures. All of this is leading to problems in fully implementing and reaping the benefits of a fine-tuned DevOps machine.

That said, working through the implementation of a DevOps techniques will be worth it in the end and to make the transition easier, here are some specific pitfalls that administrators should try to avoid.

Reliance on DevOps Technology

One burden that is often faced by DevOps admins is an organization’s over-reliance on technology. In theory, technology is the easy part, but too often organizations become enamored with the newest shiny automation tool and find themselves in a cul-de-sac of isolation, forgetting to bring the rest of their organization with them.

The wave of digital transformation that many organizations are experiencing can present additional difficulties as new technology is being implemented throughout different business units and administrators find themselves in high demand.

Automation is awesome, but it isn’t the answer—it’s part of the recipe. The solution requires attention to processes and culture. According to a Gartner study, 73 percent of what organizations should focus on consists of attention to culture and communication, information exchanges and building operations process and organizational consistency. Only 27 percent should be on tooling.

As the growing emphasis on DevOps places an added burden on infrastructure teams already struggling to cope with the digital transformation in their organizations, unification will be key to success. Ignoring the importance of creating a harmonious culture that embraces DevOps and processes that foster the methodology can be costly. Ensuring that tools are utilized in a smart way alongside people and processes will ultimately lead to more success.

Building Collaboration Between Dev and Ops

DevOps trust and transparency is at the core of release management. This level of collaboration and trust ensures that both sides of the equation agree when it comes time to ship software.

Both sides in Dev and Ops frequently misunderstand the other’s motivations around production control and release management. Stability is king for Ops, and they would love nothing more than preserving uptime in each release, even if it means a blockage of imperative software updates. Dev, on the other hand, wants to move quickly, and they often make assumptions about what is required to support software in production. To help unite the two requires administrators have the ability to fill in the information gap that exists between Dev and Ops.

In a way, DevOps admins work as the therapist for a severed relationship between Dev and Ops. Bridging the divide between these two departments can, at times, feel like counseling an organization to communicate better. But doing so through integration and collaboration is a surefire way to make life easier for everyone involved. DevOps doesn’t force people to become who they’re not; instead, it promotes cooperation, tolerance and shared responsibilities between the teams.

Lack of Transparency

What many administrators also lack is a way to clearly understand, prioritize and utilize the data they have. From an agile practice perspective, that level of visibility should be established early on in a data reporting structure—project, initiative, etc.—and with the proper software solution. A solution that combines application release, environment, quality and deployment data across the portfolio to drive continuous improvement of software delivery.

Being able to utilize this data makes a world of difference for delivery teams to be able to quantify, correlate and predict outcomes to help solve problems and make better decisions. The most basic, yet beneficial, insight you can have when making these decisions for an organization is to have the transparency to know that what you are doing is working.

Creating a Culture vs. Working in Silos

Application release delays costing hundreds of millions of dollars can be caused by poor management of complex enterprise software releases. A big part of the issue is that the very nature of agile and DevOps is to work in smaller teams, which can naturally create silos. For a given release, there can be many teams and, in some cases such as test environments, teams from different projects can collide due to lack of oversight. This complexity presents significant management problems and, without insights across the entire process, release trains are bound to crash.

A culture of working in small teams on a single release lends itself to working in silos, and that isolationism often conspires against organizationwide success. This construct needs to be broken, as it produces an additional level of complexity and unification that will be a key point of emphasis for organizations in their road to digital transformation success.

This unification will be especially important in large organizations that see frequent changes to their software release cycles, sometimes daily. It is imperative for these organizations to avoid silos and favor better communication with pronounced team roles, especially with the added confusion that the rapidly evolving delivery cycle can bring.

Creating a cohesive and harmonious DevOps culture will benefit everyone in an organization in the long run. To be successful at this, though, it’s imperative that DevOps admins are given the rein to institute the change that they see fit. Admins must educate themselves on the latest technology, learn how to use and monitor it and bring in the rest of the teams so everyone can work together. But the burdens they are up against need to be dealt with on an organizational level from the top down, creating the ultimate DevOps working environment for all.

— Bob Davis

August 22, 2018
DevOps Culture through the Prism of Maslow’s Hierarchy of Needs

Abraham Maslow proposed in 1943 that healthy human beings have a number of needs arranged in a hierarchy, which he constructed in the form of a pyramid. The theory suggests that certain needs, such as physiological and safety needs, are more primitive or basic than social or ego needs. The premise is that higher-level needs are addressed only once lower, more basic needs have been met.

By FireflySixtySeven – Own work using Inkscape, based on Maslow’s paper, A Theory of Human Motivation., CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=36551248

A similar path can be uncovered when we map out the evolution of technologies. As technologies fulfill their intended roles, developers pursue greater achievements and solutions to more complex problems. It’s a pattern that exists at every new stage of technological growth as we are driven to satisfy loftier motivations. It’s also happening now in organizational structures as our work becomes more tied to the capabilities of technology.

This progression deserves particular examination in the role that DevOps strategy plays as organizations grow and mature. DevOps encourages a mindset of communication, collaboration, integration and automation among software developers and IT operations, but it can also be swept up by a motivation to go beyond the scope of its basic needs and seek constant betterment. Through this lens, it’s helpful to look at how DevOps culture strives to keep up with the need for a more holistic approach to the end-to-end software delivery life cycle (SDLC), and how it will continue to mature as needs change and evolve.

Physiological Needs

Maslow begins at the base of the pyramid with basic needs. For humans, these needs are physiological, the physical requirements for survival: air, water, food, shelter, etc. To make the analogy in technology, this is the realm of manual processes, where operators flick a switch, crank a handle or spin a wheel. It’s the basic functions of machinery.

The base of the DevOps pyramid is the notion that I have to use software at all, much less manipulate it to fit my business objectives. Not until that basic need is recognized can further development begin to take shape. Initially, Dev teams built applications and passed them to QA teams to test them. The production environment in these early stages got the job done, but at a basic level it lacked cohesion. Teams could harbor mistrust and production environments could become destabilized by poor communication or diverging points of view. Like a simple machine, tasks were performed separately and passed from team to team only after the basics were completed.

Recognizing the need for software was the first step, in fact it’s a basic requirement for most of today’s businesses and organizations. To make progress and mature an organization must take the next step toward addressing more complex problems and make choices that promote structural growth and organizational progress.

Safety Needs

Moving up Maslow’s pyramid, human motivation addresses concerns for safety and well-being as choices of preference are made and a degree of order is sought. Organizationally speaking, burgeoning DevOps teams seek greater usefulness and functionality. It’s not enough for teams to perform a basic function, they need to adapt and find a way to get beyond a lack of cohesion. Organizations need a level of comfort (compared to Maslow’s concept of safety) in knowing that the development and operations teams can work together and have the best intentions for the project. This stage promotes an upgraded approach to software delivery and a reduction in production issues.

Once the realization that software is a basic need for companies and organizations, their sights turn to how best to make it happen across multiple teams with a common goal. And, if a level of internal stability and order can be achieved, the organization can begin to look beyond itself to develop outside relationships in the manner that Maslow describes in the third layer of his pyramid: belonging.

Social Belonging

In the third stage of Maslow’s pyramid, after the physiological and safety needs are met, human motivation shifts to interpersonal relationships and feelings of belonging. In terms of an organization that services customers or constituencies, once internal structures are in place, greater resources can be allocated to outward pursuits. This stage in the business mindset is when the focus on outcomes shifts from being vendor-focused and satisfying internal needs to the servicing the customer. They ask the question: What answers or solutions does the customer want from me? Just like in Maslow’s acceptance stage when humans consider what it takes to be a friend or to find love, organizations try to answer what it takes to provide customers what they need.

In this stage, the automation of DevOps contributes as teams integrate into a structured or organized community that can provide those answers. Teams begin working together and develop the early stages of a feedback loop and full visibility into the SDLC. The workflow takes on an increasingly automated role as a cultural shift toward full lifecycle ownership occurs and teams take responsibility throughout the deployment life cycle, not just in stages.

Esteem

If organizations succeed in putting customers first, they can take the next step to reacting to the customer’s needs at an accelerated pace. In Maslow’s pyramid, the next stage, commonly referred to as the esteem or status level, is divided into two elements: the internal and the external.

Maslow classified these two categories as: (1) esteem for oneself through dignity, achievement, mastery and independence, and (2) the desire for reputation or respect from others. To achieve everything that organizations want to for the customer, internally they must improve how the organization functions, and externally they must demonstrate to the customer that they are listening and the goal is to respond to the customer’s needs directly.

Internally, the organization requires its teams to communicate, collaborate and integrate better, and it puts pressure on its production activity to be responsive not just prolific. If this is achieved, it not only demonstrates that an organization is listening to the customer, but also that it can provide solutions at a cadence that makes the customer more successful.

Self-actualization

The final stage of Maslow’s hierarchy is self-actualization: the point of realizing potential and being the most that one can be. In any organizational structure, this may not be an end goal, because striving to be better should always be a core tenet. However, it might be argued that when the brain trust at the top—the CEO, the CIO, the board, etc.—are fully aware that the components of their organizations are working together and of what they are achieving, a level of corporate-actualization can be claimed.

Far from being the end, this is when thoughtful soul-searching should occur by asking the essential question: Is it all working? This is the most important question an organization can ask itself with regularity, and doing so is a sign of great maturity.

You can recognize your basic needs; build structure into your organization; service your customers, listen to their needs, respond promptly and make them more successful; and recognize internally what make your organization successful. But if you don’t consistently and rigorously ask if it’s all working, you risk slipping backward.

Once an organization reaches a mature level, how does it maintain or even enhance its hard-earned growth? The best thing it can do is to consistently define itself with the use of qualified actualization tools and reporting engines. True self-actualization and corporate-actualization require tools that can quantify how mature an organization is and where it stands by comparing how it’s doing against its competitors. Organizations need tools that can correlate data against a variety of peers in the vertical or in common geography, etc., and provide comfort in knowing what works and that they are leading the way.

— Bob Davis

May 18, 2018