devopsy.kinsta.cloud

Tag: agile

  • Principles Systems and Tools – What Shingo Can Teach Us About DevOps

    Principles Systems and Tools – What Shingo Can Teach Us About DevOps

    In my last post, The Power of a Game: What I Learned from Facilitating the Phoenix Project Simulation, we explored how DevOps and Agile practices are built on underlying Lean principles that go back decades to a time before the Toyota Production System. When you look into the history of Lean, it is almost impossible not to encounter the great thinker Shigeo Shingo. Shingo worked closely with Taiichi Ohno (the father of the Toyota Production System) to advance Lean thinking while building the Lean production system at Toyota.

    As someone who attempted to apply Lean to IT long before DevOps came into existence, I have always said DevOps is about creating an environment where it is safe to look deeply, question what is happening, think crazy things, experiment and learn based on outcomes. Ten years ago, I was invited by The Shingo Institute to serve as a teacher, coach and assessor. I had recently been awarded the Shingo prize for my book, Lean IT – Enabling and Sustaining Your Lean Transformation. What I learned from Shigeo Shingo’s writings heavily deepened my understanding of the role of Lean and flow in IT. These ideas are reflected in the Lean Enterprise Pyramid below and it’s amazing how effective they are with Agile and DevOps.

    Principles and IT Excellence

    A principle is a timeless universal truth that controls outcomes whether you believe in it or now. Think of it as a fundamental truth or proposition that serves as the foundation for a system of beliefs and behaviors. Principles govern consequences and outcomes–both positive and negative. “Create customer value” is a core principle in business success. If you don’t believe in customer value, or don’t deliver it, your prognosis is pretty bleak.

    In Lean IT, I introduced some key principles that drive IT success:

    Let’s begin with a working definition of DevOps—a mindset, tool set and set of practices that combine software development and IT operations to shorten the development/operations life cycle to deliver small iterative features, fixes and updates often and in close alignment with business objectives and value creation.

    Consider the Lean principles pyramid from a DevOps perspective:

    • Foundation principles build an intentional culture of trust, respect, focus and learning.
    • Behavior is how we show up and engage.
    • Perspective is a fresh new way of looking at IT work.
    • Flow is the outcome of effective DevOps in action.
    • Capstone of the pyramid fosters a regenerative culture of trust, safety, experimentation, collaboration and innovation.
    • Outcomes are higher quality IT solutions and services, in less time, with less cost with healthier teams.

    For DevOps, Agile and High-Velocity IT of any flavor to be successful, each of these principles must be addressed. You can’t address them all at once, so I suggest beginning at the base while taking steps to enhance flow. If you focus on these two elements, respect for people and flow, you will always be progressing in the right direction.

    Systems and IT Excellence

    Shingo defined a system as a collection of tools and tasks highly integrated to accomplish a specific outcome. The Shingo Model captures the important relationship between principles, systems, tools and results. It also supports three key insights which clarify how to achieve excellence in IT (or any organization). IT needs to get good and then continuously get better with its systems, tools and results (no kidding). Understanding how these elements are informed and reinforced by principles is essential to making this happen.

    The Insights

    1. Ideal Results Require Ideal Behavior
    2. Beliefs and Systems Drive Behavior
    3. Principles Inform Ideal Behavior

    For DevOps and Agile, the Shingo Model provides a logical sequence for change. Put another way, what are the key questions we need to answer?

    1. What do the ideal behaviors of DevOps look like?
    2. What beliefs do our people have today and what beliefs do they need to embrace DevOps? What systems (think: work systems, procedures, policies and technology) do we have in place today, what behaviors do they drive and what needs to change?
    3. How can our understanding of lean principles clarify our understanding of question #1?

    Take a moment and reflect on the role of principles and systems in your transformation. The more you ground your approach in the stability of principles, the greater your chances of success. If you or your organization does not truly believe in these principles, a conversation around which principles you do believe in would be a good place to start.

    — Mike Orzen

  • Why Did DevOps Become So Popular? 3 Theories

    Why Did DevOps Become So Popular? 3 Theories

    DevOps has now been a thing for a solid decade. As we look back at the history of DevOps, it’s worth asking: Where did DevOps come from, and why did it take off when it did?

    In honor of the 10-year anniversary of DevOps, I’d like to explore these questions.

    A Brief History of DevOps

    The very basic history of DevOps goes like this: First, there was agile, an idea exemplified in the Agile Manifesto that gained prominence among software developers in the early 2000s.

    Then, in 2008, Patrick Debois, a developer from Belgium, conceived of DevOps as a way to improve upon the agile concept by encouraging developers to work more closely with ITOps teams. His ideas began spreading in 2009. By the early 2010s, they had gained a sizable following and the rest is history.

    That, again, is a really basic history of DevOps. For more detail, I recommend this DevOps history post by Steve Mezak.

    Why DevOps? Why Now?

    The narrative of how DevOps originated is one thing. Explaining why DevOps became so popular in a relatively short span of time is another.

    As I see it, there are three possible theories to explain the rise of DevOps over the past decade. The first is to argue that Debois simply happened to have some very good ideas, and he promoted them well. You could think of this as the great man theory of DevOps’ origins. This seems to me, however, to be an unlikely explanation. It’s very rare for highly influential ideas to be born from just a single mind, or to spread through the singular efforts of one person.

    The second possible explanation for why DevOps became popular is it is simply an evolved form of agile. This is like arguing that World War II was simply a continuation of sorts of World War I, and therefore the origins of World War II are ultimately the same as those of World War I. From this viewpoint, the history of DevOps is really the history of agile, and so DevOps itself doesn’t have a unique history at all. I’d find this argument compelling if I believed DevOps is really just agile by another name, but I don’t. DevOps is broader, and has a much stronger implementational and practical element.

    The third theory—and the one I find most persuasive—is DevOps was born out of a particular moment in the history of computing and IT. That moment, which started in the mid-2000s, was marked by several key features that helped give rise to DevOps, including:

    • The widespread adoption, starting circa 2007, of cloud computing.
    • The success of the open source software movement around 2010 in making open source the go-to licensing strategy for many popular software projects and in making collaboration around code easier.
    • The rise beginning around 2010 of microservices and containers, which made applications much more complex to design, deploy and manage.
    • The explosive popularity of mobile computing starting with the launch of the iPhone in 2007. The mobile revolution meant organizations increasingly had to deliver software for multiple target environments (PCs, Android phones and iOS, for example) at the same time, necessitating branched delivery pipelines.
    • The demand for ever-increasing speed in software delivery as a way to gain an edge in software markets which, by the late 2000s, had grown more competitive than ever.

    When you put these changes together, you end up with a need to manage much more complex, fast-moving software delivery pipelines. Powered by cloud infrastructure and microservices architectures, the software market starting around 2010 required more responsive, better coordinated management by IT teams. DevOps provided an answer by increasing efficiency, decreasing response times and making software delivery faster.

    I don’t think DevOps would have taken off if the idea had emerged in the 1990s, when there was much less demand for fast software delivery, most applications ran locally and codebases were relatively simple.

    Nor am I sure DevOps would prove as popular if it were first introduced today. At present, we have a litany of management tools (such as Ansible and Kubernetes, to name a couple of examples) that didn’t exist 10 years ago. Those tools make it easier to handle all of the complexity of modern software environments even if you don’t have a management strategy rooted in DevOps culture.

    To put it simply, DevOps resulted from the combination of several specific developments in the world of IT between the mid- and late-2000s. Debois may certainly deserve credit for helping to spread the idea, but it would not have spread so quickly if it had not emerged at that particular time.

    — Chris Tozzi

  • Digital Transformation Drives C-Suite Discord

    Digital Transformation Drives C-Suite Discord

    In the sport of rowing, each crew member has to row in synchrony or the boat risks undesired turns and even potential tipping. It takes practice, communication and discipline for each crew member to raise their oar to the appropriate height and apply the appropriate power. Obviously, those crews who work best together are more likely to win.

    When it comes to winning at digital transformation, the same is true for the executive crew or C-suite. However, the drive to  digitally transform enterprises rapidly puts a strain across C-suite relationships.

    That’s the main takeaway from a new report published by the Financial Times Focus and business management software provider Apptio, “Disruption in the C-suite.” The report is based on a survey of 555 senior executives, about half in CxO roles and from Australia, Denmark, France, Germany, Italy, Japan, the Netherlands, Norway, Spain, Sweden, the UK and the U.S.

    The survey found considerable uncertainty regarding the IT department’s ability to be effective. In fact, 51% of respondents said they have concerns about IT’s ability to govern cloud adoption. And, surprisingly, less than one-fifth of companies using agile have defined performance indicators they use to track their results.

    Further, 47% of leaders surveyed agreed digital transformation has blurred work responsibilities throughout the C-suite. And the same number of respondents admitted digital transformation has exacerbated strategic differences among finance and IT leaders. In fact, 71% of finance leaders said the IT function needs greater influencing skills to deliver the change their business requires.

    According to the survey, while the CFO was the least deeply aligned member of the C-suite crew with the CIO, the lack of deep alignment with the CIO’s office and other C-level executives was shown to be disturbingly common. Only 56% of CEOs reported being deeply aligned with the CIO, the survey found.

    There are signs of hope. Eighty-seven percent of respondents in the U.S. said digital transformation has increased collaboration across C-level executives when it came to developing new products and services.

    Still, 63% of those surveyed said the IT function needs greater influencing skills to deliver the change their business requires.

    That lack of influencing skills—and lack of defined metrics to measure progress—has taken its toll. About one-third of CFOs said there is a lack of consensus among CxOs when it comes to how to measure the impact of new technologies.

    It’s interesting, 56% of respondents said they shift their IT strategy continuously as their technology leaders receive input from the business and customers. That’s hopefully advantageous, but with so much digital transformation discord among technology and business leaders within organizations, it’s a coin flip whether that’s good stewardship or blind steering.

    It’s absolutely vital that organizations get their digital transformation efforts right, and to also be able to do so successfully at the necessary speed. As we covered in the recent story, “Digital Transformation Vital for Business Survival,” 78% of organizations believe their company’s primary industry is evolving rapidly and digital transformation is necessary to survive the changes underway.

    Those businesses cited enhanced customer experience and customer relationship management, the ability to maintain competitiveness, enabling new ways of doing business, increased productivity, cost savings, increased revenue generation and enhanced product development.

    Few would doubt the accuracy of that assessment, nor that the entire C-suite crew must be in synchrony, understand each other’s roles and objectives and know where the entire team is heading. Otherwise, the entire effort is likely to veer off course.

    — George V. Hulme

  • Value Stream Optimization: Putting Business in the Driver’s Seat

    Value Stream Optimization: Putting Business in the Driver’s Seat

    How value stream can be a major driver of DevOps success in today’s software-only world

    Even as DevOps reaches a pinnacle in acceptance (as of 2018, only three percent of firms had not adopted DevOps and had no plans to do so), many organizations are still struggling to get high-quality software out the door. DevOps has been promoted by many experts and consultants as a solution to the cumbersome inefficiencies that were common with waterfall methods. However, as is common for many operational activities, applying a new approach or technology doesn’t automatically resolve entrenched problems.

    Nowhere is this more obvious than in value stream management (VSM). This approach, which focuses on removing the waste from processes to increase their efficiency, sounds like a panacea for resolving software delivery challenges. In practice, it often falls short of its goals.

    Why is this? There is nothing wrong with the concept of the value stream. The idea of breaking down a complex process into its components and optimizing each one of them, thereby optimizing the chain, is a long-recognized pillar of lean production. In my view, the problem isn’t with VSM or even DevOps.

    The problem is that in moving from waterfall to DevOps and/or agile, many organizations figuratively “throw the baby out with the bathwater.” They are uncertain how to adjust for the extreme acceleration that occurs when teams move from rudimentary software processes to modern ones, so they either jettison important elements of software engineering or adapt them incompletely.

    Why Continuity Matters

    By way of example, consider measurements and metrics. The traceability and proactive remediation enabled by measurements and metrics are two of the most important aspects of software development. However, it’s not just the development and test metrics that matter, but also the business metrics—metrics of quality, user acceptance, security and ease of use. Since the dawn of software development, those two groups of data have been inextricably linked, but some modern software teams appear to have forgotten that lesson.

    No amount of process improvement metrics can ensure a software release is secure or that users will like it. Yet, if a major software upgrade is flubbed, it can impact operations so negatively that sales momentum stalls. (If the company sells the software it develops, the ripple effect is even stronger.) Even more disconcerting, after a major data breach, a firm can find itself out of business.

    Don’t get me wrong. I am not suggesting that flow and other DevOps metrics don’t matter, because they do—very much. I am suggesting that many teams and their development managers have become too divorced from the business metrics, which can have staggering negative consequences.

    From There to Here

    In the days of waterfall, development artifacts, defects and other data were warehoused in an offsite location, and after each release (which might take days, weeks or even months), teams would engage in several days of data extraction. Then, a separate team would attempt to map the development artifacts to the defects and enable traceability. After hours or days of organization and cross-mapping in Excel spreadsheets, reports would be generated for management and the C-suite. Once stakeholders and management completed their evaluation, the team would be cleared to make adjustments before the next release cycle.

    Now, our development paradigm has changed so dramatically that many teams aren’t giving metrics their due. That’s an issue with defect metrics that could impact the user experience or software reliability. It’s a serious, potentially game-ending problem when the metrics are related to security.

    Completing the Value Stream

    To create and administer the complete value stream, software teams must be supported in three areas beyond the traditional DevOps framework where iterative development activities are the primary driver of traceability measurements. With these elements in place, management can map their software teams’ efforts to the business metrics that are the final measurement of success:

    • Organizations must have a mechanism for integrating development and testing tools so that defect measurements can flow back to the development team in real-time—matching the speed of DevOps.
    • Critical software activities that still take place outside of development and testing processes, such as generation and approval of change orders, must also be monitored for inertia, with approvers being held accountable for holding up paperwork that delays releases.
    • Security defects must be given parity with, and preferably priority over, all other code defects. All defects that impact the user side, such as functionality and launch speed, are important business metrics. But security must be paramount.

    If you are a development team member reading this, or even a development manager, you may think business metrics and the related activities that happen outside your purview, don’t apply to you. I urge you to think again. Software has become a critical pillar of many companies’ operational success, and that connection is only growing stronger by the day. Everyone involved in software development, from the developer to the VP of Operations and on to the CEO, must view the entire process through the lens of business.

    — Joe Schulz

  • Are Your Development Processes Truly Agile?

    Are Your Development Processes Truly Agile?

    Many organizations are making a move to agile-powered environments, but agile development is more than a name. How do you determine if your organization is truly agile? The evaluation starts with the Agile Manifesto. If you’re following the principles laid out in the manifesto, that’s a good sign that you’re on an agile journey.

    For those not familiar with the manifesto, however, let’s review its points quickly. The Agile Manifesto’s core is simple—the primary points of the manifesto include the following:

    Individuals and interactions take precedence over processes and tools. This means the organization and its leaders place more value on people than they do on processes. People respond to business needs and they drive the development process within the organization.

    Working software takes precedence over comprehensive documentation. This is not about not providing any documentation at all, but you should streamline documentation. Any documentation that does exist should be necessary only for current and future development.

    Customer collaboration over contract negotiation. Engaging customers throughout the development process makes it easier for your people to meet the needs of your customers. Doing so means all business case features are clear and useful to customers.

    Responding to change over following a plan. This means change is welcome throughout the project, each project and allows for priorities to shift over time versus following a rigid or scripted development plan. 

    The Agile Manifesto 12 Steps

    Essentially, the core values of the Agile Manifesto can be fully detailed in 12 steps, or principles. These principles support the key points of the manifesto and provide additional detail as to how the practices can be followed in an agile environment. Again, for those who currently follow the Agile Manifesto, the following may be redundant. For those new to the concept, these steps provide clarification and direction to drive agile processes overall.

    • Self-organizing teams encourage great architectures, requirements and designs.
    • Regular reflections on how to become more effective.
    • Frequent delivery of working software.
    • Simplicity.
    • Support and trust for those involved in the process.
    • Enable face-to-face interactions.
    • Working software is the primary measure of progress.
    • Agile processes to support a consistent development pace.
    • Attention to technical detail and design enhances agility.
    • Collaboration between the business stakeholders and developers throughout the project.
    • Customer satisfaction through early and continuous software delivery.
    • Accommodate changing requirements throughout the development process.

    Based on these principles, there are three points—the final three listed here—that we often find the most compelling. A colleague of mine, Tobias Spöcker, scrum master and QA engineer, says these points naturally float to the top for an organization such as TOPdesk.

    The reasons for this are fairly obvious. We create products that are sold to thousands of customers around the world. To pinpoint any development problems, we look to the scrum process described by the scrum guide, with an emphasis on the last parts of the process. For example, the following illustration is edifying for explaining this process:

    Agile scrum process

    Notice in the “product release” phase illustrated above that the team listed here focuses on finished work for the development sprint. This portion of the model should be able to present a product incrementally to the stakeholders, including the customer. Ideally, these increments are released through a continuous delivery model. Here are two principles that further illustrate the point: Collaboration between the business stakeholders and developers throughout the project and customer satisfaction through early and continuous delivery of the software.

    There’s an additional consideration when providing value: Develop what is right; don’t simply develop something right.

    Adding Value

    At TOPdesk, providing the proper value to customers is a major goal for us. To achieve our organizational goals, we consider the impacts on our customers so when we start on a new product development project, we always work with some of our customers so they are involved in the development process, which makes the process and the product better for everyone. Those of our customers that have participated in our development processes usually tell us that they find the experience valuable and that being a part of the process also helps them when they use the solution. These same customers also express satisfaction in our efforts—as partners in the development process—and with us as an organization.

    Direct Feedback is the Key to Better Development

    The immediate and direct feedback we receive also helps to achieve another of our key operational principles: accommodating changing requirements throughout the development process. Through the partnership with our customers during the development phase, we’re able to receive regular, constructive feedback—even criticism—that leads to changes and evolutions in the product map and development stream. This feedback facilitates refinement of the most important processes and features, and moves them to the top of the priority list, leading to a project that’s undertaken with a focus on the most valuable areas so that development capacity (on the part of our organization) is spent the most wisely.

    Agile-Organization

    In Sum

    The agile mindset and the scrum framework are powerful tools if they are used in the right way. When considering following any development principles, don’t go in blindly. It is a good idea to see which parts of the toolbox help your organization be the most agile, efficient and customer-focused, and utilize the processes and principles that work best for your organization.

    Consider direct feedback from customers, which can be implemented immediately into the product scope. Provide simple-to-use customer communications vehicles, including metrics built into the software, surveys and feedback portals. Live demos of the product, when ready, can be extremely effective in helping you collect copious feedback efficiently. However, if your feedback loop is too big—meaning there are too many cooks in your kitchen—the whole process will quickly balloon beyond profitable, resulting in a less agile development process.

    The projects we develop using these principles are the ones that deliver the most value to our customers the most quickly. Including feedback from customers makes the development team more confident in what they are creating, and our customers tell us that they feel their needs and input have been heard, which they greatly appreciate.

    — Nancy Van Elsacker Louisnord

  • 5 DevOps Terms That We All Use, Yet No One Can Define

    5 DevOps Terms That We All Use, Yet No One Can Define

    I’m all for DevOps. Sometimes, however, I have a hard time talking about DevOps. The reason is because many of the key terms related to DevOps lack clear and precise definitions.

    Let me explain …

    DevOps is such a big deal that it has bred its very own lexicon. Search Google for “DevOps terms,” and you’ll find a slew of “dictionaries” and “glossaries” dedicated to defining the various terms that go hand in hand with DevOps.

    Yet despite these resources, some deep ambiguities remain regarding the actual meaning of various terms that you hear frequently in DevOps-centric conversations. Let me walk through some of the biggest.

    Continuous

    DevOps is all about continuous this and continuous that. Continuous integration. Continuous delivery. Continuous deployment.

    To me, the problem with the word “continuous” in this context is that it lacks an exact meaning. No one is integrating, delivering or deploying software in a literally continuous fashion. That would mean having zero delays between different releases, which is not the case. Even organizations that do CI/CD really well manage deliveries and deployments of only once per day or so.

    Maybe instead of “continuous,” we should just say something like “fast.” Because that’s what “continuous” really means in most cases.

    CI/CD

    Speaking of CI/CD, that’s another term that is ripe with ambiguity. It’s clear enough what “CI” stands for (continuous integration). But does the “CD” stand for “continuous delivery,” “continuous deployment” or both? Or does it not matter?

    Moreover, does a your application delivery process qualify as a CI/CD pipeline simply by including a continuous integration server (to do the CI) and some kind of automated deployment tool (to cover the CD side of things)? Or does a CI/CD pipeline also necessarily include build and test automation? I think most people would say the latter, but the term itself does not imply that.

    Microservices

    Microservices are all the rage in the era of DevOps. Depending on how you define a microservice, however, they have been all the rage for decades, long before anyone was talking about DevOps.

    The idea of distributing applications into multiple services is not new. That’s what SOA was all about in the 2000s. Going back further, there were microkernels, a concept popular in the 1980s. In microkernel architectures, the operating system kernel runs as a set of distinct services, rather than as a monolith.

    The question, then, is what makes a microservice, or a microservices architecture, different from other types of distributed, service-based architectures. Maybe it’s the size of the service, but there’s no official definition of what distinguishes a microservice from a non-microservice.

    Agile

    It is often said that DevOps builds upon the principles of agile software development. Those principles were defined, at a high level, in the early 2000s and spelled out in the “Manifesto for Agile Software Development.”

    It’s clear enough what agile means, but I’ve long struggled to piece together the exact relationship between agile and DevOps. Is DevOps “agile applied beyond the software team,” as one DevOps company puts it? Or are agile and DevOps distinct practices that just happen to have similar goals?

    It’s hard to say, because there is little consensus about what exactly “agile” means in the context of DevOps.

    DevOps

    Last but not least is “DevOps” itself. Different people have defined DevOps in very different ways. AWS says it is “the combination of cultural philosophies, practices, and tools.” Other definitions focus on philosophy, and leave tools and specific practices mostly out of the picture.

    Blurring the definition of DevOps further is the fact that virtually every tech company in the world calls itself a DevOps company today, with little concern for how, specifically, its products or services meet that nomenclature.

    What does DevOps really mean? I don’t think we’ll ever have a singular answer to that question.

    Conclusion

    In fact, I don’t think there will ever be clear definitions for any of the terms I’ve listed above. I’m not suggesting there should be.

    But I do think it would be helpful to the DevOps community as a whole to recognize the ambiguity in many of these terms. We use these words as if everyone knows what they mean, but the reality is that that is often not the case. If you want to use terms like “continuous” or “agile,” do so, but remember to clarify what those terms mean to you, because you can’t assume that they mean the same thing to others.

    — Chris Tozzi

  • Agile vs. ITIL: Two Divergent Paths to DevOps?

    Agile vs. ITIL: Two Divergent Paths to DevOps?

    Truth be told, there are dozens of different frameworks in existence and all have their unique perspectives on how a project is managed and executed. Frameworks such as Agile, ITIL, lean, cobit, six sigma and many others were built on an ideology of following a path defined by best practices.

    With DevOps, two of those frameworks, Agile and ITIL, have become the basis for arguments on how to do DevOps correctly. Yet, until recently those frameworks were very different in the paths offered to bring about project success. With the announcement of ITIL V4, things have changed a little: ITIL is becoming a little more like Agile, adding to the confusion of what path makes the most sense in the world of DevOps.

    A Quick Glance Backwards

    The foundations of DevOps were built upon the ruins of non-iterative application release cycles. Or look at it this way: DevOps is all about not taking a plunge over the waterfall. While those observations are little more than metaphors for something that rings true, DevOps has brought a new paradigm to software development and deployment. A new paradigm that comes with its own recommended practices.

    Yet, current DevOps practices—or let’s call them rules—can come up short in the modern enterprise, meaning that other management best practices—such as those illustrated in the Agile and ITIL methodologies—are borrowed to drive DevOps further forward. That said, the common DevOps ideology is built around a few foundational practices that define the framework of DevOps. The primary goal here is to bring application development and operations together as a functional team. Some of the generic best practices include:

    • Implement collaboration and teamwork tools to improve communications as well as improving the ability to share knowledge, ideas and automatically document interactions. Simply put, incorporating teamwork with an audit trail goes a long way toward bringing order to DevOps.
    • Incorporate tools to capture requests. One of the most important capabilities delivered from DevOps is the ability to adapt to change. Traditional application development frameworks are usually based on waterfall processes, where everything is defined at the outset and then an application is eventually delivered. Waterfall ideologies offer no leeway to adapt to change in the middle of a development cycle. DevOps eschews those limitations by incorporating the understanding that change is constant. However, requests for changes must be recorded, tracked and reported upon.
    • Adopt a framework ideology that can match workloads to team capacity. In other words, team leaders and managers need to have a clear view of what work is in progress and the context of that work as it applies to a team’s abilities.
    • Use systems to log metrics on automated and manual processes, which will give insight into productivity, as well as identifying where improvements can made.
    • Implement test automation. The idea here is to automate the testing of code using test data in a sandbox environment. Code that passes the tests can be forwarded further along the DevOps process, while code that fails is automatically returned to developers for correction.
    • Use a continuous feedback system in which a chain of communications can be established to keep developers and operators informed of the status of any issues, as well as change requests and the progress of any code through the framework.

    While the above best practices are simplified representations of a full DevOps framework, they do establish that DevOps relies heavily on collaboration and iterative processes to support the ideology of continuous development and deployment. Now, comparing the DevOps ideology to what constitutes either Agile software development practices or the ITIL Framework brings up some additional questions and establishes that Agile and ITIL may very well have different goals. A more simple way of looking at it is to think of Agile as a set of practices to provide faster software delivery, while ITIL is a framework squarely focused on processes and not practices.

    The Path Becomes Muddy

    Therein lies the real dilemma of establishing a path to DevOps using either Agile or ITIL ideologies. ITIL is designed from the perspective of the governance of IT service management and establishes a framework that supports continuous measurement and should deliver improvement of the quality of IT services. Agile is a set of processes designed for software development that meets the needs of customers and cross functional teams.

    Interestingly, Agile and ITIL prove to be complimentary in the world of DevOps, where ITIL can be used to ensure quality and value while Agile brings forth a continuous stream of improvements in applications. Ultimately, with the arrival of ITIL V4, Agile and ITIL will become more aligned with each other and offer benefits to DevOps.

    — Frank Ohlhorst

  • Will ITIL V4 Bring Order to a DevOps World?

    Will ITIL V4 Bring Order to a DevOps World?

    It is difficult to dispute the importance that the ITIL (Information Technology Infrastructure Library) approach has had on the world of ITSM (IT service management). After all, the best practices established by ITIL V3 have come to be the accepted way ITSM providers conduct business—so much so that many other IT processes have come to look to the ITIL framework to bring order and control to complex processes.

    Yet, ideologies such as DevOps have struggled to adapt to the somewhat rigid doctrine of ITIL V3, instead favoring Agile methodologies to meet the rapid deployment of updates and the iterative cycle of software development. Perhaps, the Agile-versus-ITIL differences can be attributed to how life cycles should be managed and how the management of those life cycles impact both development and operations in the IT space. Nevertheless, change is coming to the world of ITIL, which may help to make the framework more suitable for DevOps practices, and perhaps make ITIL and Agile symbiotic in the field.

    The ITIL V4 framework is scheduled to be released in Q1 2019, some two years after it was announced by AXELOS at the Fusion 2017 conference. “ITIL is the most firmly entrenched ITSM framework in the infrastructure of large, complex organizations, with many—if not most—Fortune 500 companies using it,” said Cathy Kirch, president of itSMF USA, the organizer of FUSION 2017 and the leading independent professional forum for IT Service Management professionals in the United States. “Currently, ITIL and complementary practices, such as DevOps, Agile and Lean, are used successfully in conjunction with each other; however the next evolution of ITIL will promote and underpin dynamic collaboration.”

    The More Things Change, the More They Stay the Same

    ITIL V4 doesn’t propose to completely rewrite the ITIL V3 narrative; in fact, the core elements are expected to remain the same. ITIL V4 will simply include new material on integration with additional best practices aimed at mitigating some of the biggest criticisms around ITIL V3, the less-than-ideal relationship with the ITSM community and failure to keep up with recent trends in software development and IT operations.

    In essence, ITIL V4 will address those criticisms by shifting V4 to a community-driven initiative and integrating other practices such as Agile, DevOps and Lean into traditional ITIL best practices. That means ITIL may become complimentary to those other frameworks and begin to embrace other services management best practices in the context of new technologies, such as serverless, containers, microservices and multi-cloud.

    With the DevOps community picking up steam, the Agile Manifesto and Lean development displacing ITIL V3 in some practices, it will be interesting to see what impact ITIL V4 will have on the IT industry as a whole. While it is far from ITIL’s last gasp, ITIL V4 will face significant challenges in a DevOps world that is constantly changing. However, with more than a million ITIL V3 certificate holders worldwide, ITIL V4 will definitely garner a great deal of interest when it hits the streets on Feb. 28.

    — Frank Ohlhorst

  • CXOs Overestimating DevOps and Continuous Testing Maturity

    CXOs Overestimating DevOps and Continuous Testing Maturity

    More work is needed to achieve continuous testing, Forrester report shows

    Forrester recently completed an interesting new research project exploring what separates DevOps leaders from DevOps laggards. After analyzing data collected from 603 global enterprise respondents responsible for their firms’ DevOps strategies, Forrester learned that firms with the most mature DevOps practices do several key things differently:

    1. Allocate proper testing budgets and focus on upgrading their testing skills.
    2. Implement continuous testing to meet the demands of release frequency and support continuous delivery.
    3. Include testers as part of their integrated delivery teams.
    4. Automate end-to-end functional testing.
    5. Shift left testing to earlier in the development life cycle.

    Firms who follow these five best practices experience testing as a way to gain speed and efficiency, rather than a bottleneck to delivering faster.

    This was not terribly surprising. On the one hand, study after study shows that testing is the No. 1 bottleneck to accelerated application delivery. On the other, we’ve seen firsthand how organizations that commit to transforming testing successfully shift from waterfall-style manual testing processes to automated continuous testing integrated into CI/CD pipelines.

    What caught us off guard, however, was CXOs’ highly inflated estimation of their continuous testing, DevOps and Agile maturity.

    According to the report, only 26 percent of non-CXOs reported that their organization was following all five of these best practices—but 57 percent of CXOs believed that they were. According to the report:

    While most Agile and/or DevOps firms have implemented some of these five core testing practices, few have actually implemented all of them — only about a quarter of respondents say they somewhat or completely agree with all five best practice statements.

    This is an important distinction. While many CXOs and high-level decision makers believe they are ahead of the pack, our research shows that only the small fraction of firms that follow these best practices have more advanced Agile and DevOps practices compared to their peers — and their behaviors and attitudes support that assertion.

    Moreover, CXOs also reported much higher levels of Agile and DevOps maturity across the board than non-CXOs did.

    The good news for DevOps professionals is that CXOs are your champions: They believe you’re doing a great job! The bad news is that, at some point, you need to tell them about the gaps that currently exist in the process. You also need to collect and share some of your own data to make the case that actually adopting these five proven best practices will help you increase delivery speed while reducing business risk.

    Detailed Report Findings

    This was just one of the many interesting findings from research project, which was led by Forrester Vice President and Principal Analyst Diego Lo Giudice.

    As a high-level summary, the research uncovered the following:

    • Understanding of business risk is the most important determining factor of DevOps and Agile maturity.
    • Experts focus primarily on contextual metrics (e.g., requirements coverage) while others focus on “counting” metrics (e.g., number of tests).
    • Experts are more likely to measure the user experience across an end-to-end transaction while others rely on application-specific or team-specific metrics.

    Detailed results and analysis are available in the complete report: “Forrester Research: What Separates DevOps + Agile Leaders from Laggards?”

    Report Methodology

    In this study, Forrester conducted an online survey of 603 enterprise organizations in North America, Europe and Asia Pacific to evaluate current software testing practices and metrics tracked during survey development. Survey participants included decision-makers and individual contributors responsible for their organizations’ Agile and/or DevOps development strategies. Questions provided to the participants asked about their firms’ attitudes toward software development automation, risk management and testing practices, as well as the metrics they track and value in the software development life cycle.

     

    — Cynthia Dunlop

  • Is Your Organization ‘Fit for the Future’?

    Is Your Organization ‘Fit for the Future’?

    Takeaways from a recent conference suggest more work is needed in DevOps adoption among organizations

    This was a question explored by CIO Platform Nederland and the DevOps Agile Skills Association (DASA) in a recent event I attended, hosted by Transavia. Many organizations have already embarked or are about to embark upon Agile transformation initiatives, driven by growing business demands. This event focused on sharing experiences, learning from the practical examples (successes and lessons learned) from Transavia and ABN AMRO, and exploring the way forward to successfully create a high-performing IT culture that is “Fit for the Future.”

    The event was well-attended—unsurprising, really, when you remember the wise words from the late Robert Stroud, former Analyst at Forrester: “To unlock the promise of DevOps, CIOs must lead and support a cultural change within their technology management organization, but as any leader knows, changing institutionalized behavior is the toughest of all management challenges. CIOs must recognize when a trend becomes an imperative for survival. DevOps has become this imperative.”

    DevOps and Agile transformations being seen as imperative for business survival. But not just business survival, according to the “2017 State of DevOps” report, which declared, “By 2020, half of the CIOs who have not transformed their teams’ capabilities will be displaced from their organizations’ digital leadership teams.”

    Rik Farenhorst, CIO of Transavia and chairman of the DASA Leadership Forum, presented lessons learned from Transavia. He stressed the importance of changing mindsets and giving equal, if not more attention, to creating desired “behavior” change and not relying too heavily on structure, processes and automation and hoping that the right culture emerges. “People first,” “live your values” and “lead by example” were his key messages, stressing that values must be translated into corresponding desired behaviors as well as unlearning old ways of doing things. Another important discovery was accepting that not all teams are so far and not all teams can mature at the same speed.

    A great example of “lead by example” was the leadership team’s daily standups and visualization of strategic priorities, gaining more situational awareness and shared understanding and commitment.

    Matthijs Dee and Ivo Carton from ABN AMRO presented their journey from Agile to DevOps, also pointing out the fact that teams mature at different speeds and the need to share lessons learned between teams. ABN AMRO adopted a DevOps playbook that teams could use, containing a kind of toolkit for improving team capabilities. Equally, Dee and Carton noted ABN AMRO’s challenges in moving from “command and control” and traditional hierarchic management models to new leadership styles and behaviors. In ABN AMRO’s case, the organization needed to focus on BizDevOps—it’s not all about code, it’s about delivering business value.

    Farenhorst mentioned “values” in his presentation, while Dee and Carton mentioned “principles.” Both of these concepts need translating into team behaviors and leadership behaviors must help empower teams to make this shift.

    My presentation, “DevOps Will Fail … Unless,” I revealed the findings culled from more than 400 global teams having participated in the Phoenix Project DevOps simulation and gave a Top Three set of takeaways.

    The first was the need to focus on BizDevOps—making the shift from “features” to “outcomes” and from a flow of “code-to-commit to a flow of “idea-to-value.” This was also one of the top challenges for organizations at the most recent DevOps Enterprise Summit.

    My second tip mirrored advice from Transavia and ABN AMRO: The need to define, agree and commit to “desired behaviors” and what is effective collaboration (here is an example of what I mean).

    My third tip was embedding continual learning and improvement as core behaviors. DevOps is new and all about experimenting—it is a long journey and needs to be done as iterative improvements. At the same time, the pace of change coming at us is increasing, as is the amount of change. Teams need to be able to respond to change. I also quoted Mike Orzen, author of “Lean IT”: “Improving your work is just as important as doing your work”; as such, impediments and improvements should also be made visible on the backlog of work for teams and leaders must ensure adequate time is given and reserved for work-in-progress, for teams to improve.

    Following the presentations were three break-out sessions: “Value Creation and Management,” “Organizational Design – Agile/DevOps at Scale,” and “Sustainable Culture Change.”

    These are some of the challenges raised in the Culture break-out session I attended as a follow-up to my presentation:

    • An interesting discussion was around “control” and how to empower the management layers and have them empower the teams. As a CIO, this layer between you and the work floor can diffuse your vision by the time it reaches the shop floor and at the same time leave you unaware of what is happening.
    • Management by walking around was suggested, as a CIO being visible on the shop floor and soliciting feedback. Listen to concerns and enter into a dialogue about vision, goals and what it means.
    • Teams often are faced with fear, uncertainty and doubt (FUD) about new ways of working. What will it mean to their job? To their responsibilities? And will they be able to do it? Leaders must remove the FUD and create a safe environment. Safe to take risk, to experiment, to fail, to give feedback. Managers must foster a “no blame” culture.
    • A study by Google (2012) called Project Aristotle, named in tribute to Aristotle’s quote, “The whole is greater than the sum of its parts,” looked into why some teams work and others don’t. One of the key conclusions was that “psychological safety” was the “it” factor: A sense of confidence that the team will not embarrass, reject or punish someone for speaking up.
    • Change team behaviors. We adopt these approaches and don’t see the behaviors we expect. Not everybody wants to change. Leaders need to be able to identify the silent killers, what is under the water, what are drivers for working against. At the same time, teams need to be willing and able to work effectively as a team and need team training (not just individual training). Team training also needs to address soft skills such as communication, feedback and collaboration.
    • Leaders also need new skills in organizational behavior management to be able to identify behavior drivers and how to create sustainable behavior change.
    • Lead by example was named, but what is that? What behaviors are expected from leaders? Sometimes managers are only one step ahead in this new working. DevOps is new, there is no fixed framework; it is about experimenting what is right for your organization. This is all new.
    • Ask yourself, “Why are we doing Agile, DevOps, Lean? Are we adopting these approaches as a goal in themselves? Or are they to solve a specific set of problems or deliver a specific set of results? And how does this relate to strategic goals?” This must then be translated into, “What is in it for ME?” for the various stakeholders (business, IT management, teams, customers). Success can only be achieved end to end, which means all end-to-end stakeholders must understand the “why” if they are to commit to the journey.
    • Build empathy and trust and asking questions to get open feedback—not telling (command and control).
    • Authenticity: To build trust, be empathetic to FUD; listen to concerns; demonstrate (words and deeds) how these will be taken seriously; and recognize achievements. Empathy must be authentic; if your behaviors are not seen as real, you will lose the trust you are trying to create.
    • Shift from a (real or perceived) “blame culture.” Failure is OK as long as we learn and improve as a result. If there is no trust, then people fear consequences of making mistakes and are less willing or likely to want to experiment or speak up.
    • Leaders should show vulnerability, too. Set an example of having failed and learned. Admit in these turbulent times when new ways of working really are new, managers don’t have all the answers—it is a journey of experimenting, learning and improving and giving feedback.
    • Allow teams time to learn and discover themselves. Recognize that not all teams mature at the same speed, appropriate leadership styles and coaching for teams. Use external (to the team) coaches.
    • Be prepared for the difficult discussions and decisions. Not everybody can and not everybody wants to change. If you are dependent upon heroes, get rid of them—literally or metaphorically. One example noted was a toxic working culture with teams held to ransom by the hero, while another example was having the hero share knowledge, coach and create more T-shaped capabilities.
    • Look for “change champions” in teams—peer level respect. Empower them with the right skills and recognize their contributions and the team successes. “Give a podium” to success from desired behaviors such as experimenting, learning and improving.

    From the discussions it was clear that C-level managers need to learn new skills and demonstrate new behaviors, while management layers must learn more behavioral and organizational change skills. Teams must not only learn DevOps technical skills, but also team-building and soft skills.

    CIO Platform Nederland and DASA will be organizing future events and sharing content and experiences from member organizations to address these needs as a follow up to this kick-off event.

    — Paul Wilkinson