Category: Enterprise DevOps

  • Funding Failure by Intent

    Funding Failure by Intent

    Why is it that when we purchase a lotto ticket, every part of our brain tells us we won’t win, yet some portion of it is able to inspire hope where there shouldn’t be any? Could it be the same part of our brain that inspires hope in the face of nearly certain failure has been nurtured by the software development projects we have been exposed to over the years?

    It would appear that the ability to prognosticate has been the foundation of nearly every waterfall project that had a timeline of three years or longer. It is this myth of “certainty” that agile methods are designed to eliminate. In our new approach, we simply break down and digest the scope of work until it is small enough for our risk tolerance, and easy enough to implement in our abbreviated time horizons. We try, we evaluate and we try again.

    But no matter how low we believe the risk to be (and generally the reward we expect from it), risk does remain. And now instead of a three-year project plan well-equipped with milestones, corrective actions and projected costs, agile gives us only the current sprint, based loosely on our historical velocity and a literal “addiction” to continue this process going forward without any real regard for a predictable end-point or total project cost. Continuous delivery infers continuous innovation (emphasis on the word “continuous”). And if done right, why would a business ever choose to end innovation that works?

    Imagine the analogy of Mozart sitting down to compose a quick advertising ditty that your company could play as background music on its website. Mozart begins with a few bars, and notices a theme that is worthy of far more exploration … so he does. He continues exploring this until an epic symphony emerges. This was not the plan and does not meet the original requirements. This could be considered scope-creep in the waterfall world, and thus a failure, as no one stares blankly at a website for the 30 minutes it takes to hear an entire Mozart epic … or do they?

    Every audience grasps the concept of background music we pay little attention to but subconsciously understand it improves our experience. But nearly every audience also realizes the beauty of a Mozart epic, and would gladly buy CDs, MP3s or concert tickets to hear one of those over and over again, relishing every note. The question this analogy begs is simply this: Would Mozart’s employer—more specifically, his development manager—have been free to fund this “failure” or not? What happens when a project changes direction significantly away from the first vision we had of it? Does DevOps enable us to pursue this radical change in direction, or are our legacy financial systems and methods binding us to a waterfall delivery and reporting structure—and, in so doing, limiting what we will ever be able to accomplish through radical innovation?

    The beauty of agile construction and DevOps delivery is that we may deliver “unexpected” innovation. Nearly every time a user “sees” what we have delivered to meet their need, they immediately think of “x” more things we could do to make it even better. This may result in taking a project in a radically different direction within the confines of a single financial year. To further enable our ability to do this, our legacy financial funding and reporting systems must evolve to match this demand. Our new risk profile must include the ability to fund failure on purpose, understanding that failure itself is part of the maturation process of innovation, not the death-knell of the company.

    A more honest approach to funding agile/DevOps efforts would be to determine the annual investment the organization wants to make in certain capabilities, and innovations it expects to see in each thematic area. Overarching themes are funded on an annualized basis; however, great autonomy is granted and expected within the themes to ensure delivery is achieved. Pushing financial responsibility down in the organization to as low a level as possible has the effect of pairing the team leaders with the costs of what they do. Spending too much time on delivery of any given feature or function may cause the team to re-evaluate the approach and propose a better solution. This is more unlikely when budget responsibility is managed several levels above the team doing the work—in that instance, the budget resembles more of an infinite resource they need not be concerned with, until it is too late.

    When proponents of DevOps claim it is changing the culture of a company, they are not limiting this view only to IT. Tertiary services in a company, such as the supporting financial budgeting systems, hiring systems and even contract management systems, may require significant change to match the pace of delivery and innovation DevOps delivers. As an enterprise looks to implement DevOps, it should look broadly at what else may be required to fully enable the pace of innovation it is capable of.

    To continue the conversation, feel free to contact me.

  • 3 Reasons Why Automation is Critical

    3 Reasons Why Automation is Critical

    We’re hearing a lot about the “outcome economy”—how businesses must be results-oriented to meet evolving customer demand and gain greater market share. Today’s marketplace is connected, always on and increasingly competitive. Companies are adopting what we call an “as-a-service” approach to achieve better outcomes fast, consuming and leveraging leading-edge technologies such as cloud and automation.

    As IT systems grow exponentially, and cloud solutions proliferate, non-automated, manual systems increasingly are becoming a major business liability. Today’s systems are simply becoming too big and complex to run completely manually, and working without automation is largely unsustainable for many enterprises across all industries.

    Automation involves a set of tools, processes and insights that allows IT environments to self-modify and adjust, and some enterprises have started using intelligent automation to drive a new, more productive relationship between people and machines.

    For example, IT automation is often used to auto-scale and load-balance large fleets of servers, manage global content distribution based on geographic demand, enable self-healing of IT systems and manage security mostly with limited ongoing manual intervention.

    Moreover, automation enables the ability to adapt and improve the service experience without manual intervention. However, while these tools offer new strengths and capabilities, they are meant to complement and enhance human skills.

    Effective automation depends on adequate insights collected from all the systems relevant to the service experience and business outcome you’re trying to augment. Insights from data is necessary to create opens paths to automated predictions and ultimately using machine learning, or artificial intelligence, as part of a full scope the as-a-service construct.

    Specific insights known as telemetry allows signals to be harvested and interpreted so automation can better adjust production systems to maintain a healthy business. The insight gathered from such analytics allows automation to validate and compose modification rules. For example, sensors that detect a supply chain issue could automatically reroute or fine-tune related functions, such as dispatch or logistics, to solve or generate a workaround for the issue. The business flow can adapt and realign automatically with the ultimate goal of improving the customer experience.

    Automation Creates High Resiliency

    Two common business outcomes that depend on efficient automation are highly resilient systems and experimentation platforms.

    Highly resilient systems include automation that can detect, avoid, heal and remediate any deviations from normal, healthy business function. To detect deviations, automation capabilities need to understand what the “steady state” of the system is and what constitutes the “health” of the system under varying conditions. For each detected deviation from an established steady state, a specific automation is triggered that attempts to return the system back to the steady state.

    The best way to determine if resiliency automation works effectively is through a process known as “fault injection.” Highly resilient systems run under constant fire drills in which operations insert faults into the system while developers continuously build responding resiliency automation.

    Automation also can provide a higher degree of experimentation and increase agility, two key attributes of as-a-service economy. Automatically provisioning a component such as a virtual machine, for example, is only a piece of the puzzle since automation is most valuable when it contributes to improving a customer experience or delivering a business outcome.

    A platform that’s constantly testing, experimenting and developing allows companies to try new ideas in production quickly without fear of failure or outage. When confidence in system resiliency is high, it allows businesses to test new things directly in production (A/B testing). If an experiment fails, there is no harm done as automation returns the system to steady state. If an experiment succeeds, it is quickly absorbed into the production itself.

    A fast, efficient experimentation platform enables businesses to react faster to failures and successes—and pivot accordingly without excess wasted resources. For example, a retail company might change a shopping basket feature for 1 percent of its customers. With constant measurement and instrumentation, the company can automatically derive insights, determine if the change is effective and create a chain of automated reactions. If, say, the demand spikes for a new offering based on a limited customer pilot, the system can reset stocking levels ahead of geographic or further customer segment rollout. This ability increases a company’s agility and adaptability, improving the customer experience and delivering on the most important factors determining success in today’s as-a-service business environment.

    Moving forward

    In this fast-paced marketplace, it is becoming clear that automation is the next critical step in IT maturity—building trust into IT systems and enabling innovation of new capabilities.

    However exciting, impactful and resourceful automation is, it is important to note that automation will not replace people. Machines and artificial intelligence will be the newest recruits to the workforce, bringing new skills to help people do new jobs, reinventing possibilities across the business spectrum. The companies that succeed with automation take a “people-first” approach, adapting their enterprise’s organization, culture, skill and experience to use automated technology.

    Although automation becomes critical to stability when systems grow to global scale, it also provides great benefits to smaller systems. Every modern enterprise should start thinking about an automation strategy. Companies that aren’t experimenting with automation are going to be outpaced quickly by the competitors that are quick to analyzing data, yielding insights that make their processes more agile and unchains their systems and infrastructure from undue age-old complexities.

    About the Authors

    Marshall Wells_PhotoMarshall Wells, Managing Director, Hybrid Cloud

    Marshall Wells is a managing director at Accenture, focusing on Accenture’s Hybrid Cloud platform. He holds an MBA in Finance from theUniversity of California, Berkeley and a BS in Chemical Engineering from Colorado School of Mines.

     

     

    Miha_PhotoMiha Kralj, Managing Director, IT Strategy and Cloud Computing Transformation

    Miha Kralj is managing director at Accenture, where he is responsible for Accenture’s IT Strategy and Cloud Computing Transformation platforms. In this role, Kralj helps to define core cloud strategies that help clients transform traditional IT, creates and delivers IT transformation programs, and identifies the right cloud disposition for various workloads.

    Prior to assuming his current position in 2015, Kralj served as principal architect at Amazon Web Services, leading the domain of IT Transformation and modern IT delivery. Before that, he was senior technology director at Microsoft, where he designed Enterprise Architecture offerings through assessments, planning and value realization.

  • DevOps Debates: Monoliths or Microservices

    DevOps Debates: Monoliths or Microservices

    From the time software came into existence, application software has been booming to leverage the benefit of the computing paradigm. During this cycle, it has transformed decades of design and architectural thinking, and has crossed all aspects of application development and management needs, be it configurability, reusability, maintainability, scalability or security (a good list is of these “ilities” is here).

    While many have been addressed, one “ility” that has turned a lot of eyes and attention in the last few years is deployability. More than that, it is the ability to do so continuously. This requirement has changed the thinking of building software for two main reasons: (a) Businesses have started asking to release software faster than ever, even in verticals unheard of before; and (b) Cloud and infrastructure as code has made faster deployment a easier possibility. When you look at application design with deployability in mind, given the need to be agile, what you end up getting is a microservice.

    Most applications of old style addressed most “ilities” in an architecture but did not consider “continuous deployability” as a core need as well. I call these monolith from a deployment perspective. They are tightly coupled, and for any small change a complete build of the application was necessary.

    Now the question is, Should you really look at microservice as a silver bullet to deploy all software? Should you consider that as the de facto standard to deliver any application functionality? Will applications cease to exist the way we know? Here are my thoughts. As usual, feel free to add yours or write to me personally, as some of you do as well.

    Applications as monoliths will stay as they are

    1. Monoliths have evolved over generations of design and architecture tests and stood the test of time. The focus on modular development is, in fact, a way of creating modules, which are built with separation of concerns in mind.
    2. Monoliths were built to run for a long time. It had addressed some of the “ilities” well before they became considerations.
    3. Applications will remain easier to debug, as most of the debuggable areas are in developer’s hand, instead of having to work with a team.
    4. Applications will perform better, as they can share data in memory; they don’t have to do out-of-process calls to get one task executed.
    5. Applications will be more secure, as they will be a black box from an end user perspective or even a hacker perspective, with little coming out from the system.

    Microservices will be a monolith-killer all the way

    1. Deploy, deploy and deploy to correct your problems away. Fail fast and fix faster is the mantra that you can practice.
    2. It helps bundle two-pizza teams owning microservices and hence embrace better ownership and faster response.
    3. The overall architecture to deliver an end-to-end use-case may become complex, but each piece of the jigsaw puzzle can be handled faster.
    4. In the faster-moving world, business wants to try a concept rather thinking through all the “ilities” ahead of time, and progressively make everything better. Microservices helps abstract all complexities much better.
    5. Microservices will help developers think out of box from day one, while helping business deliver smaller chunks, without having to make the system a “big ball of mud.”

    At the end of the day, irrespective of the approaches, the key is to ensure the applications are managed well, changes are refactored well, the code that is written is readable and maintainable, and significant test assets are created to regress the entire system quickly. So whether it is a large system or microservices, you are able to make any necessary changes quickly and can deploy fast.

    While microservices surely will help this new faster-deployment thinking, I—and I am sure you all—have also seen large monolithic systems that are making delivery at an astounding pace.

    Thoughts?

  • 5 Things DevOps is Not

    5 Things DevOps is Not

    For those who have been embracing and championing DevOps for years, the growing popularity of DevOps is as exciting as landing your first job out of college! Okay, maybe not that exciting, but still really cool. More and more companies of all sizes are embracing DevOps by including the adoption of DevOps practices directly into their fundamental strategies.

    On one side, this is giving teams the leverage they have needed all along to either begin their DevOps journey or to increase the maturity of existing DevOps adoptions. On the other side, when DevOps becomes a mandate, some of that grassroots passion can become watered down, as the use of the word becomes increasingly overused or incorrectly used.

    DevOps incorporates a new collaborative culture that embraces numerous practices combined together for a continuous software development methodology that places significant emphasis on feedback loops, collaboration and continuous improvement. DevOps requires fundamental cultural changes and includes so many practices that can be overwhelming to those just starting their journey.

    Instead of highlighting everything that DevOps is, I thought it would be good to take a step back and highlight what DevOps is not instead.

    1. DevOps is not simply combining Development & Operations teams

    We’ve all seen this one when the term DevOps comes up. “Let’s combine the Development & Operations team and, voilà!, we now have DevOps.” DevOps combines a set of processes and practices to be adopted throughout the entire delivery pipeline and spans multiple stakeholders. A couple of the key practices within DevOps adoptions include continuous integration (CI) and continuous delivery (CD). Simply combining two teams and calling it DevOps cannot accomplish those practices.

    2. DevOps is not a separate team

    Setting up a separate DevOps team is another trap many organizations fall into when beginning their DevOps journey. In general, I’m not a fan of the DevOps team as I believe it leads to more silos in the end. I’ve also found the creation of these teams can lead to further confusion when the mission is not clearly defined.

    There are some cases where a temporary DevOps team may make sense to help enable the processes and potential tooling that may be needed for adoption, but the key is making it temporary—which is often better in theory than in practice. There are several great blogs out there discussing DevOps teams such as Matthew Skelton’s blog, “What Team Structure is Right for DevOps to Flourish?”

    3. DevOps is not a tool

    Let me be the first to say I actually love the growing number of tools that enable us to continue maturing our DevOps adoptions, but I’ve noticed the use of one or two tools starts to lead to the perception that DevOps is a tool. How many times have you heard?

    “We’re already doing DevOps. We have Chef.”

    “We do DevOps. We automatically deploy through Jenkins.”

    Keep in mind, I’m a big fan of both Chef and Jenkins, but I think the power of DevOps is drastically underutilized if you’re equating the utilization of a single automation tool to DevOps success. Adopting automation and tooling is, of course, a part of DevOps, but only when combined with end-to-end practices of increased collaboration with continuous integration/continuous delivery, amplified feedback loops and continuous improvement (to name a few)! DevOps is a journey and that journey may start with a tool, but the goal should be to first identify your DevOps strategy and then find a tool or tool chain that meets those goals.

    4. DevOps is not a one-size-fits-all strategy

    Because there are so many different business drivers and technologies to consider when setting up an overall DevOps adoption strategy as well as identifying your DevOps tool chain, it’s important to apply the same tenets of DevOps to your DevOps strategy. Embrace change, gather metrics, understand feedback, fail fast and correct your course quickly. As an example, if you initially identify a tool that no longer works for your technology or environment, abandon it and move on. Just because you used it on the last project doesn’t make it a silver-bullet fix for the next one. We need to first understand our current strategy and environment, then react accordingly.

    5. DevOps is not automation

    That one may have caught the eye of a few people, so I’ll clarify: DevOps is not solely automation. Automation is absolutely a huge part of DevOps; however, it is not the only part, and deploying some degree of automation is often used interchangeably with DevOps. I think understanding the key DevOps practices is a great precursor to ensuring DevOps is viewed as more than just automation. Understanding the core DevOps principles is key to understanding the true benefits of DevOps adoption.

    There are a lot of things that DevOps is and I think I share a widely spread passion for DevOps. There are also a few things that DevOps isn’t—or isn’t solely. If you’re just starting your DevOps journey or still maturing your existing model, making sure everyone on your team has some basic DevOps education and is aware of the DevOps strategy for your project is key. It will go a long way in helping everyone understand that DevOps is about a lot of things. It’s also not about a lot of things.

    The list above isn’t exhaustive and I’m sure you all have a few you’d like to see on the list. I’d love to hear you thoughts and suggestions at @Shelbee_SE.

    You also can download and browse through the DevOps for dummies e-book.

    About the author/Shelbee Smith-Eigenbrode

    Shelbee

    Shelbee Smith-Eigenbrode is a Senior Software Engineer/IT Architect at IBM Cloud Infrastructure Center of Competency. She has worked across varying roles spanning the software development life cycle. Her experience across functional roles as well as performing that work deeply inside traditional silos has contributed to her passion for embracing the DevOps culture and transforming the way teams are delivering technologies by adopting DevOps practices. She is a member of the IBM Academy of Technology committed to driving innovation and technical advancement.

  • Interoperability’s Impact on Transparency

    Interoperability’s Impact on Transparency

    We fix what we see is broken. Providing “transparency” into the DevOps-enabled software development life cycle (SDLC) has tangential goals of quickly illuminating what is broken, running slower than it should or working flawlessly.

    Automated testing systems, for instance, are generally good at highlighting pass/fail results from integrated testing in the development process. Once they have proven their reliability, they become a de facto part of the development automation and rarely are thought of again. However, when a post-DevOps legacy build begins to take longer and longer to complete, are we able to quickly examine the runtime of our entire body of integrated tests to ensure they continue to perform within expected execution timing parameters (i.e., could the code be fine and the tests themselves be the culprit)? And is that problem exacerbated when the number of different testing tools begins to explode on the same coding unit of work?

    Connecting the dots has always been the goal of DevOps. In the ideal world, the business requirements—or the epics and stories—are part of the “dots” that remain connected or associated with each change in coding, builds, deployments and ultimately release. But are these “dots” easily transparent in each part of the DevOps continuum?

    Transparency from multiple perspectives

    To think engineers are the only audience of DevOps is too restrictive. Product and project managers are also interested in knowing “the state of change” as it relates to the business issues they are tracking, and in real time. Whether the requirement originated from a bug-fix or as a new feature, the audience of people who want to see what is where within the DevOps set of tools is not confined only to people within the development or operations of IT. The ability to provide sensible, near real-time presentation of what changes are where gives the entire organization transparency to the progress of change. It impacts every organization from customer service, to program management—executives want to know as much as casual viewers in many different business functions.

    Audit is another key driver for full transparency. Regulatory compliance is at one end of this need; corporate quality standards should be on the other. Organizations always should be more concerned with quality standards than the regulations they are subjected to. Therefore, the process of audit always drives the need for greater transparency. Imagine the ability to have “push button audit” enabled by DevOps for any change under inspection. Imagine what further cost delta this represents to organizations that are still bound to only manual methods of examination. With DevOps, not only is this possible, it also should be normal. The entire history of any change entering production should be stored by each of the DevOps tools within the journey, from ideation to implementation. All test records should be immediately accessible (and automatically associated with the particular version of change in question). Everything from failed builds to the exact time it was released into each impacted data center around the world can—and should—be available for a consolidated, push-button recollection for audit. However, the ability to make this happen often is directly proportionate to the interoperability of the DevOps tools used.

    Transparency of key data and processes

    When point solutions are selected for the DevOps tooling (defined as best-of-breed tools, with no inherent interoperability with other tools in the process), the particular functions they perform are thorough and generally complete. However, the only way to integrate key information between different tools is through APIs and/or a homemade amalgamation of the data from each point solution tool into a common dashboard. Amalgamating data is always a challenge, particularly when you consider the point of view of the audience varies by function and business discipline. If you eliminate or reduce the points of view down to only engineers (as is what usually happens), you destroy the value of DevOps transparency for every other business discipline in your organization. If you spend the labor to create a very functional, multi-perspective dashboard, your investment can be much higher than what you make in the tools themselves.

    Integrated solutions—generally pushed by the same vendor as the preferred (albeit more license-cost-heavy) solution—are not the ultimate panacea, either. While a given vendor may be able to offer tools that track requirements, construct builds, integrate testing, execute deploys and orchestrate releases, more often than not, the vendor is stronger in one discipline over the others. They may ask you (the customer) to use a limited number of tools designed for one thing to do more functions than the tools were intended to perform—which creates a recipe for constant customization and lucrative service deals for the vendor.

    What becomes important to measure in any of these instances, is the level of inherent interoperability that will exist between each type of DevOps tool in the offering. If the “connecting of dots” is inherent, and the state of change can be viewed in any of the tools, that is a win. If the point-of-view offered extends to more than just an engineer, but includes other business audiences who may have interest, that is a win. If the reporting is push-button, that is a win.

    Transparency of tool usage and roles

    But beware the dark side. Whether your organization relies on point solutions or integrated solutions, both can be corrupted by an inconsistent methodology of user profiling within the tools themselves. For example, let’s assume we have a business requirements tool, a build tool, a deployment tool and a release tool. Generally, a programmer, who we’ll call “Dave,” has a very specific role in the SDLC process that doesn’t include releasing code (programmers usually don’t release code into production). Generally, the level of granularity in defining tasks and roles within a given tool is proportionate to the rules your auditors require (whether in-house or regulatory). So for a given bank, we might define the role of a programmer “Dave” to be able to see business requirements, execute builds all day long, deploy them to the test class (or first level) of environment in the SDLC and then simply watch what happens as it progresses to production, never directly interacting again.

    One of the biggest problems DevOps tooling has is that each tool handles user management separately. That may allow me to define the roles of a programmer properly within each tool, but it does not enforce that my guy Dave is properly mapped to that role. I could make Dave a business analyst in the requirements tool, a programmer in the build tool, a deployment manager in the deployment tool, and a release manager in the release orchestration tool. This would allow Dave god-like abilities in each tool within the process. He could, in fact, create the requirement to move all our money to his account, build the code to do it, deploy it and release it into production. Dave would not have triggered any alarms in the tools, because each tool thought he was doing what he was supposed to be doing in that role. And I have never seen tools audit what role Dave is supposed to be mapped to given his job in the company or even spot-checking what role he carries in other DevOps tools to see if there is an inconsistency. Perhaps abstracting user management from the entire set of tools may be the answer—but then, who offers that capability today?

    Transparency, when properly managing access controls and particularly at the push of a button, can provide solutions—even if only after the fact—to a great number of problems that exist in our organizations and in our SDLC process. We fix what we can see is broken. If our DevOps solutions can highlight transparency from ideation to production release, we can have confidence that what we see is real and worth our attention. When our transparency varies by tool, or loses connection within the process, our confidence of the state of change diminishes and our certainty about what needs our attention goes away. Using DevOps to shine a light on our true priorities is yet another cultural change enabler we have the responsibility to pursue.

    To continue the conversation, feel free to contact me.

  • InterConnect 2016: Culture Matters

    InterConnect 2016: Culture Matters

    The recent IBM InterConnect 2016 focused on plenty of tools and technologies, from cloud/SaaS solutions to containers and automation, but there was another popular topic of conversation heard in the halls—the culture of organizations. The basic premise of these conversations was that unless you get the culture right, all the technology in the world won’t get an organization to where it needs to be to compete in the ongoing digital transformation.

    At Nationwide, we have developed a foundational DevOps model that has two main pillars built on top of Gene Kim’s “The Three Ways” and lean and agile principles.

    While there are initiatives to implement the DevOps-related practices such as continuous integration using Jenkins, continuous delivery using TaskTop and UrbanCode and version control everything using GIT, how does one influence culture? Because clearly if the culture is not correct, technology solutions will not be enough to reach the goals of DevOps in terms of reducing lead time and enabling the business areas to be more responsive to market needs.

    Dev Cul 22

    Trust: The Foundation of Culture

    A key component of what underpins culture is trust. This was emphasized in a blog written by Lee Reid last year titled, “The Simple Math of DevOps.”

    So how does one build trust? Here are 13 behaviors that build trust from Steven Covey:

    Dev Cul 33

    Colin Powell, in a video on the Essence of Leadership, talks in a powerful way about this, saying that one way to build trust is to “Do what you say and say what you do.” He also stresses that successful leaders are those who provide selfless service to their teams.

    I don’t claim to have any magical formula to build trust, but clearly an organization’s transformational success must include a focus on culture and trust. Without this, the true promise of DevOps won’t be realized.

    You can read more about my session, “Achieving continuous visibility across the DevOps Lifecycle” at InterConnect 2016.

    About the Author| Carmen DeArdo

    Carmen-DeArdo-sq
    Carmen DeArdo is technology director at Nationwide Insurance. An ardent advocate of DevOps and a leading speaker in the DevOps and technology related events, Carmen is responsible for driving continuous delivery utilizing DevOps, lean and agile techniques across mobile, distributed and mainframe and other technologies.

  • Trick Question: What is DevOps?

    Trick Question: What is DevOps?

    DevOps suffers from the lack of a clearly defined definition. A Google search displays results that state that DevOps is empathy, culture or a movement. There are also derivations of DevOps, such as ChatOps and HugOps.

    A lot of speakers mention DEVOPS but no-one seemed to have a widely agreed definition of what DEVOPS actually means.

    — Stephen Booth (@stephenbooth_uk) November 19, 2015

    Proposed Definition of DevOps: “Getting more done with fewer meetings.”

    — DevOps Research (@devopsresearch) October 12, 2015

    The real head-scratchers are the number of job postings for DevOps engineers and the number of certifications for DevOps popping up all over the web. The job title “software engineer” is contentious within the technology community, so the job title “DevOps engineer” is just begging to take pointless debates to a new level.

    How, then, do you create a curriculum and certification course that has any significant value on an unclear subject?

    For a methodology that has an emphasis on people, empathy and communications, it falls woefully short of heeding its own advice and values. On any given day, you can see the meaning debated on blog posts and tweets.

    My Understanding of DevOps and Why it Exists

    DevOps is an extension of the agile methodology that is hyperfocused on bringing customers extraordinary value without compromising creativity (development) or stability (operations). DevOps is from the two merged worlds of development and operations—operations in this context include all aspects of IT such as system administration, maintenance, etc.

    Creation and stability are naturally at odds with each other. The ripple effect is a good way to explain how these two concepts have friction. Stability wants to keep the pond from becoming turbulent and causing harm. Creation leads to change, which can act as a random rock thrown into the water and sending ripples throughout the whole pond, leading to undesired side effects that cause harm to the whole ecosystem.

    DevOps seeks to leverage the momentum of controlled ripples to bring about effective change without causing enough turbulence to impact the whole pond negatively.

    The natural friction between these two needs often drives a wedge between development and operations. Operations worry that a product update may include broken functionality that customers have come to depend on, and developers worry that sorely needed new features may not make it to customers because of an operation’s resistance to change.

    Instead of taking polarizing positions, DevOps is focused on blending those two positions into a force that effectively provides value to the customer without compromising the creativity and stability that a product or service needs to compete in an ever-evolving world.

    Why is a Clear, Singular Meaning Needed?

    The understanding of a meaning is an important part of sending a message.

    If an unclear word is used to send a message, the word risks becoming noise and the message risks becoming uninterpreted or misinterpreted.

    Without a clear, singular meaning, you risk losing the message that you want people to hear. In technology, I see messages get drowned in noise all the time.

    In communications theory, noise is anything that interferes with understanding. Noise is more than just the sounds of static, loud music or machinery. Creating noise can be simple as using obscure words to explain a topic or providing an unclear definition that muddles the comprehension of a given subject.

    DevOps suffers from too much noise that increases people’s uncertainty of the word. After reading a few posts on DevOps, each one with its declaration of the essence of DevOps, people can get confused by what the word means. DevOps is empathy! DevOps is culture! DevOps is a movement!

    Because of noise, DevOps seems to stand for multiple ideas plus agile operations without setting any prioritization or definitive context. Okay, so which is it? Is it one of those or is it all of them? Which idea is the most important? Furthermore, these ideas can cause friction, as not everyone shares the same view on these topics. DevOps is supposed to reduce friction between naturally opposing groups within a business, not create more of it.

    Remove the Barriers

    Another aspect of noise is using words that are a barrier to understanding a given definition.

    DevOps is the practice of operations and development engineers participating together in the entire service life cycle, from design through the development process to production support.

    — the agile admin

    People from a non-technical background or outside the world of agile and development may have a difficult time understanding a definition like that. When I read something like this, all is hear in my head is same sound the teacher in Peanuts makes: “Blah, blah blah, blah.”

    If you want people to remember something, make it easy to understand and memorable.

    Appeal to All Personality Styles

    In marketing, you’re taught to write to appeal to four personality styles: driver, analytical, expressive and amiable. Getting people to work together in the workplace also requires appealing to these four personality types.

    There is a need for a single definition of DevOps that appeals to the four personality styles—or, at the very least, refrains from being a barrier to entry. If a person needs to persuade someone who has a driver personality, but the definition includes language that invokes an automatic “no,” that puts people who want to use DevOps at a disadvantage.

    Give people every advantage you can for them to adopt DevOps.

    It’s Time for a Definition for DevOps

    At a time when the DevOps role is becoming increasingly important for businesses, it is crucial that a globally recognized definition be created to reduce uncertainty. It’s hypocritical to introduce a word that touches upon the importance of communication and then expect people to take it seriously when the definition constantly changes.

    It’s time that we have a singular definition for DevOps so people can use it for the hiring process or certifications without the risk of the message being lost or co-opted into something it is not.

    About the Author/Michael Herndon

    UntitledMichael is the head of DevOps at Solovis, creator of badmishka.co, and all around mischievous nerdy guy. He is writing on behalf of Packt Publishing.

  • Changing the Culture of Metrics Analysis

    Changing the Culture of Metrics Analysis

    You could say the development and operations teams gave birth to the idea of DevOps, as it is believed the chief benefits are garnered in those two functional disciplines. Because of this, metrics nearly always are conducted from the points of view of cost reduction, speed and efficiency. This makes perfect sense when business treats IT (and IT sees itself) as a service- or cost-based organization. But what if DevOps has a few more characteristics that could actually change that culture? What if, for the first time, DevOps could enable IT to become a real partner with the business organization it serves?

    To make this happen, business and IT must begin to share attainable goals. Imagine the impact of a highly functioning, enterprise-class DevOps service within IT; upon the time to market of products and services the business is looking to launch. A proven, consistent ability of IT to deliver on time or ahead of schedule (i.e., reliability) then begins to change the dynamics of what marketing proposes and delivers. Time then becomes a valid metric to consider when looking at customer adoption rates. DevOps usually does not directly impact initial market demand, but the impact of delivering new services or features months or years ahead of previous projections can dramatically impact time to market. For example, if your business is able to change its billing model or billing methods six months to a year ahead of every one of your competitors, what impact might that have on market adoption? At that point DevOps becomes a key business differentiator, until the rest of the universe has it.

    Consider an additional impact of a highly functioning DevOps service on feature delivery or the feature road maps themselves. The implementation of DevOps inherently delivers smaller units of change on a higher frequency, reducing risk and allowing iteration and alteration of future plans based on the real-time feedback of change success. Marketing focus groups may provide an original road map, but the impact of DevOps can be integrated back into those same focus groups. Instead of imagining what might be possible, DevOps allows delivery of what is possible to see, feel, touch and prompt further imagination. The net impact is an increase to the pace and depth of innovation itself. You know what your customers really want, and you can give it to them quicker. What kind of impact does that have on the reputation of your business if it becomes a market leader because of this? Or what happens when a large organization is then able to deliver with the speed, agility and flexibility of a small firm?

    Cost Reduction vs. Revenue Generation

    Now consider the value of cost-reduction metrics compared with revenue-generation metrics. It is easy enough to determine that the pre-DevOps delivery cycle might have been 12 weeks per feature as an example, or that each build took six hours to complete and now post-DevOps takes only 12 minutes to run. Metrics like these will begin to show an increase in productivity by existing development staff—or, maintaining the same feature delivery rate, a reduction in the number of personnel needed to achieve them. Similarly, it is easy to show a reduced number of trouble tickets post-DevOps, and a concurrent reduction in the average time to resolve each ticket. DevOps then has enabled the CIO to continue to take arbitrary budget cuts demanded by the business each year. This is not partnership—over time this is a recipe for the reduction, pace and quality of innovation (the smartest people leave first during layoffs; they don’t need to stay).

    However, even significant cost-reduction metrics pale in comparison to revenue-generation metrics. As an alternative, imagine what the net growth in customers of a given product might be, over a six-month period of time, because a desired feature set was delivered early? And what if it is two to three years before your competitors can implement the same high-functioning DevOps services you already have in place today? If a business makes $2 per-widget profit and captures 20 percent additional market share over six months, equating to “x” million new customers … well, you begin to get the idea. All of the sudden IT is no longer viewed as a cost center by business; it is viewed as a revenue-enabler. The immediate business response is, “If I allocated more budget into the types of staff facilitating DevOps services (running the gamut from coding to operations to marketing), could I gain even faster delivery rates or deeper feature sets or more market share by pioneering what has never been pioneered?”

    The two main reasons this realization hardly ever occurs in business today are that 1.) IT has failed at delivering on promises so often it has trained the business to believe it “cannot” deliver on time or on budget; and 2.) few businesses conduct accurate time-to-market metrics measuring the delta of what additional “time” does to the equation, largely because they never had to. DevOps provides the catalyst to begin to re-examine both of these phenomenon.

    But it goes deeper than that. Changing the culture of innovation, in which I am eager to test new ideas faster, means I need more ideas all the time. The competitive advantage our nation, our businesses and our careers have always benefited from is our ability to create and innovate. Other nations around the world, or other organizations your company competes with, may be able to do repetitive tasks faster or more efficiently. But rarely do worker-drones create or produce a wealth of ideas. But if we begin to focus on the long-term impact of fostering innovation in our own country, and in our companies, we will remain perpetually out front. The goal of DevOps is not merely efficiency, it is also to spark the imagination. When our focus and our metrics reflect this, we all win.

    If you would like to continue a conversation on how to create and implement a highly functioning enterprise class DevOps service, feel free to contact me.

  • In DevOps, Standardize to Minimize Architectural Debt

    In DevOps, Standardize to Minimize Architectural Debt

    I recently reading a great Q&A with the Etsy CTO John Allspaw over on The New Stack, which contained some nuggets of good advice that increasingly are important as we continue the journey toward DevOps adoption. Of particular note were statements on the introduction of “new” things to solve problems. In a nutshell, Allspaw reinforced the importance of standardization as a means to reduce architectural debt.

    If you aren’t familiar with the concept of architectural debt, it’s similar to technical debt, in which choices made today incur longer term investments of time and money when future changes are introduced. If you settle on a caching solution such as Redis, for example, there are technical and architectural implications. You can’t simply switch out Redis for memcached later without significant changes to both code and the infrastructure.

    Even your choice of load balancer today implies architectural debt because of the growing inclusion of scalability as part of the application infrastructure itself. The result is integration with automation and orchestration systems that are not easily changed later on if you decide you want a different solution. The style of service registry you choose is an architectural decision that impacts clients, servers and systems across the microservices architecture. Switching from client-side to server-side will later be costly.

    That’s architectural debt.

    Allspaw noted the need to not reinvent the wheel when possible, and instead encourage the collaborative aspects of DevOps as a means to avoid reinventing the wheel and potentially increasing the architectural debt load on the organization.

    This is not a new concept. There’s a reason enterprises in particular proudly proclaim themselves “Java shops” or “Microsoft shops” and primarily require development in one or perhaps two languages, but no more. Maintainability is a component of technical debt, just as middleware decisions are a component of architectural debt.frameworks in use soad2016

    So as we continue to see wider DevOps adoption, it’s important that we carefully consider the impact of our choices in terms of the automation, orchestration and integration required to implement continuous delivery and deployment. Once you’ve invested time and energy (and probably budget, too) in Puppet or Chef, VMware or Cisco, OpenStack or SaltStack, you’re going to find an unavoidable amount of technical and architectural debt in development as well as production across infrastructure and networking teams.

    Each of these systems, solutions and frameworks comes with its own components. It isn’t just a language, it’s a comprehensive system with management and repositories and agents and reporting components. All of these impose certain architectural requirements on organizations, from needing certain software running to prescriptive details on networking that must be enforced, lest the system break down.

    We’re already starting to see that fragmentation in enterprises that are engaged in automation and orchestration efforts. When asked the number of automation frameworks (Cisco, VMware, Python, OpenStack, Chef, Puppet, etc…) in use, one-third today reported using three or more.

    That’s not necessarily “bad,” as it could be indicative of standardization across silos (networking wants to use Cisco, infrastructure is already on VMware, and Python is being used to cobble it together by ops), but it also could be an early sign of the kind of fragmentation that causes exponential growth in technical and architectural debt.

    Certainly there is room for experimentation in these early days, but it behooves us to maintain a watchful eye on the use of frameworks and systems and languages we employ for automation and orchestration to ensure we don’t end up buried in technical and architectural debt.

    Standardization, even loosely enforced, can help minimize the impact of that debt before it spirals out of control.

  • DevOps and Agility for Multi-Speed IT

    DevOps and Agility for Multi-Speed IT

    Let’s step back for a moment and look at the IT world a bit differently from the traditional technology-centric lens through which most of us IT peeps tend to look at things. Line of business (LOBs) need us, the IT organizations, to deliver capabilities to their customers and users that deliver business value. They also rely on these systems to provide feedback on how customers and users are consuming the business capabilities and deriving value; how they are interacting the business systems; and what capabilities and features they need to get better value. The LOBs are asking the IT organizations to deliver these capabilities with:

    • Velocity
    • Agility
    • Innovation
    • Cost Management

    IT organizations, in most cases, are not designed for velocity, agility or innovation. They are large organizations hindered in their ability to deliver these by legacy systems, myriad practices and tools, overbearing governance and compliance regimes, archaic and siloed organizational structures and cultural inertia. This lack of ability of the IT organizations to deliver agility, velocity and innovation in some cases even results in LOBs creating their own “Shadow IT” organizations to deliver their IT needs.

    Finding Velocity, Agility and Innovation

    The solution of late has been to undergo a DevOps transformation that enables IT organizations to adopt the necessary practices, automation tools and organizational and cultural change to deliver with velocity and agility, and become innovative by being able to deliver rapid experimentation. The origins of DevOps lie in Lean IT, and adopting DevOps is really applying Lean principles to help achieve agility, velocity and innovation.

    DevOps does so by making organizations that are delivering IT systems and software more efficient and lean. Organizations that have adopted DevOps are able to:

    • Get ideas out to production faster
    • Have users/user surrogates use them
    • Get rapid feedback

    Using this feedback the organizations can improve three things:

    1. Applications delivered
    2. The infrastructure delivered
    3. The process by which IT systems and software are delivered

    However, large organizations have been struggling to achieve the results promised by DevOps and achieved by the poster children of DevOps, with whom the movement started. For every Netflix and Etsy out there, there is a large, complex, organization that has been struggling with simply where to start. They may be able to achieve success in some parts of their IT organization—with small, innovative ‘two pizza’ teams working on isolated projects—but not at enterprise scale. They struggle with how to achieve even the results of DevOps adoption they see in their own successful teams, at scale. The organizational and cultural inertia appears to be too difficult to overcome.

    The major reason of this inability to achieve at scale is the persistent struggle in all large organizations between the desire to innovate, and the need for stability. This push-and-pull exists in all organizations—even an innovative startup feels it. The Dev teams are focused on creating innovation thru rapid change and experimentation, and the Ops teams are focused on creating stability and continuity. DevOps addresses this existential struggle by adopting a series of practices that drive cultural change, automation and process adoption that allow these Dev and Ops teams to work toward common goals and balance their needs. Even in large organizations, this can be achieved on small teams that are able to circumvent the legacy organizational structures and practices and figure out how to get all the stakeholders working toward common goals. That is why this works great on small, strategic projects that have well-defined goals and timelines and can “fly below the radar” of corporate governance enforcers. As we move to larger projects with large, potentially distributed teams, spread across different groups and corporate “fiefdoms,” adoption struggles and often fails.

    The Rise of Multi-Speed IT

    One fact these large organizations need to recognize is that they really must have two modes of delivering IT: one focused on innovation (and velocity and agility) and the other focused on optimization (stability and continuity, with agility and velocity). This view is being referred to in the industry as “bi-modal IT” and is actually more of a continuum than a clear-cut two modes model. Hence, the term is evolving now to “multi-speed IT.” However, classifying the continuum into two broad categories based on the business intents of innovation or optimization serves the purpose of determining how to adopt DevOps very well.
    Multi-Speed ITMulti-Speed IT

    One interesting way to look at these two modes is to classify them as:

    1. The ‘Industrialized Core’ that delivers core business capabilities—things that keep the business going, and the primary business intent is optimization
    2. The ‘Innovation Edge’ where experimentation happens and new systems get identified and developed; in other words, the primary business intent is innovation

    Innovation vs OptimizationInnovation vs. Optimization

    The goal of the Innovation Edge is to allow for rapid experimentation, development of minimum viable products (MVP), rapid delivery and fast failure. The goal of the Industrialized Core, on the other hand, is to be lean and efficient and deliver business capabilities in an optimized, predictable, stable and secure manner, with the ability to scale as needed. Note: Eventually systems from the Innovation Edge will need to be “industrialized” once they start serving a core business function. This is a continuum from innovation to optimization.

    The Industrialized Core itself needs to be continuously incrementally industrialized to provide the stability and reliability needed, while getting leaner and more efficient, at scale. The core needs to get to optimization, with agility and velocity, as it delivers the core systems, again at scale. The core, too, needs to adopt DevOps and agile development practices, even for legacy systems, including those on the mainframe. This Industrialized Core is, after all, what keeps the business going and delivers the ongoing business value as the Innovation Edge seeks out new and disruptive business capabilities and avenues to deliver them.

    However, they both need different DevOps approaches and platforms delivering Agility, Velocity and Innovation at different levels.

    These two systems to deliver business capabilities are, in reality, joined at the hip. The systems delivered on the Innovation Edge are dependent on the services provided by the Industrialized Core to deliver the innovations they are developing. The Industrialized Core, on the other hand, consumes most of the budget in typical large organizations, and needs to be optimized to free resources for innovation.

    Consider a typical mobile application (say, mobile banking). To deliver that application, there is a team charged with developing the mobile front end. These kind of applications are developed and delivered in the Innovation Edge. Does all of the data and business logic sit on your mobile phone when you use that application? Of course not. Most likely, the business logic and certainly the data are components of the application that are delivered by pre-existing systems or services in the Industrialized Core. So, the mobile application is hybrid, comprised of components running in both places. To deliver even some experimental features of this mobile app, the company needs to release a hybrid or composite application, delivering components delivered by both the Innovation Edge and the Industrialized Core.

    During the Dev/Test cycle, these two modes of application delivery must be as decoupled as possible, so the development schedules and cycles of one does not drag the other, to prevent the Core from becoming a drag on the Innovation. Creating an API ecosystem is thus essential to decouple the Industrialized Core from the Innovation Edge. The Core provides the business services consumed by the innovative applications being delivered by the Innovation Edge as APIs. A key benefit of developing this API-based architecture is that these APIs potentially can be monetized to third-party partners, generating a revenue source at the same time.

    It is important to note that the architectural and process needs for adopting DevOps in both of these modes are different in many ways. Their needs and how they deliver applications and services differ as they are serving different business goals. In my next post, I will start delving into these architectural and process needs for adopting DevOps for both the Innovation Edge and for the Industrialized Core, which allow the Innovative Edge to allow for experimentation and innovation, and for the Industrialized Core become more optimized.

    Watch this discussion between industry practitioners as to how they have adopted DevOps in a multi-speed IT model: the challenges faced, successes achieved and lessons learned.

    About the Author/ Sanjeev Sharma

    Sanjeev

    Sanjeev Sharma, CTO and Distinguished Engineer – DevOps Technical Sales and Adoption, IBM Cloud Unit, is a 20-year veteran of the software industry with expertise in DevOps, Mobile Development and UX, Lean and Agile Transformation, Application Lifecycle Management and Software Supply Chains. He is a DevOps Thought Leader at IBM and speaks regularly at conferences. He has written several papers and is the author of the DevOps for Dummies book.