devopsy.kinsta.cloud

Category: DevOps Culture

  • The Third Way of DevOps: Stacking the Cards in Your Favor

    The Third Way of DevOps: Stacking the Cards in Your Favor

    In the Third Way of DevOps, change is as much attitude as it is behavior

    In our first post, “The Third Way of DevOps – From Knowing to Being,” we shared our thoughts on effectively applying knowledge to put the Three Ways of DevOps into practice. This blog explores a complementary dimension: How to build confidence when faced with a seemingly impossible task.

    This is the all-too-familiar setting. You’ve seen an opportunity to improve things and genuinely want to do something about it. But you soon feel the clarity and anticipation draining out of you. First there’s the pressure of the day job. Then there’s the inertia of your co-workers, who are oblivious of or indifferent or hostile to your ideas. And finally, as if these concerns weren’t enough, you’re faced with the difficulty of unlearning old habits and developing new ones. In short, you don’t take action on what you have learned and know.

    Self-efficacy

    Expressed in the somewhat abstract language of the theories of Planned Behavior and Reasoned Action, there’s a gap between behavioral intention and actual behavior. A plausible explanation for the gap is low self-efficacy, defined as the conviction that one can successfully execute the behavior required to produce desired outcomes. It’s the belief we have in our own abilities, specifically our ability to meet the challenges ahead of us and complete a task successfully. This is a crucial precondition because investigations have shown that peoples’ behavior is strongly influenced by their confidence in their ability to execute that behavior.

    If-Then-Else

    So why doubt your ability to change? Well, when a change seems large, the gut reaction is to feel the need to come up with a project plan. Because a plan creates the confidence that when the steps are taken, the desired results will be achieved. Without a plan, we’d be irresponsible risk-takers. That’s how we’ve been taught to think. And rightly so—at least, in a predictable environment, where you can determine which steps will have which effects. If you work with IT, you are likely to think in terms of If-Then-Else. This is how we would like to believe IT always works. Predictably.

    If-Then-Maybe

    However—and this might be the biggest “however” you’ve come across for a while—things are not always predictable. Not unpredictable in the sense that it should have worked but something just went wrong because we didn’t do enough homework; unpredictable as in simply unknowable. Unpredictable as in possibly seeing a pattern of behavior in hindsight, but never having been able to predict it, let alone control it. Unpredictable as in “complex adaptive systems,” as they are referred to in the world of complexity thinking. It’s no longer If-Then-Else; the name of the game is If-Then-Maybe.

    Complexity Thinking 

    Complex adaptive systems are all around us but if you’ve been used to thinking that things are—or could or should be—predictable, you tend not to see them for what they are: one of kinds of systems that exist in nature. The Cynefin sense-making framework describes systems in terms of five domains: obvious, complicated, complex, chaotic and disorder. Obvious and complicated are both predictable domains, where something in the complicated domain needs a bit of work to discover the predictability than when it is obvious. Complex and chaotic are unpredictable domains, the main difference being that something chaotic is dangerously unpredictable and therefore needs direct action. There is a fifth domain: disorder. You are in this domain when you don’t know which of the other four domains best describes the current situation. A word of warning: People’s cognitive bias will often lead them to think that they are in their usual domain, in which they feel most comfortable.

    Step by Step 

    The way to introduce changes in complex systems successfully is to take it step by step. Because you can’t predict the results of a step, you have to closely monitor what effects—both desired and less so—each step creates. Then, based on your assessment of the new situation, you determine which next step to take. Rather than moving in a straight line toward a “to be” end state, you move each step to the “adjacent possible.” You make good use of the disposition of the system; in other words, where the energy is tending to flow.

    No Big Answers, Just Little Questions

    So, it’s not a case of looking for the big answer. It’s about a series of small questions. The Toyota Kata approach recommends asking ourselves (and others) these questions:

    • What are we trying to achieve?
    • Where are we now?
    • What obstacle is now in our way?
    • What’s our next step, and what do we expect?
    • When can we see what we’ve learned from taking that step?

    A key concept is that we should work towards the next “target condition.” This is an interim goal on the way to whatever we want to achieve. It just describes where you want to be next, not how to get there. It labels the future set of circumstances that lie just beyond our current level of understanding. It will typically have to be achieved between a relatively short period between a week and a few months; otherwise, it will be ineffective. How we will get there will emerge though the process of experimentation.

    Stack the Cards in Your Favor 

    So the next time you feel inspired to make a (little) difference, don’t be discouraged by the prospect of having to think everything through in detail. This is often even contra-productive. The liberating realization that you can take things step by step, is good for your self-efficacy, increasing the chances of actually achieving results.

    This article was co-authored by Mike Orzen.

    — Mark Smalley

  • DevOps and Collaboration: Fraternizing with the Enemy?

    DevOps and Collaboration: Fraternizing with the Enemy?

    DevOps is all about collaboration, we hear.
    “You must collaborate,” say managers initiating DevOps adoption programs.
    HR make new posters declaring the corporate values “We value Collaboration & Teamwork.”
    Can’t fail!
    Then when it appears not to be as effective as we had hoped we tell teams they are “Empowered” to collaborate.
    Blank stares!
    When things still don’t appear to be working we tell teams they are “self-steering,” “self-organizing, collaborative teams”….
    Still, we don’t get what we were hoping for.
    Luckily the tool vendors come to the rescue and tell us why it is failing. We need “collaboration tools and software” – Throw some “Slack” at it.
    Can’t fail!

    This is when we usually get frustrated managers knocking on our door asking about the Phoenix Project DevOps simulation. They are moaning about communication and collaboration! When we talk to the teams they are frustrated and moaning about “empowerment” and managers not “walking the talk.”

    In my mind these lack of collaboration skills is not really surprising. Why?

    Impression: Well, it doesn’t help when the word has also historically been connected to global conflict situations. “He faces charges of collaboration,” “fraternizing with the enemy.” Dev & Ops. Traditional adversaries?

    Interpretation: The interpretation of the word itself is also part of the problem. I googled an English Language Help Desk:

    Collaboration: “To work together with somebody in order to achieve a single shared goal.”
    Cooperation: “Work with other people by achieving one”s own goals as part of a common goal.”

    In an HBR article titled, “There’s a difference between cooperation and collaboration,” it was argued  “… Managers mistake cooperativeness for being collaborative,” adding, “… most managers are cooperative, friendly, willing to share information – but lack the ability and flexibility to align their goals and resources with others in real time.” If managers don’t get it then what chance do the teams have when told – “You must collaborate”!

    Induction: Part of our failure to be able to effectively collaborate could come from our schooling. We are chastised at school for asking others for help, for looking to see how somebody else is doing it—”That is cheating”—things are changing with more “collaborative assignments.” Yet, my children (now in their early 20s) were not taught what effective collaboration is, other than discovering for themselves. As my daughter explained about her school assignment, “You have to do it yourself if you want it done because others will slink off and not do it, then my individual grade may suffer.” They are told they need to collaborate but individual performance grades are scored. My daughter once came home stressed and almost in tears, declaring “Oh no! I have been given another collaborative assignment for my most important subject!” Children leave schools distrustful of collaborating.

    Indoctrination and Imperative: There is also the historical perspective. When we, the current workforce entered our chosen profession we had decades of functional silos that created a “them” and “us” attitude. This stimulates a sort of “territorial imperative” or “tribalism,” not helped by our own “technology” languages and tribal cultures. We have learned over time that other teams “dump things on us” or other teams “stop us from getting stuff done.” In other words, Other teams frustrate work.  (Territorial imperative—”the drive in animals and man to take, hold and defend a particular area, zone, turf,” —e.g. defending their best practice framework! And attacking others that come near!).

    Inconsistency: We then built walls between these teams and if they wanted something from each other they needed to escalate it to their manager. Managers would collaborate or not, especially when they were given conflicting, inconsistent goals and KPIs. Development teams are given goals to release as fast and as often as possible. Operations are given goals to make sure everything is as reliable as possible and available at all times.

    I had an example of this “them and us attitude,” once with a board of directors. I split them into two teams and gave each team a pack of ABC cards (Attitude, Behavior and Culture) and within 10 seconds heard, “Oh good, our team will win this!” automatically creating an atmosphere of competition and winning and losing, I hadn’t even explained the task yet! Teams compete.

    ‘It’ factor: A study by Google (2012) called Project Aristotle, named in tribute to Aristotle’s quote, “The whole is greater than the sum of its parts,” looked into why some teams work and others don’t. One of the key conclusions was that “psychological safety” was the “it” factor, “… a sense of confidence that the team will not embarrass, reject or punish someone for speaking up.” Because we have not learned to collaborate and because we have a “them and us” history often accompanied by blame, there is fear, uncertainty and doubt (FUD) about speaking up, sharing ideas or giving feedback.

    In the dark: However, there is another significant reason for poor collaboration, as I have discovered from running hundreds of Phoenix Project simulations with teams globally. It is the fact that nobody has defined the “behaviors” that demonstrate effective collaboration. We assume everybody knows what we mean when we say, “You must collaborate.” Fact: We don’t!

    Below are the results of a series of recent Phoenix Project simulations. One of the goals was to learn DevOps practices such as CALMS and the Three Ways, while another goal was to learn effective team-working and collaboration.

    I asked the teams at the start of the sessions, “What is effective collaboration? What behaviors will I see today that demonstrate this”?  The teams came up with a list of behaviors (each team came up with a top five or six behaviors; this is the consolidated list from seven teams).

    • Transparent communication – “open and honest” and “frequent.”
    • Calm and constructive dialogue (no blame, accusations and shouting).
    • Feedback “asking for and giving feedback.” Constructive feedback.
    • Shared team goals (and cross team). What are we aiming for? Impact on others and their goals. Maybe conflicts that need managing.
    • Listen – is it clear and understood? Show that you listen.
    • Being open for dialogue, being approachable –not closing the door or hiding behind.
    • Giving and taking responsibility.
    • Sticking to agreements.
    • Central moment to get together (e-2-e).
    • Knowing the roles and responsibilities, sticking to responsibilities.
    • Mistakes are OK.
    • Be proactive to get information needed.
    • Equal treatment: “Respect” ideas from ALL
    • Don’t HOPE, confirm.
    • “Willingness.” Confirm and agree.
    • Ownership of responsibilities.
    • Clear agenda and purpose of meetings.
    • Pleasure – celebrating success.

    “Are these behaviors you want to see more of in your daily work?” I asked.

    Yes! Was the answer.

    “Do we now all commit to sticking to these behaviors today, in this simulation?

    Yes! Was the answer. Easy! See there’s nothing to this collaboration stuff.

    However …

    During the simulation it soon became evident that roles and responsibilities were not clear or not being carried out; it was not agreed which information was needed by the business nor when; goals were unclear, meaning that priorities were arbitrary; people did not understand procedures and did not ask for help; assumptions were made (“I told you …,” “I thought you meant …”); people taking responsibility were not respected (“Let’s stop and try and sort this out …”)— everybody just carried on running around! “Headless chickens,” said one delegate; people were not listening to each other; there was a lot of “assumicide”; mistakes were made and people blamed—there was no exploration as to why and what can be done to stop these  mistakes. Mistakes were often corrected by somebody else but without giving any feedback to correct future mistakes; agreements were unclear across the team; there was no end-to-end flow—work and information was ping-ponging back and forward.

    “All in all, just like reality!” declared the teams.

    See how easy collaboration is. Within one hour of making a list of desirable behaviors, they were totally, utterly and universally ignored. Nobody appeared to own these behaviors or to confront each other on them.

    After the Experiences in Simulation

    At the end of the simulation round we looked at the team scores (number of releases, number of successful releases, business value realized). It wasn’t a pretty sight. The CEO wasn’t happy. The team felt stressed, unhappy, frustrated and powerless.

    We explored what this had to do with “effective collaboration.” Had anybody used the list of behaviors? No.
    Had anybody consciously practiced these behaviors?
    No.
    Did anybody give feedback on these behaviors when they were clearly not done?
    No.
    “Giving feedback feels strange, it feels like we are criticizing and blaming,” said one delegate. Another assumption!

    “It’s hard,” said one delegate. “The manager needs to manage this,” said another.
    “It was YOUR list and YOU all agreed this, why is this a manager’s job to get you to do what you said you wanted to do?”
    There was silence. “It’s not easy, we fall back into old ways of doing things, we need help!”

    Is it a Bird, Is it a Plane, Is it a Bus? No, it’s a coach!

    As the simulation progressed, the team refined their list of desirable behaviors based upon their experiences. They were coached for two rounds and consciously practiced and experimented with the behaviors. The coach role became a critical catalyst.

    In the final round the team owned, and were executing on these behaviors without coaching.
    Why were they suddenly executing?
    They felt less stressed; they were trusting each other; they felt the positive impact on their work; they saw, felt and shared the value gained by these behaviors. They felt immediate positive consequences from these behaviors.
    Why no more coaching?
    The team were taking it in turns to lead their own stand-ups; they were giving feedback and correcting their own behaviors. These were the additions the teams made to the behaviors as they practiced them.

    • Transparent communication – “open and honest” and “frequent” (Relevant, and timely as agreed with the various stakeholders).
    • Feedback “asking for and giving feedback.” Constructive feedback. (Related to agreed behaviors and responsibilities, calling stop, then swarming to solve an impediment, constraint, or remove waste; immediate fact based feedback).
    • Shared team goals (and cross team). What are we aiming for? Impact on others and their goals. Maybe conflicts that need managing. Understanding strategic portfolio and how we contribute to strategy, show you understand and agree common goals and prioritize accordingly. Goals for both value creation (features and benefits) and value leakage (risks, debt).
    • Leadership within team – team members took turns in facilitating the stand-up and retrospective. This role was respected.
    • Listen – is it clear and understood? Show that you listen, sender: Ask for clarification, ask to confirm understanding, listener: summarize what was understood and agreed. Avoid assumptions.
    • Being open for dialogue, being approachable—not closing the door or hiding behind. All were engaged and equal in the stand-up and retrospective. All were asked for feedback, ideas, suggestions.
    • Giving and taking responsibility. Respecting team behaviors; e.g if somebody calls “stop,” or “respect somebody taking responsibility for an improvement stand-up.”
    • Sticking to agreements. Confront people to understand why agreement not met and what can be done about it in the future (e.g conflicting priorities, lack of understanding).
    • Central moment to get together (e-2-e). Scheduled stand-ups, retrospectives, and improvement prioritization.
    • Knowing the roles and responsibilities, sticking to responsibilities, confirming your responsibilities, confronting people on responsibilities (feedback). Showing that you take ownership.
    • Mistakes are OK – no blame, so long as it is used as an opportunity to learn and improve.
    • Be proactive to get information needed (or ensure information is available, e.g. visible management board, avoid unnecessary “disruptions).
    • Equal treatment: “Respect” ideas from ALL. Ask people for input, engage ALL in ideas generation and decision-making.
    • Don’t HOPE, confirm.
    • “Willingness.” Confirm and agree – “Willingness” and “ability,” empower to be able. Share knowledge, cross train, coach, help.
    • Ownership of responsibilities. Confronting people on responsibilities.
    • Clear agenda and purpose of meetings. My role, what is expected from me? What is in it for me? avoid “wasting” time in meetings.
    • Team coaching to give feedback on behaviors.
    • Create trust by demonstrating team behaviors.
    • Check understanding and commitment to “agreements” and confront people (feedback) on agreements.
    • Improving own work as a team.
    • Asking for and offering help.
    • Having insight, overview, visibility.
    • Limit WIP (reserve WIP for learning, improving, technical debt).
    • Pleasure – celebrating success. Product owner shared results including customer feedback. Team complemented each other.

    The teams were then asked to make a list of behaviors and actions they wanted to take away and apply. Their line managers and assigned team coaches were there when the list was made and gave commitment to help guide, coach and empower teams to experiment and practice these new behaviors in their daily work.

    The team would prioritize one or two behaviors to start with and consistently practice in the coming weeks. When the new behavior becomes the accepted norm and a “habit” they would focus on more behaviors.  This fits in with the Third Way of DevOps: Continuous experimentation and learning. “Understanding that repetition and practice is the prerequisite to mastery.” And to tie this back to Aristotle the philosopher (not the Google project): “Excellence is an art won by training and habituation … We are what we repeatedly do. Excellence, then, is not an act but a habit.”

    Meetings, Including Training Sessions!

    And finally, one of our industry’s most popular, and wasteful, examples of collaboration are poorly run meetings, which are the biggest time-sink in organizations and a source of frustration. I include some training classes as forms of meetings.

    In more than 100 training sessions in the last year, less than 10 percent have started on time.
    Nine minutes late is the average. Delegates say this is normal in their organizations. When I ask people what their own personal learning goals are for sitting eight hours in this “training meeting,” more than 25 percent had not defined this and more than 70 percent had not agreed this with the “sponsor” of the training.

    We need to manage our overload of wasted meeting effort. How? One way is the ask for goal and agenda in advance, asking what value you need to bring in and what value you need to get from the meeting; also to limit your time to the agenda items relevant for you or your team.

    Efficient collaborators decide when they do, or don’t, have unique value to add—“maintaining focus on what matters.”

    The same applies to training, asking, “Why am I going on the training?” “What am I expected to learn?” “What problem is this training expected to help solve?”  “What will I be expected to do differently after the training?” “How will the learning be transferred back to the workplace?” “Will I be mentored and coached?” “Will I be given opportunities to experiment?” “Will I get constructive feedback?” “How can we demonstrate that the new knowledge delivers expected benefits?” This is better than the most common answer, “I don’t know I was told to come to this training!”

    A training is also a form of collaboration between you, the trainer and the person sponsoring the training.

    There is an “I” in Team, after all. “I” am responsible for my effective contribution to team performance.

    — Paul Wilkinson

  • The Third Way of DevOps: From Knowing to Being

    The Third Way of DevOps: From Knowing to Being

    Why is the Third Way of DevOps so difficult to master?

    We all know the feeling. You’ve seen an opportunity to improve things and genuinely want to do something about is. But you soon feel the clarity and anticipation draining out of you. First there’s the pressure of the day job. Then there’s the inertia of your co-workers who are oblivious of, or indifferent or hostile to, your ideas. And finally, as if these concerns weren’t enough, you’re faced with the difficulty of unlearning old habits and developing new ones. You wonder why you bother. In a way, it’s a bit like a lottery: You buy a ticket fully knowing that the odds are stacked against you, but it’s the worth it for the uplifting short and illusory dream that it might just happen.

    This is, of course, all about the Third Way of DevOps—creating a culture that fosters two things: continual experimentation, which requires taking risks and learning from success and failure, and the understanding that repetition and practice are the prerequisites of mastery. It all sounds great in theory, but why is it so difficult to put into practice?

    Information Abundance  

    It’s not due to lack of knowledge and skills. We have more information available to us and more easily accessible than ever in the history of our planet. We can thank technology for that. But has it really made us more effective at what we do? If I know something and don’t apply it, how is that so different than not knowing it at all? If I know how to read but choose not to, how is that so different that not knowing how to read? If I don’t use what I know, how long will I retain it and how will I integrate it with the other things I know? Most importantly, can and will I apply what I have learned to make things better?

    Lifelong Learning 

    We have been learning and studying all our lives in an attempt to make things better: our ability to understand IT, frameworks, methodologies, capabilities, practices, work processes and, of course, tools. Consider how many thousands of hours you invested in school, self-study, read out of curiosity, listen to podcasts and do work-related training and certification. Most of my friends proudly describe themselves as “lifelong learners” and one claims the day they stop learning shall be the day they die!

    Training in Skills

    We’ve also invested considerable time and effort into improving ourselves so that we may work more effectively with others: communication skills, teamwork, shared rituals, structured problem-solving, leadership and coaching all fit into this category. Who (in IT) has not been a part of some Service Management, Lean, Agile, Scrum, Kanban or DevOps training? After all of our training, you’d think we’d be further along than where we find ourselves.

    Knowing and Understanding 

    Knowing and understanding are not the same thing—I can know the Four P’s of IT Service Management (People, Process, Products and Partners), yet not view it as a lifecycle model and understand that to obtain the benefits of this knowledge, my team must determine the roles of people and objective of work of processes and then implement tools to automate the processes enabling people’s roles and tasks.

    Understanding and Doing

    Nor are understanding and doing the same thing—I can know that structured problem solving is based on the scientific method and can be broken down into four stages (Plan, Do, Check, Adjust). I may also understand that structured problem-solving is preferable to the reactive educated guesswork my team engages in whenever it encounters a problem. But if we do not change our behavior when it comes to problem-solving, our understanding will not manifest as action.

    Doing and Being 

    Neither are doing and being the same thing—I can be doing something and still not have internalized it so that it becomes who I am. I can know the core principles DevOps (engage in systems thinking, amplify feedback loops, foster a culture of experimentation and learning). I may understand that all three principles must be applied to foster a sustaining DevOps environment. I may be even be holding initial planning sessions with my team during which we map out the DevOps value stream, identify bottlenecks, create feedback loops and introduce changes to create a create a culture of experimentation and learning.

    Moment of Truth 

    But what happens when we meet with our next P1 incident? Does the knowing, new understanding and behaviors get tossed aside while we fix the problem, or do we hold fast to the new way of doing things as we grapple to not only remediate the incident but to view it within the context of DevOps?

    So, What’s a Person To Do?

    The velocity and degree to which we can move from knowing to being determines how effectively we can apply what we know. Here are a few things you can do to smooth the transition from knowing to being and help put the Three Ways of DevOps into practice:

    1. See and feel the potential impact from moving beyond knowing to being
    2. Use the uplift to get motivated
    3. Take action by changing something within you and your team’s circle of control
    4. Reflect on the fact that you have realized potential (this is very powerful when you share it with your team)
    5. Check and adjust based on the outcomes of step #3
    6. Allow the feedback from the previous step to motivate you to carry on!

    Curiosity and Humility

    This approach requires a certain attitude. One of curiosity and humility. You need to be inquisitive so you keep striving to understand. And you also need to be unassuming and you see yourself as having plenty of room to learn and grow; being deeply (almost obsessively) interested in what’s going on and how things could be improved; trying to be aware of and to set aside any preconceived ideas—adopting a beginner’s mind.

    To Be Continued …

    In our next post, we’ll explore specific steps you can take to leverage The Third Way of DevOps and make tangible progress based on what you know and don’t know, understand and don’t understand, and do and don’t do.

    This article was co-authored by Mark Smalley.

    — Mike Orzen

  • Software Delivery: Shaking Loose of a Local Optimum

    Software Delivery: Shaking Loose of a Local Optimum

    This is an article about software delivery, that magical or onerous field between pushing code to version control and using it in production. Why does software delivery matter? Because software matters, and because software developers matter. I don’t want my software delivery to get out of the way; I want it to make me a stronger developer.

    The path from code to production is crucial because it determines the length of our feedback loop and, therefore, our rate of improvement and that all-important lead time.

    Our current tools think of delivery as generic—they take control, and we fill in the blanks. They let us build, test, deploy. But these words mean something different for every application, on every team and within every business. And there’s a lot more that goes on, or can go on, between the writing and the running of code. There’s often linting, approving and coordinating. There’s data to think about and security policies, business team structure and auditing. There are other steps we don’t bother with, because there’s no slot for them: updating libraries, fixing the code to meet standards, taking a different set of steps for a documentation-only change.

    What else have we not thought of, because it doesn’t fit inside the current set of tools? I want to think about delivery from the top. In doing so, I can gain a greater understanding of my work.

    We’ve seen major shifts in Operations technology: containers, log aggregation, observability, programmable infrastructure and Kubernetes. In my opinion, the killer feature of Kubernetes is its API—it is controllable by automation. It accepts instructions and sends out events.

    The potential here is even bigger than we have realized, for operations and for delivery.

    Modern options for operations have achieved new levels of abstraction and potential. Yet progress in delivery has only been incremental.

    Our tools are very configurable, with plugins and scriptable bits, and we can always fall back to running shell scripts of unlimited length. But are we stuck in a local optimum? Are we thinking big enough?

    What We Have

    Today’s popular software delivery tools are fill-in-the-blank. This is great for getting started, when you don’t know what you’re doing. What about when your business grows, when your delivery is no longer cookie-cutter? Let’s look at two examples.

    Travis CI is a favorite of mine because it’s so easy to get going. It will happily build any public GitHub repository, on its infrastructure, for free. If my repository fits a few simple profiles, such as a Rails app that deploys to Heroku, poof! Some very simple YAML in your repository:

    I like that YAML. It is declarative. It contains data. It fills in the blanks that Travis provides. But the real measure of a tool comes when you get off the happy path. You can do just about anything in Travis; just call out to a shell script. Here’s a small piece of one of my .travis.yml files:

    It has shell commands. Shell syntax for an if-condition. And that infinity of flexibility, a call to a shell script. That travis-build.bash script happens to contain four functions, eight conditionals and (if I’m reading it correctly) up to seven program calls.

    This is programming, OK? As soon as your YAML calls out to a shell, it is not declarative. It is imperative. Or a mix. 😕

    This is programming, but my YAML and Bash do not qualify as software. Bash is a scripting language. It is designed to glue programs together, not to construct programs; it lacks composable modules with encapsulation. I can live without types, but in Bash you don’t even declare function parameters! And it’s full of pitfalls. Use Bash for what it’s good at: calling other programs. As a fallback for flexibility that we lack at a higher level, Bash does not shine.

    YAML is not a programming language, although it does support variables and has its share of pitfalls. Until the YAML contains commands that get executed; then it becomes a program, with even fewer options for error-handling or clear naming than in Bash. Please use YAML for data, not for code.

    There is a gulf between programming and software. We have standards for software: it has design—domain-driven design that we’ve worked through with experts. It has automated tests. It has modularity. It uses libraries and abstractions. I want to use a language designed for all this.

    Skip ahead to Jenkins. This is more widely used than Travis for more extensive delivery flows. It can run on-prem. There are hundreds of plugins available. So many blanks to fill! So many options! And now, there’s pipeline as code. For one thing, you can write a piece of Groovy to define which repositories and branches get a pipeline. That makes Jenkins more responsive to new repositories and branches. It’s a step in the direction of responding to new projects.

    The people building Jenkins recognize that delivery needs to be controlled by the people who control the code. Instead of constructing each repository’s build in a GUI (hello, TeamCity), we put a Jenkinsfile in each repository. That file is in Groovy. Groovy is a programming language with all the modularity and libraries of the JVM behind it. Except, you don’t get all that in your Jenkinsfile. You’re limited to what is available when Groovy runs within Jenkins. If you want consistency from one repository to another, you can install custom-shared libraries in Jenkins and call them in each Jenkinsfile. Rolling out change is challenging; any of those disparate Jenkinsfiles can break after the shared library changes. It’s pretty fragile and still limited. It’s progress.

    This is a step up as programming, but I still wouldn’t call it software. When I write an application, I work in more than a programming language; I work in a language system. A language system includes dependency management, frameworks, linters, build systems, editors, community and the whole ecosystem of libraries available in the language. Great language systems exist around the JVM, Node and .NET, for instance. Groovy that you run can participate in the JVM language system, but Groovy running within Jenkins does not have this access. The Jenkinsfile DSL is still fill-in-the-blank.

    What We Need

    Conceiving delivery as software invites us, development team members, to sit down with the local experts on our deployment environment and security and other concerns. It invites us to have a conversation: What does delivery mean for this application, on this team, in this organization? Where do we want consistency and variability? Then we can create and encode a model—a model of the domain of software delivery, within the context of our business and our environment. We can brainstorm events such as a push or a new repository and what we want to always happen. We can divide the parts into team-specific, organization-specific and community-wide (open source).

    Instead of, “How do run the build? How do we run tests?” we can ask, “What needs to happen after a code push?” And we can think carefully about how we work, how our software works and how our business works with it. We are the domain experts in our own delivery.

    This is the top reason I want delivery as software from the top down. By thinking about delivery as software, we think rigorously about our work as software developers. And we get better at it.

    I want more than delivery-with-some-code. I want delivery as software. Controlled top-down by my team, with a rich ecosystem of libraries, with frameworks and services for the parts that are common in the community. I want delivery that’s event-driven, that’s testable and that communicates with me when it’s time for a decision.

    We’ve come a long way since the days before automated deployment. Our existing continuous integration/continuous deployment solutions have served us as far as they can. Incrementally better (fill-in-the-blank) won’t take us to the next level of software delivery, and it won’t help us reach the next level as software developers. It’s time to break out of this local optimum.

    Let’s Go

    To this end, we composed the Software Defined Delivery Manifesto. Say what you want about manifestos: They have a certain flair, a certain impact, a certain dramatic longing for a better world. That is what we are expressing here. Yes, it’s a better world only for development teams, and only for development teams in complex organizations. But we matter! Software has a huge impact on the world, and to make that software better, we need to get better at delivering it. Let’s push past what we’re used to. Accept that our job is not writing code, but running useful software in production. Let’s get better at delivery by modeling it in software.

    If you’re in, join the movement. Sign the manifesto.

    — Jessica Kerr

  • Roadblocks to Successful DevOps Often Human, Not Technical

    Roadblocks to Successful DevOps Often Human, Not Technical

    Usually, when people talk about DevOps, they focus on the technical aspects. However, in my experience we need to highlight what lies behind technology: the people. To effectively implement DevOps, we have to understand that many obstacles are not technical, but human.

    In working with various organizations in their DevOps journeys, I have seen six main roadblocks to a successful DevOps implementation:

    1. We mistakenly blame technical obsolescence: Tech teams should avoid justifying slow processes and lengthy deployments just because of the existence of old systems. To do the best with current systems, analyze your workflow and decide whether the problem is less a matter of technical tools, but one of attitude and organization.
    2. We don´t cooperate: DevOps doesn’t work unless teams can work together. There are three steps to building collaborative teams. First, work hard. Your good work speaks for you; it gains you the respect and cooperation of your colleagues. Second, highlight the contributions of others. Offering recognition shows that each member of these diversified teams is important. Finally, learn how other teams are being evaluated so that you better understand their behavior and can work with them in achieving real cooperation.
    3. Our egos get in the way: Ego leads us to believe that our approach and knowledge apply to every problem and that we can simply solve these issues. It prevents empathy and the ability to properly help to those who come from a different context or background. In DevOps teams where people come together from different perspectives and backgrounds, this is particularly important. So please check your ego at the door.
    4. We are unwilling to leave our comfort zone or the status quo: DevOps is about speeding up processes and providing new ways to work. As a result, DevOps teams should question traditional methods and understand that some rules no longer apply in the digital world.
    5. There is a lack of leadership: DevOps teams succeed under real leadership. Real leaders give visibility to everyone and understand that anyone can offer critical contributions, even when least expected.
    6. We suffer from cognitive bias: Cognitive bias comes from our background, our personal experiences and our careers. However, we shouldn’t let this condition us. It is essential to move forward and understand that our experiences and knowledge do not apply to every context. DevOps teams must be aware of cognitive biases to build a proper solution for problems that may arise.

    These six human aspects can often form obstacles to DevOps implementation. To help overcome these challenges, I typically recommend three broad strategies to organizations:

    • Technology is a means but not an end. It is important to understand that technology is what helps us solve problems, but it’s not the solution in and of itself.
    • Understand the problem and find balance. Avoid over-engineering; you do not need to prove how much you know about technology in general, you need to find the proper solution.
    • Be humble and curious. Technology is never static. There is always something new to learn. In the field of DevOps, it’s critical to constantly investigate and renew ideas.

    A successful implementation of DevOps is not only about automation. It is also about people who make decisions and use tools. However, for many enterprises, transitioning to DevOps represents a fundamental shift in how individuals and teams will have typically worked in the past. This is why I encourage DevOps teams to evaluate their workflow and be aware of aspects such as ego or biases, as they do affect performance. Handling these situations might require professional assistance. I suggest working closely with human resource professionals or turning to business psychologists to understand these human aspects. In turn, this will enable your DevOps teams to walk toward success.

    — Ernesto Cardenas

  • Too Many Projects: Breaking Initiative Overload

    Too Many Projects: Breaking Initiative Overload

    Initiative overload. Those words send a chill down the spine of IT leaders everywhere. Not only are overloaded teams juggling too many projects, morale is usually low, with long hours every week and no end in sight. Yet, leaders can’t arbitrarily kill or stall projects without suffering blowback from somewhere. If you’ve been tapped to lead a DevOps team and you see too many projects in play, breaking initiative overload is a top priority. Here’s where to start.

    It’s very common that highly productive DevOps teams suddenly end up with a list of 20 or 30 “priorities” and rapidly developing gridlock. As the team’s newly anointed coach, you get one shot to fix it. What’s your strategy? Addressing these objectives is critical in breaking initiative overload:

    • Prioritize the right initiatives.
    • Shorten the priority list, ideally to five or fewer projects.
    • Allocate resources so priority projects start and finish promptly.
    • Communicate and defend prioritization with a transparent metric.
    • Identify and plan the “next project up” as priority projects finish.
    • Empower the team with “not a priority right now” if approached.
    • Get buy-in and hold all team members accountable.

    Gather the List

    Sounds complicated, right? Keeping it simple is usually good advice. When you’ve got a long list of projects, first recognize they are not all created equal. Some are truly initiatives, such as that mandate from the CEO for tracking widget sales in real-time. Functional departments have their important projects, providing needed cost reduction or efficiency improvement. You’ll probably see a few wish list projects from various sponsors with weaker justification.

    Next, understand that every project carries a couple of hidden costs. One is the opportunity cost of assigning precious resources to it and not to something else. Another is the switching cost teams incur when they are multitasking too heavily. Neither of these costs can be eliminated; however, they can be optimized within the context of what resources are needed.

    So, make a list of every development project on your radar. List the untouchable initiatives with C-level sponsors. Add the functional projects and those pet project requests. Don’t filter the list at all—anything consuming resources or awaiting start needs visibility. Now, you should see what all the initiative overload chatter in the hallways is about. Take a deep breath, call your team leads together and order your preferred brainstorming chow for a meeting.

    Create the Plot

    Seated around you should be a mix of your best technologists and line-of-business experts. Look at the list together. Sponsors may have justified a project request without a real assessment of merit. If you need someone to speak to a project that team leads haven’t seen, bring in a departmental expert. If they can’t explain it or it’s an outdated request they want to revise, put it in a parking lot. When you’re sure your team leads all understand every project on the list, assign a letter designation for each project.

    On your whiteboard or shared screen, plot each project against two dimensions: implementation ease and business impact. Use a relative scale of low, medium or high for each axis. Implementation ease can incorporate many variables including cost of tools, technical difficulty, in-house expertise and more. Business impact can address revenue increase, cost reduction, competitive advantage or regulatory compliance. Don’t worry about absolutes; you’re looking to see how a project compares to all other opportunities. Adjust the pins if necessary after the first few projects are plotted. You’ll get the hang of it in just an hour or two, even if it’s a long list.

    Line Up Priorities

    Few projects are priority no-brainers with high business impact and high implementation ease. More likely, you have several projects in the “priority box,” scoring better than medium/medium. If you have more than five projects there, take another look and jostle pins a bit to get a manageable list. Make sure your available resources match projects identified in the priority box. Allow one exception: That C-level initiative may score as high business impact and low/medium implementation ease. Killing it just because it’s difficult isn’t a good move, but prioritization within context can help address resourcing.

    That’s the priority list you take back to your team. If it’s on that list, work it and finish it. If something isn’t on that list, your team kicks any inquiries on those projects back to you. Push projects with low business impact and low/medium implementation ease back to their sponsors for justification or removal. When one of the priority projects finishes, then and only then start the next best ranking project. Rescore the entire project list after the first group of projects is complete.

    Too easy? Here’s a recent HBR article, “Too Many Projects,” describing a more elaborate approach for bigger initiatives. You may need such in-depth analysis for a C-level initiative requiring more resources. For most projects on your list, more analysis takes substantially more time and likely produces the exact same prioritization as this faster method.

    The key to breaking initiative overload is starting and finishing the right projects. Creating a vetted priority list quickly and getting the team rolling again is well worth the price of an afternoon and some food. Morale will improve rapidly as your team sees progress and the business sees value faster.

    I’d enjoy your feedback on how this process works for you.

    — Don Dingee

  • Psychological Theories Behind Agile Software Development

    Psychological Theories Behind Agile Software Development

    As a software engineer who spent nearly 20 years working in social services—including working in institutions such as the U.S. Peace Corps, Child Protective Services, Chicago public schools and for governmental agencies in probation and parole programs—before making the leap into Silicon Valley, I found what helped ease my transition were my team’s retrospective meetings and the individual support from veteran engineers.

    This article is an attempt to draw from my clinical psychological background to make sense of the psychological theories of practice that I believe drive the success of any software development team’s reflective practices.

    Agile Software Development

    When I joined PagerDuty, I was immediately impressed by the implementation of Agile software development, the “you build it, you own it” philosophy, support of customer concerns and other diverse experiences necessary for companies and individual engineers to grow in today’s software market. However, I was particularly drawn to PagerDuty’s embrace of the Agile Manifesto’s last principle: “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.” At PagerDuty, we call these retrospective meetings.

    Our retrospective meetings embodied social and psychological theories of best practices so well, I realized that such meetings could help engineers, particularly those who find social interactions challenging, become exemplary leaders with interpersonal soft skills.

    Psychotherapeutic Theories of Practice

    Since many people in the software industry are likely already familiar with retrospective meetings, let’s dive right into the psychological theories underlying reflective practices rather than how retrospectives are practically implemented. The main theories of practice that I will focus on are psychoanalysis, vulnerability, stages of change, restorative justice and systems theory.

    Psychoanalysis

    At the root of all psychotherapeutic practice is some type of introspection that both the client and therapist together engage in understanding. Most notably, psychoanalysis attempts to discover and interpret past, unconscious or unknown thoughts, emotions, impulses and memories that currently impact the client’s thoughts, feelings, behaviors and entire life.

    Similar to psychoanalysis, successful retrospective practices reflect on the past thoughts and behaviors of the individual and team to make all group members aware of the things that perhaps only certain individuals were cognizant of. In Sigmund Freud’s work, free association allows individuals to express whatever comes to mind, without censorship or even organization. Engineering teams can adapt this method by encouraging free and uncensored verbal and written communication by making only certain information public and keeping managers from attending meetings.

    Vulnerability

    With the recent popularity of TED Talks, many have already been introduced to Dr. Brené Brown’s research on vulnerability. Dr. Brown discusses the power of connection in the human experience across all spectrums and how people with the strongest sense of connection believe that they are worthy of healthy human connection, which increases the courage to be authentic and vulnerable through risking rejection from others.

    In software development, reflective practice has the potential to cultivate a group that embraces, respects and supports each member’s vulnerability. According to Dr. Brown’s theories, a team will become more bonded, productive and creative when reflective practice supports members working through feelings of vulnerability, such as the uncomfortable feelings that come along with being dismissed, disrespected, not considered or facing difficult challenges that impact technical output, job performance and productivity. In contrast, healthy retrospective practices embrace and accept uncertainty, group responsibility and failure.

    Stages of Change

    Piggybacking off of vulnerability theory, the theories of the stages of change required for individuals to adjust habitual behavior is relevant because retrospective meetings should provide an atmosphere to “tune and adjust behavior” as needed. This model is relevant to the many successes of 12-step support groups for individuals recovering from trauma and addiction. It also helped me understand why I witnessed so many people continually returning to maladaptive behaviors. The 12-step programs and other theories of change suggest that, to make positive changes, individuals need to accept and move out of denial and into understanding of their problematic and unmanageable behaviors.

    As stated previously, healthy retrospectives will embrace uncertainty and failure, allowing individuals in supportive environments to face and accept the behaviors requiring adaptation. Denial can be a form of self-preservation, especially for individuals who believe they are powerless to make changes; however, denial will also lead many software engineering teams into further problematic behavior if retrospective groups do not have proper techniques for confronting maladaptive behaviors that need to be changed.

    Restorative Justice

    Restorative justice models of practice focus on both the offender and the victim (and sometimes their respective communities) to come to an agreement about the response to violations that will restore balance in all parties involved. For more on restorative justice theories, you can read this comprehensive review of research on restorative justice models in schools, published by The WestEd Justice & Prevention Research Center.

    While (most) engineering teams have not dealt with criminal activity—which is one of the main use cases for restorative justice models—they can still exhibit some components of restorative justice theories when processing events that caused disturbances in team productivity or emotional well being. I worked with restorative justice models by being a facilitator for “Peace Circle” groups, which have outlines for planning actionable items that all members support and by holding everyone accountable to follow through on these items in upcoming retrospective meetings. By focusing on recurring events or events with the most impactful and widespread consequences, teams can prioritize inclusivity of all participants, which is also imperative for social justice models. Healthy retrospectives will come to a common understanding of the group’s goals and help those in disagreement come to a way to accept group norms.

    Systems Theory

    Systems theory, often considered a cornerstone of social work, suggests that a person’s psychology and behavior often affect and are affected by the individual’s family, community, work and other societal systems. Being from Chicago, a city with a population consisting of roughly 30 percent African-American and 30 percent Hispanic people, I was taught how the white majority that I’m a part of implements societal and systemic factors that impact minority lives.

    To help understand systems theory, consider Dr. Deborah Prothrow-Stith’s research on violence that parallels current black urban violence with Alexander Hamilton’s death in a duel. She discusses how all the factors that reformed white men’s dispute resolutions (such as wealth, legal counsel, alternative means to power and criminal justice interventions) were entirely unavailable to black youth. In fact, the net asset difference between white and black families has increased since the Civil War. Despite what most perceive as advancements in equality and civil rights, darker-skinned groups have relatively less wealth compared to their white counterparts.

    These observations are important because they provide systemic context to individual behavior, which is primarily attributed to the autonomy of an individual. Many systems will continue unchanged if their norms are attributed to individual acts and the policies in place are not questioned. While individuals are often blamed for their behaviors, retrospective meetings acknowledge that teams influence individual behavior and that the entire team takes responsibility for the system and individual experiences. Successful retrospective meetings will provide a team the chance to change systems rather than placing the responsibility for systemic changes on any one individual.

    — John Coleman

  • Life and Death in the Age of Automation

    Life and Death in the Age of Automation

    Has automation changed the way we perceive interaction with others?

    A few weeks back I got an email from an old friend, Jim. Typically, that’s no big deal; I get emails from old friends all the time. Except, in this case, it was different. According to a posting on Facebook, Jim passed away about a year ago.

    Another friend who is still among us reports having a similar experience of receiving emails from those who are dead. It’s an eerie event that’s becoming more common. Bad actors hack into a data store and pull up some email addresses, which they blast about nefariously to unsuspecting recipients. It’s a strange, almost morbid experience when you’re on the receiving end.

    But it’s gotten me thinking: What does it mean to be alive on the internet? And, more importantly, how much of what we perceive on the internet as a human really is? Jeepers—for all I know, the Facebook post could have been a ruse. Maybe Jim uploaded his consciousness into eternity and he really is trying to make contact. How would I know it otherwise?

    Allow me to elaborate.

    Technology that Captures the Moment

    Mankind has been using technology to represent reality since early caveman figured out how to draw pictures of hunters and animals on the walls of caves. The more real a depiction seemed, the more valuable it was. A sign of significant wealth was having the wherewithal to pay an artist to paint your portrait. The painting passed your likeness onto future generations. Your posterity knew what you looked like. The poor just drifted in away into imageless anonymity.

    However, whether you were rich or poor, there was one thing that technology couldn’t do: It couldn’t capture the moment. A painting took weeks, maybe months to complete. And, no matter what, the rendering was an interpretation.

    Photography changed all that. It captured the moment, which was a first in human history. And it democratized portraiture. In practically no time at all, it seemed that just about every town and city had a photography studio. Affordable photography made it so images of mothers and fathers could be passed on—future generations had a very clear idea of what great-great-grandma Mary and grandpa William looked like.

    Portraits of George Washington left a lot to the imagination. Photographs of Abraham Lincoln left no doubt.

    Figure 1: Portraits of George Washington are renderings of the artist’s perception created over time, not in the moment.

     

    Figure 2: Photographs capture a moment in time. Abraham Lincoln in 1858 (left). Abraham Lincoln in 1861 (right).

    The first time that 19th century former slave, abolitionist and author Frederick Douglass had his photograph taken was in 1841. By the time of his death, he was the most photographed man in America. Douglass understood the power of the technology both in terms of information representation and dissemination. Photographs of Douglass made his humanity real at a time when a good portion of the population of the United States was deemed subhuman.

    Figure 3: Frederick Douglass, born a slave, was the most photographed man in 19th century America. The reality that the technology revealed confirmed his humanity.

    The power that photography brought the technological and cultural landscape created trust in the image. The veracity of information in a newspaper article could be argued, but the image the camera captured represented reality beyond a doubt—at least until the practice of doctoring photos to misrepresent reality came into practice.

    Reincarnation as Seen on TV

    And then came television, which made it possible to represent reality as a stream. Whereas a photograph captured the moment, television captured as many moments as the viewer had to spare. But, it was a trade-off. Television distorted perceptions of life and death. Before TV, no child ever had the experience of viewing an actor portraying a character who died in an episode of “Gunsmoke” one week, only to reappear as another character on another TV show the following week. Death became impermanent. Also, the line between fact and fantasy blurred. The same television screen that brought the nightly news also delivered episodes of “Star Trek.”

    In terms of time and space, the only “real” thing is the TV set. Everything else is content. Sometimes you could tag the content as “fact” and other times as “fiction.” Or, you could tag it to a third category, “I’m not sure,” which gave rise to the harebrained idea that the 1969 moon landing really happened in a TV studio at a secret government location.

    Now, here’s where it gets really interesting. In the past, most people interacted with others most of the time in close proximity. You went to Grandma’s house for the holidays. You bought your groceries at the neighborhood supermarket. You sat in classrooms under the supervision of one or many teachers for at least 12 years, maybe more.

    That was then and this is now.

    Today, visiting Grandma for the holiday might be nothing more than a call on Skype. More people are making essential purchases online. The online class is replacing the brick-and-mortar school building.

    The fact is, we’re spending a lot of time in an environment that is essentially representational. Every day, more of our interactions with the “world” take place on the screen of a smartphone, tablet or a desktop computer. In some cases, it’s nothing more than talking to a device by name, as in “Alexa, what time is it?”

    Is it Human? Does it Matter?

    And thus we have the problem of distinction. We’re at the stage now where we’ve come to accept that some of the interactions we have online might be with a human and some interactions might be with a machine powered by AI. Sometimes it’s easy to tell. After all, I know there’s is no miniature person living inside my Echo device. And I know there is no little man inside my refrigerator who turns on the light as I open the door. This is low-hanging fruit; it’s obvious what the machine is. Still, in the long run, how can we tell? And, even if we can’t, does it matter?

    These days I have a lot of interactions with people on video conference and on the phone. I estimate that of all the people I interact with regularly, I’ve shaken the hand of about half. Twenty years ago, I physically touched nearly all. Such is the way of progress.

    I’ve had the benefit of living in a time in which the person came before the photograph. Now, we live in a time in which the photograph can come before the person. We think up the ideal actor and send it over to the special effects department for rendering. Today, we interact with images that have no human origin and yet behave in a way that is indistinguishable from a human. For those born today, it’s entirely possible that most of the meaningful relationships they have throughout their lifetime will be with non-human intelligence. Yes, there is a good argument to be made that people will still gather together in real time for concerts, religious services and sporting events. But consider this fact: 73,000 people attended the 2018 Super Bowl in a stadium, in real time. However, 103 million watched the event on TV. The “attendance” at the digital representation of the event exceeded the real-time experience by more than a factor of 10. And, companies paid millions of dollars for advertising time during the televised game. Today, that which we perceive as real is just as meaningful as that which actually is real.

    Which brings us to life and death in the age of automation. Unless Kurzweil proves right and uploading ourselves to a computerized host becomes possible as the Singularity approaches, all of us will die. And yet the images, both the ones we make and the ones that were made for us, will live on forever. Will these images take on a life of their own? Will we be able to use artificial intelligence to inject these images with the behavior that made us us, using information about us that was gleaned by observing every aspect of the online interactions we had over our lifetime?

    I’m not sure. But the possibility has stopped me from replying to that latest email from my friend, Jim. To be honest, I’m afraid of what would happen if I did.

    — Bob Reselman

  • DevOps is Failing, But Here’s How to Succeed

    DevOps is Failing, But Here’s How to Succeed

    As the Data Economy continues to gain momentum, many companies have turned to DevOps in hopes of achieving agility and velocity to compete and win. Whereas it’s no secret that DevOps can have a major impact on agility, what’s not widely known is that most DevOps projects actually fail. In fact, a simple search of “DevOps failures” returns a range of recent results from minor issues to major fiascos. So, what are the warning signs that an initiative is failing to live up to expectations, and how can you fix it before it’s too late?

    Focus on People, Not Tools

    There are several types of potential problems that undermine the efficacy of DevOps, and the most common one—which is the easiest to overlook—is failing to recognize that DevOps is a changing of processes. So, as with any kind of process change, it can only be effective when it’s embraced and adopted by all people involved. Organizations that are successful are able to not only shift processes, but also win the hearts and minds of workers and get their buy-in early on.

    When leaders approach DevOps as a tools-first methodology without addressing outdated legacy systems or the underlying IT cultural issues required to make newer tools work for their organization, projects have a tendency to either fail or fall short of expectations. In today’s competitive digital business landscape, DevOps teams need to be able to deploy multiple test environments multiple times a day to collaborate and test effectively.

    But deploying DevOps technology is often the easiest part. The real problem is trying to use the same tired data management techniques with those new tools. If marrying new tools with old structures (or systems) were simple, most companies today would already have DevOps approach in place. For initiatives to be successful, though, requires a significant shift in thought from leaders and workers alike to achieve both short and long-term development goals.

    Slow-to-Change Culture

    DevOps is all about using rapid iterations and collaboration without requiring formalized meetings to move forward, broker alliances or seek permissions. Given that DevOps is a people-driven process at its core, you know you’re in trouble when your teams are holding routine meetings lasting 30 minutes or longer. If you are doing this, it means the organization is still operating in a siloed manner, which undermines DevOps efforts.

    Creating a successful initiative takes clear communication, because even though everyone wants the same result, they often want said result on their own terms. Introducing DevOps efforts into a well-established organization isn’t easy, and often there can be culture shock and even pushback from rank-and-file workers all the way to mid-management or senior leaders.

    With that in mind, to build a culture that enables agile development, it’s important to communicate objectives clearly and act with discipline early. There should be a well-defined explanation of the objectives, metrics and goals by which your DevOps initiatives will be measured across the organization. It’s not enough to adopt DevOps practices just because it’s the buzz right now; rather, you need to be able to understand how it can help you rebuild your business process and achieve specific development goals—or else your success is not likely.

    You can’t evolve a company culture overnight, but with the right approach and commitment to the DevOps journey, you can lay the groundwork for organizational change and achieve more agility.

    Inefficient Workflows (Data Friction)

    DevOps initiatives are often dependent on applications, which are powered by business data, but getting that data to the right teams can create an enormous amount of drag on the workflow. We often call this data friction. This is a prime example of data friction, or the tension between demand for data access and the constraints that stand in the way.

    This commonly leaves DevOps teams in one of three places, each with their own drawbacks: using stale and polluted data sets, stopping Dev and Test to wait on data provisions and refreshes, or just developing on purely synthetic data (punting the data problem to Test). To avoid returning to the same constraints, we need to rethink the way our teams access data in a way that minimizes the current friction so that we’re no longer choosing between quality, security and agility.

    Data should flow securely, freely and at lower cost, eliminating the friction around data. Synthetic data should be used for development because it is the correct data to use for development, not simply because it is the fastest. Taking a DataOps approach—tackling the data constraints around people, process and technology—is the only way to ensure you are eliminating data friction, rather than shifting it to someone else.

    Conclusion

    Companies set themselves up to repeat mistakes when they try to throw even more technology at the problem, without considering how existing infrastructure, culture and tools will work together in a DevOps strategy. At the end of the day, a tool is only as good as the teams who use it, and DevOps as a methodology should always be considered a continuous cycle of improvement. It’s time to broaden our perspective on the aspects of data management that we can improve upon to also include the people behind the tools. The future of DevOps depends on our own team’s flexibility and willingness to continually improve our own processes.

    — Adam Bowen

  • Steps on the Journey to DevOps

    Steps on the Journey to DevOps

    There was a time when people thought DevOps was a simple movement that didn’t need to be defined. I have always been told that big leaps won’t get you to your destination immediately, but little steps will get you there one day. For DevOps, the day has come, gaining mindshare among organizations and given importance like never before. However, certain companies are unable to understand what’s going on in their respective industries or how one they make their digital operations more productive.

    The Evolution of DevOps

    In 2007, Patrick Debois, a Belgian engineer, saw a growing friction between developer and operation teams. As a developer, he understood really well that development and operations can never go hand in hand, but he saw a golden opportunity to make the most of agile methodologies to manage infrastructure management processes in the similar way developers do. Initially being described as agile infrastructure, the concept was later coined as DevOps—a portmanteau of development and operations.

    A year later, in 2008, O’Reilly Media ran the very first Velocity conference, founded on the same insight: software developers and web operators never being on the same page even after sharing the same goals and language. The effort was meant to gather the tribe into one room, to talk with each other and share insights. It is said that much of the DevOps movement grew out of the Velocity conference.

    With the internet growing rapidly and sites such as Amazon, Google, Flickr, Facebook and YouTube becoming monstrous entities, those in IT witnessed seeing websites grow from one server to a dozen and even thousands. As a result, the business consequences of downtime, or of poor performance, skyrocketed.

    The need for DevOps grew with the growth of websites. And to blur the boundaries between development and operations, DevOps came into existence. Today DevOps became more than an idea. But unfortunately, the shift hasn’t been easy for everyone. DevOps has been focused too heavily on development and operations hasn’t enjoyed an equal share of improvements.

    5 Stages of DevOps Evolution

    Normalize the technology stack: The process starts from here. Several teams have started adopting true agile methods and are implementing proper version control as they seek to provide continuous integration and delivery. In fact, this is when redundant technologies are eliminated and the stack is normalized, which means practitioners must build on a standard set of technologies and place application configurations in version control.

    Standardize and reduce variability: As noted, development and operations teams are working toward one goal. This means teams must work to make sure their technology is further consolidated to a single OS or OS family, process complexity is reduced and early collaboration opportunities are explored. This is also when system configurations should be placed in version control and applications should be rearchitected to fit business needs.

    Expand DevOps practices: At this point, all the foundational pieces should be in place. This phase is a cleanup phase for discrepancies created by previous changes. For instance, the output of the application development team must match that of the delivery for true effectiveness. At this point, individual contributors can do work without approval from outside the team and changes can be made without as much wait time.

    Automate infrastructure delivery: At this stage, a company can help lessen the discrepancies between development output and operations delivery times. As a result, everyone gets closer to being one on the same page. And things such as security configurations, system configurations and provisioning are automated, which means the DevOps team can deliver faster and is better set up for future self-service.

    Provide self-service capabilities: DevOps deployments are created here. Developers can deploy testing environments on their own, and success metrics are clearly visible to the entire team.

    Helpful Resources:

    — Jaymin Vyas