Category: Enterprise DevOps

  • Always-on: The New Normal

    Always-on: The New Normal

    One of the many challenges of DevOps is finding great team members that can handle the always-on nature of the job. Nearly everyone at VictorOps has been “that person” and has done the weeklong stint of carrying the virtual pager.

    What we noticed in our years of doing that job is that team member behavior tends to be polarized. If it’s your week to be on-call, then you are “all in”. You don’t venture far from home, you can’t attend special events and you have to apologize to your significant other a lot. When you are not on-call, you tend to unplug from the company. These are normal behaviors and completely understandable given the difficulty of the job.

    The problem is that this on-call way of life is not conducive to actually fixing problems faster. In a perfect, company-centric world, everyone would get paged for every problem with the logic being that one of the team members actually has the information needed to solve the problem. But that’s not going to happen. And unfortunately, the truth of the situation is that you’re not sure who that one person with the correct knowledge to solve the problem will be.

    The question then arises…how can team members have more knowledge about what is going on with the complex systems in their architectures while also maintaining their quality of life?

    Consider Twitter and Facebook user behavior. I have a cousin that lives in Chicago. I have seen her in-person once in the last 30 years, yet I know she remodeled her living room, I know about her work and I know where she took her last vacation. I know these things because the Twitter and Facebook experiences leverage something called Continuous Partial Attention, a term coined by Linda Stone in 1998. Basically, I pull my phone out of my pocket, swipe once, and gain a disproportionate amount of information for the effort (or conscience thought) I put into it.

    If you’re in DevOps, then the idea of Continuous Partial Attention is something you’re already living daily. Now imagine putting CPA to work for your team. Everyone can have an eye on the infrastructure, but not “feel like it”. The on-call team member is still responsible but if everyone has an idea of what is happening, the likelihood of someone knowing the answer is revealed sooner. Now everyone wins because the problem is solved faster, getting the team back to their lives and the company back to revenue generation.

  • In the DevOps Journey, Culture Comes First

    In the DevOps Journey, Culture Comes First

    Etsy, the global marketplace where people sell handmade goods, vintage items, craft supplies and more, has grown by more than 10 times since Chad Dickerson joined as CTO in 2008.

    While the organization’s success is obviously the result of numerous factors, it’s no coincidence that upon arrival, Dickerson initiated a significant culture shift in how the company’s development and operations teams deploy updates to the front- and back-end of the website.

    What kind of culture shift? If you saw my title in the byline, you probably know that I’m talking about DevOps. But before we get into why culture is so important to DevOps, let’s back up a bit and make sure we’re all on the same page about what DevOps actually means.

    At CA Technologies, we define DevOps as best-practice application delivery methods that improve communication, integration and collaboration between development and operations. In layman’s terms, this means development and operations teams work on software releases in tandem, rather than throwing code back and forth across a silo. The result? Their collaboration helps companies release high-quality software and services to market faster and at lower cost.

    The “why” of DevOps is simple: In an ultra-competitive landscape where companies are racing to release the next big thing, software quality and speed-to-market are essential for achieving business goals and creating differentiation. It’s the “how” of DevOps that can be confusing. So let’s talk about where that begins – namely, with culture.

    A Vision for Transformation

    When it comes to something as significant as a culture shift, it’s not about just getting development and operations teams in the same room for a few meetings. We’re talking about breaking down long-standing silos and changing the human fabric intertwined with them – an undertaking that requires both top-down and bottom-up strategies for change.

    I talked earlier about the change Chad Dickerson brought with him when he joined Etsy as CTO. In a blog post from his early days in the position (he’s now the company’s CEO), Dickerson talked about the importance of “intellectual honesty,” which he described as “admitting when something is not working so you can work together to figure out how to fix it” and “measuring your successes and failures so you can properly reward for excellent work and learn from your failures so you can strive to never make the same mistake twice.”

    It’s clear from his words that Dickerson had a high-level vision almost from day one for how to transform Etsy’s IT operations, even as he admitted that there would be “bumps” along the way. This kind of vision is essential for driving the DevOps journey and keeping it on track as opposition, challenges and other issues crop up. It’s been said before that DevOps takes a village, but you can’t have a village without a chief.

    Populating the Village

    Speaking of the village, if the chief is supplying the top-down vision, it begs the question, who is going to carry that vision out? This can be tough to answer, especially in organizations where development and operations teams have been entrenched in siloed practices for decades.

    In my own experience, I’ve seen the village successfully get on board the DevOps journey when its involvement begins as a grassroots movement and grows with small wins over time. In other words, the chief pulls together a group of IT professionals with similar characteristics – think driven, highly skilled, unafraid of change – and they work to transform just a single deployment scenario (e.g., pushing out Web updates). At the end of that experiment, the team shares its experiences, challenges and successes with the larger IT group in the hopes of bringing more people on board and moving up to more significant deployment scenarios. Then, it’s rinse and repeat until the culture shift has occurred.

    It’s worth noting that in situations where the development and operations silos run especially deep, a rip-and-replace option may be the only viable one for achieving DevOps. In such cases, leadership has to put forth an edict saying “this is the way it’s going to be” and expect to deal with some attrition, as there will be employees who simply refuse to change.

    A Formula for Change

    We’ve talked about the chief and the village, but I’d be remiss if I didn’t touch on the nuts and bolts that are necessary for engineering such a culture change. Yes, top-down vision and buy-in from the bottom up are important, but so are the new tools, skills and processes required to support that change.

    There’s a reason the word “journey” is so often used with DevOps – it’s a never-ending cycle of optimization among the people, processes and technologies driving development and deployment operations. So as you evaluate your own culture and think about the next stop on your DevOps journey, remember this formula: vision + expertise + tools = change.