Category: Doin’ DevOps

  • Does DevOps Improve or Undermine Quality?

    Does DevOps Improve or Undermine Quality?

    In my earlier blog, “From QA to Continuous Testing,” I explained that “several industry developments converged to make continuous testing the winning method for software engineering quality assurance (QA).” The “combination of improved test methods, agile processes, continuous integration speed afforded by automation and orchestration tools and low-cost test dynamically configurable infrastructures afforded by virtualization has resulted in DevOps.”

    However, a recent survey by Parasoft indicated that some important concerns such as non-functional requirements (NFRs) (examples are: reliability, testability, availability, resiliency) are being overlooked in many organizations that are operating fast-paced Agile and DevOps environments.

    Some results gleaned from Parasoft’s survey run counter to popular expectations:
    1. Agile teams are less likely to monitor NFRs than non-agile teams.
    2. Ownership of quality, especially for agile teams, tends to be QA team members.
    3. Business members think NFRs are being measured but technical folks do not.
    4. Confusion and disagreements exist regarding who owns NFR objectives.

    The report implies that quick Agile and DevOps processing cycles steal time from broader QA concerns.

    The Question

    So what is the truth? Do agile, DevOps and continuous testing improve QA, or do they undermine quality?

    My own observation is that DevOps can improve or undermine QA. The outcome depends on whether an organization explicitly takes a holistic view of quality that assures all of the “ilities” are taken care of during agile, DevOps and continuous testing.

    A complete DevOps environment requires a more sophisticated view of continuous testing than simply “shift left,” “fail early,” “fail often” and “test fast,” which are four popular tenets espoused by the DevOps community. A fifth tenet, “relevance,” must also be fully considered. With “relevance” as a tenet of DevOps QA, the entire DevOps and continuous testing pipeline has woven into it, from end to end, the concept that all of the quality aspects relevant to customers are emphasized in continuous testing and analysis of test results. When relevance is fully considered as an equal tenet to the others, then NFRs and “ilities” are not forgotten and the tools, goals and tests explicitly take into account all aspects relevant to customers’ perception of quality. What is more, when relevance is applied to continuous test selection and results analysis, the system not only produces better quality but it also is more effective and efficient.

    At Spirent we think that a holistic approach to DevOps and continuous testing, including “relevance,” is critical for successful DevOps. When continuous testing is implemented with relevance as an equal tenet, quality is improved.

    You can read more about our views at Spirent.com/solutions/DevOps

    What are your experiences and suggestions?

  • DevOps Without Scale, Part 1

    DevOps Without Scale, Part 1

    When most people think of DevOps, what comes to mind first are continuous integration (CI) and continuous development (CD) pipelines. While they are arguably the most important parts of DevOps, they are not all of it. So, then, what is DevOps? It is a philosophy of how to build and operate software products and systems to provide the best user experience. It’s a way of thinking that blends the previously independent disciplines of development and operations so that software products can both evolve functionally and be stable operationally. In practical terms, it means development teams are involved in operations and, in many instances, fully own the operation of their products.

    The top companies are already there because they otherwise couldn’t deliver their massive and feature-rich products with 99.9 percent uptime. But there are still too many companies and developers who aren’t “doing it.” And it’s typically reflected in product quality.

    It seems DevOps has been stigmatized as difficult, requiring expensive tools and generally classified as needed if you’re big. In this blog I’ll walk you through different aspects of DevOps and share the business value behind each, the common implementation techniques and the free (or inexpensive) tools that will let you jump-start your DevOps in a matter of weeks.

    Product Health: The User Experience

    Our goal as engineers is to build products that will be used by others. Typically we call them users and sometimes we have a strenuous relationship with them. But if users don’t like our products and aren’t using them, we have failed at the very core of our job. So what do users want—besides features, that is?

    Users want products that are reasonably fast and work when they need them. Your site going down is an obvious example of why it’s important to be aware of your product’s health. But there are more nuanced examples. Sometimes a new release breaks an edge case and isn’t caught by regression testing. Sometimes a service becomes slow—it still works and nobody is complaining, but it’s dangerously close to being unusable. You need to be aware of your users’ experience so you can be proactive in fixing problems and delivering quality products.

    Thus, the first set of metrics you need to measure are user experience metrics. These typically include success/failure and latency of all external endpoints: REST, web pages, etc. If the users are external, you want to measure the experience from their point of view, which includes the network latency and HTML/CSS/JS execution time for web pages. If this was a car, these metrics would be your low coolant red light. If the light is on, you must stop as soon as possible and add coolant before your engine block cracks from overheating.

    To measure and monitor customer experience for web pages, Google Analytics is by far the most popular choice. It’s easy to set up and gives you exactly what you’re looking for: error stats per page, page load times, including individual page components and even browser parsing time for HTML. You can slice and dice the data by geographic region, browser type and so on to zero in on improvement or problem areas. Google Analytics is a great tool for tracking other product aspects as well, and you can kill many birds with one stone by adopting it.

    However, Google Analytics will provide data only as long as there are users and your site works. But what if the site breaks, maybe partially or maybe only for a certain geographic region? And what do you do for public APIs that aren’t instrumentable with Google Analytics? You’ll need a synthetic monitoring service. These services emulate clients by calling your APIs or opening your web pages in an automated fashion. They are typically scriptable and run from multiple places around the world. They can tell you when your services are up or down and can measure (emulated) user latencies by geographic region. Pingdom is one of the newer generation of such services that delivers what you need for a modest price. It also integrates with a graphing and alarming tool called Datadog, which I’ll talk about later.

    Product Health: Technical Innards

    Now you have a solid understanding of your user experience and can detect user problems in real time. But how do you find the root causes? And can you do even better by detecting user problems before they happen? What you need to do is go a level deeper and monitor key technical aspects of your apps so you can correlate them with user problems. The technical metrics are important to watch because they can be early signs of trouble, which can be averted if you act quickly.

    This can be a dangerous territory as the first desire is to monitor everything. Please keep your cool here, lest you drown yourself in hundreds of metrics that nobody understands and miss the important ones. In software, failure typically happens at integration points, so that’s what you should focus on: database and cache interactions, calls to other services, disk operations, queue sizes (for pub/sub architectures) and so on. Typically you don’t need to instrument business logic, as it either works or doesn’t and problems are caught in testing. This is where it is easy to create many metrics in a blink of an eye. A notable (and rare) exception is load-intensive parts of code that you want to monitor for performance reasons.

    If you see latency in database calls creeping up, cache hit ration going down or thread pool or queue size growing, you can frequently identify and fix the problem before it affects the users. Going even deeper, you want to keep an eye on the virtual machine (VM) and server performance. If your garbage collection times are increasing, if your thread count grows or if your server is showing signs of strain, it’s time to pay extra attention. If this was a car, these would be your check engine yellow light. If the light is on, you need to check it out soon but can keep on driving for a little bit longer.

    The most popular way to measure application metrics is with code instrumentation. Presently, Coda Hale (a.k.a. Dropwizard) Metrics framework and its ports to most popular languages are the way to go. Given the abundance of documentation and integration with all sorts of systems, this isn’t difficult. There are a plethora of coding patterns or external libraries you can use to avoid writing boilerplate code. Note that you want to include as much as possible in measuring requests (serializing, logging, etc.) so you need to put this logic as high up the filter chain or middleware as you can.

    The last piece of the puzzle is VM and server monitoring. Traditionally, this has been the territory of specialized ops tools, but Datadog, a service that I will talk about in the next section, provides this out of the box. Its agents run on each machine or VM and collect server metrics, JVM/JMX data, IIS info (yep, it supports even Windows) and more.

    Processing and Displaying the Data

    Now your code is instrumented and generates all sorts of interesting data. What do you do with it? And where does the data go in the first place? All this data is useless if it just sits there. You want to be able to look at the trends, correlate them and get a basic set of statistical information such as averages, means, standard deviations and so on. You also want to see pretty graphs (who doesn’t?) and be able to create dashboards. This is the cool part, where all the previous work comes to fruition.

    Datadog is a great way to start. It provides everything you need and more. All you have to do is install its agent on each box and configure the Metrics framework to send data to Datadog. Most Metrics ports already include Datadog adapters so there is no extra coding required.

    Now you have everything and can slice and dice the data whichever way you want. You can export data for further analysis. You can check it periodically for problems. You can even display dashboards on big screens around the office (looks cool, eh?).

    Putting it All Together

    I have seen all of this put together in a couple of weeks. However, if this whole area is new to you it might take longer, as you’ll be learning the concepts along with the tools. Sometimes the deadlines are tight and even a few weeks is a luxury. However, launching a product without any insight into user experience or product health is not a good idea. If I had limited time and had to prioritize, I’d start with instrumenting user experience metrics and hooking it up to Datadog. Even for web pages you can start with only monitoring the REST services that power them, or the controller layer, if you’re on model-view-controller. Soon you will realize how nice it is to have hard metrics and will want to add Google Analytics and deeper code monitoring to your product.

    All of this might seem like a lot, but you’re only dealing with four tools here: Pingdom, Google Analytics, Metrics framework, and Datadog. If your product is an API then you don’t need Google Analytics and if your product is internal, you don’t need Pingdom.

    As I mentioned earlier, Metrics framework is ported to multiple languages already. But even if you have specific requirements that aren’t supported out of the box, it’s easy to extend it. For example, at Guaranteed Rate we wrote and open-sourced the Metrics/DataDog adapter for .NET and a library for code instrumentation and DataDog integration for Clojure.

    .NET is particularly known to be behind on DevOps. Until recently one of the reasons was the lack of tools; however, that’s not the case anymore. All tools mentioned here, as well as Part 2 and 3, work in .NET and have adapters and ports where needed. With .NET entering the new open-source age, there are no more excuses to stay behind.

    Give yourself two weeks, read the docs, go one step at a time, and you’ll be surprised how far along you can get!

    What’s Next?

    Part 2 will cover what you can do to maximize product uptime based on the data you now have. It also will discuss making sense of the data, automated alerting and troubleshooting when the inevitable production problems happen.

    About the Author/Jack Korabelnikov

    JackKorabelnikov_avatar (small)Jack Korabelnikov wrote his first computer program when he was 10. Another 15 years later, after success in the role of senior engineer at Orbitz.com he transitioned into management. He still enjoys the tech side of things and continues to tinker with cool new tech. He is presently the VP of Engineering at Guaranteed Rate, leading the company’s technical evolution and the buildout of next-gen mortgage products.

    Prior to that, he worked at Orbitz Worldwide in multiple business areas such as hotels, private label and B2B web services. He played a key role in rolling out agile and, most recently, was the main driver behind the continuous delivery and DevOps adoption across the organization.

    jack@korabelnikov.com | LinkedIn

  • Culture Change: Metrics, Tools Make a Difference

    Culture Change: Metrics, Tools Make a Difference

    Change is in the wind.

    This time of the year, as winter weather gives way to springtime renewal, this concept isn’t a cliché but rather a concrete, visceral experience. You go outside, smell the earth defrosting and know for certain that change is happening all around you. You can feel it.

    Such is the case on the current DevOps landscape. Seemingly every conversation, with everyone from end users to industry analysts, reaffirms that people are making significant progress toward greater adoption. For an increasing number of organizations, DevOps is maturing from a set of budding goals into a tangible reality.

    At the center of this massive shift within the software factory is the issue of cultural change; as we know in all things DevOps, culture trumps all else. So how are organizations affecting, empowering and measuring all this seismic transition, in particular the human elements?

    During the recent DevOps Virtual Summit, leading practitioners representing industries ranging from banking to travel and technology shared their views on the progress of cultural reinvention.

    From management tips, to tools and even metrics useful for driving transformation, here are some of their observations:

    1. What’s the most critical cultural aspect of planning?
      Kazi Whitfield, director, Quality Engineering, MUFG Union Bank, N.A.: Scalability on this is the key; if you’re doing this at a new organization, you need to understand how to scale it, and starting at some smaller level helps you get there. It also helps each individual and those components that are changing to truly understand what’s changing, and why it’s changing; I think that’s a huge part of the conversation that has to happen.
    1. What’s the biggest obstacle to cultural change?
      Eric Warheit, senior manager, Environment Management, Southwest Airlines: The challenges that we’re working through are organizational in nature. It goes back to the question of localization, and people maintaining a mindset that they’ve previously had, and what we do to create the conversation around what’s possible and the key impacts that we want to have. Ultimately, it’s about being productive with quality, and creating those common values.
    1. How can tools enable cultural change?
      Eugene “Bear” Lehenbauer, group manager, Information Technology, Citrix: If I have to pick one that links them all, a ring to rule them all, that’s when you have an orchestration platform that can take what individual teams are already using and plug right in, or you can take what needs to be automated and automate it; that orchestration harness can now become that common thing that links them.
    1. How can you ensure cross-culture collaboration?
      Warheit: Creating that concrete example, that proof case tying analytics to it, so that we have real measurable ways to make good business decisions, and being able to foster that perspective so that we can look across the organization, as opposed to in a localized way, is really critical to be able to foment that collaboration that’s needed.
    1. How do you measure cultural change?
      Daniele Romano, product owner, Continuous Delivery Pipeline, ING Bank: The most important thing as an organization is to able to apply an analytical approach, meaning that for each team or department you should look at the current situation and have in mind the target situation, and then have different means to guide everyone toward that target.
    1. What’s a key cultural change metric?
      Lehenbauer: We’ve a developed an agility rating scale that allows people to earn scores for something as basic as change management. Based on how those key metrics behave, and there are several of them, but they are fact-based and they’re user impact driven for the most part, they reward the right behaviors. Those types of things can draw out some of DevOps principles results, but also have hard data behind them.
    1. How do you recognize advocates of cultural change?
      Whitfield: Being able to phase reward and recognition into the process is key. Organizations are doing a lot of great things and making millions of dollars, and as the individuals that are coding or testing are driving it, you have to understand how you get to those people’s hearts, to make them passionate about what they’re doing. That’s something that becomes key and it’s something that’s different per organization.
    1. What’s the best way to gauge successful cultural transformation?
      Romano: There is a really only one critical human metric, which is happiness. So, if engineers are happy and they can deliver fast, at a high quality and with intense focus—that’s a very unique but important metric.

    So there you have it, some of today’s most critical elements in driving the cultural transformation necessary to succeed in the DevOps world.

    If you missed the DevOps Virtual Summit you can still view recordings of all the sessions including this customer panel, and stay tuned to this space for more excerpts and highlights.

  • Our Sprint to Agile Success

    Our Sprint to Agile Success

    Adoption of the agile methodology is essential to the success of a DevOps culture. This is not revolutionary, but is an important foundation.

    We view agile as the tool for developers to achieve DevOps. Our team, which is developer-centric, is focusing on the agile piece of the equation moving forward.

    But what happens when you think you have a grasp on the agile methodology but it turns out you were completely wrong?

    A change is needed.

    Getting back to the Agile Manifesto is a great starting point. This is our story of DevOps failure to success by implementing the correct agile methodology.

    Our Perception Was Wrong

    When thinking about the agile methodology, and, to a greater extent, DevOps, open communication is at the core. However, it is not the only thing agile and DevOps is.

    At the beginning, we thought as long as there was open communication we were agile. Seeing as we sat in the same room, this was pretty simple. But we quickly learned that this was not agile, and the revelation was painful.

    As we experienced development bottlenecks and difficulty in defining responsibilities, we knew there needed to be an adjustment in how we viewed the agile process. Also, we had begun adopting projects in areas where we did not have a high level of expertise.

    This is what brought us back to the drawing board.

    Getting back to the basics meant looking at the reason for implementing agile in the first place. For us, it was releasing better software, more often, with fewer updates. In detail, it was looking at how long it took the team to identify and define the need. Then, it was looking at how long it took from problem definition to software release. We consider this one software cycle.

    The reason for implementing agile is to decrease the time it takes to move through one of these cycles, as well as to decrease the number of cycles on any one project. Defining this process to the team is how we brought our software development back to the basics.

    Renewed Focus

    Once there was a renewed focus, it was time to implement a set of tools and processes. The concepts we focused on incorporating are the following:

    • Adoption of sprints
    • Open communication
    • Version tracking
    • Environment of continuous testing
    • Responsibility definition

    None of these are particularly revolutionary, but when implemented effectively, it made a world of difference. Once we defined these five areas of focus, we adopted the daily sprint meeting. This implementation was the core of our transformation into a truly agile environment.

    The Daily Sprint Meeting

    In the developer environment, meetings can be against the team culture, especially in a startup. This is why a meeting needs to be strategic. The daily sprint meeting should have both a question constraint and a time constraint. The result is no dragging meetings.

    The two metrics for the daily sprint meeting are: 1) each team member gets two minutes to speak, and 2) four questions need to be answered. Those are:

    1. What did I accomplish yesterday?
    2. What will I achieve today?
    3. What obstacles do I face to accomplish my projects?
    4. How am I feeling today?

    With a team of five people, this meeting should take 10 minutes at the beginning of each day. The adoption of the daily sprint has helped us achieve a level of focus not possible without it. This focus has produced increased productivity, as well as adopting an effective agile environment.

    The daily sprint meeting is how we successfully implemented a DevOps culture. This has made all the difference within our team.

    About the Author/Logan Rivenes

    LRivenes PictureLogan Rivenes is a Marketing Manager with Datapath.io, specializing in decreasing latency through network traffic optimization for DevOps.
    Twitter: @loganrivenes

    LinkedIn: https://www.linkedin.com/in/logan-rivenes-01a8964a

  • Puppet Labs Launches Annual State of DevOps Survey

    Puppet Labs Launches Annual State of DevOps Survey

    Puppet Labs has announced its 5th annual State of DevOps Survey. In the past five years, we’ve surveyed more than 20,000 technical professionals around the world, spanning all industries and companies of all sizes. We’ve captured a tremendous amount of data about how IT performance, culture, technical practices and organizational performance intersect.

    What sets our survey apart from the many other surveys out there is that it’s science-based. Yes, there is a science to creating surveys and analyzing data, and there’s also an art to translating all of that amazing data into something that’s useful to the industry at large.

    This year, we’ve partnered with DevOps Research and Assessment (DORA), founded by our friends Gene Kim, Jez Humble and Nicole Forsgren. If you’ve been following the DevOps conversation for a while, their names should be familiar. Together, our goal is to help advance the DevOps community and help technology organizations improve their practices through rigorous scientific research.

    Through our analysis, we’ve been able to quantify IT performance. We know that high-performing IT teams deploy code 30 times more frequently, with 60 times fewer failures, and they recover 168 times faster. We also know the factors that predict IT performance—technical practices such as continuous delivery, cultural norms and lean management—which in turn predict organizational performance.

    This year’s survey seeks to dive deeper into each of these areas, while also shedding light on new areas of research, including:

    • ROI of IT performance. What can you expect from your investment in DevOps? Our goal is to understand how throughput, stability and quality affect the bottom line.

    • Security and compliance. What role do security and compliance play in a DevOps environment? Some people see security as a roadblock to DevOps, but we want to investigate if it has to be that way.

    • Quality. So you’re deploying code faster than ever. What’s the quality of that code?

    • Containers. How do containers affect IT performance? Are they making IT’s job easier?

    • Lean management. Last year we found that lean management practices such as limiting work in process and creating and maintaining visual displays showing key quality and productivity metrics predict IT performance. This year’s survey looks at additional lean management practices to help us better understand what it really means for a technology organization to go lean.

    The survey seeks answers to questions we get all the time and we believe they reflect the broader concerns of the DevOps community. Your experience matters and we hope you’ll share it by taking the 2016 DevOps Survey. It should only take about 20 minutes, and when you complete the survey, you’ll be entered to win some great prizes (see the official rules), including :

    Please take the survey and share it with your friends and peers. Together, we can make IT better for everyone.

    About the Author/Alanna Brown

    unnamedAlanna Brown heads up DevOps research and go-to-marketing strategies at Puppet Labs. She also works closely with Puppet Labs product owners and engineers to develop a roadmap and support for Puppet on Windows.

  • The Forgotten Fourth Question

    The Forgotten Fourth Question

    A very well-written paragraph about DevOps appears in Mike Loukides’ book, “What is DevOps”:

    … Modern applications, running in the cloud, still need to be resilient and fault tolerant, still need monitoring, still need to adapt to huge swings in load, etc. But those features, formerly provided by the IT/operations infrastructures, now need to be part of the application, particularly in “platform as a service” environments. Operations doesn’t go away, a good part of it becomes part of the development. And rather than envision some sort of Uber developer, who understands big data, web performance optimization, application middleware, and fault tolerance in a massively distributed environment, we need operations specialists on the development teams. The infrastructure doesn’t go away – it moves into the code; and the people responsible for the infrastructure, the system administrators and corporate IT groups, evolve so that they can write the code that maintains the infrastructure. Rather than being isolated, they need to cooperate and collaborate with the developers who create the applications. This is the movement informally known as DevOps.

    These days, recruiters to managers, engineers to analysts and, of course, CXOs all are interested in DevOps. They have questions such as, “What is DevOps?” “What are the DevOps requirements?” “What are DevOps implementations challenges?” But the key question is, “Why does DevOps remain difficult to implement?”

    Adopting DevOps does not simply mean adopting new processes or toolsets. “Why DevOps” can be understood better by focusing on the benefits it will give you. So let’s dive deeper into one of the key best practices we discovered in our journey into DevOps.

    Agile to DevOps

    Business challenges, complexities and competition pushed companies to innovate and solve their existing business problems in a better way, adding more features and integrated solutions. Ways of implementing solutions vary from organization to organization, adoption of tools and technologies also differ by small and large organizations based on complexity of problems, but it’s clear that many have adopted agile practices in the past two decades. Agile was made clear by a few well-defined processes such as iteration planning, daily scrum meetings, scrum of scrums (in larger enterprises), iteration demos, retrospective meetings and small, empowered teams.

    One of the key mindset changes most of the IT world went through while transforming themselves into agile execution was to have more verbal communications and daily scrum meetings. Daily scrum meetings required the scrum master to ask three questions:

    • What was completed yesterday?
    • What will be completed by end of today?
    • Any blockers?

    These three simple questions kept things simple and quick.

    As agility is extended to Ops now, there is a need to revisit whether these three questions are enough. There is a need to revisit the attendees in scrum meetings too. There is a need to add some more responsibility to the role of the scrum masters’ communication radius.

    Before we look at the fourth question, lets let look at the key reasons to adopt DevOps implementations in organizations:

    • Achieve higher speed of time to market without compromising Ops stability
    • Better resource utilization across Dev, Test and Ops
    • Be more responsive to customers

    Why would speed of delivery hamper stability in Ops? In our firsthand experience, the communication between Dev and Ops is not that rich in most of the organizations today. If one studies the root cause analysis of problems faced by Ops in trying to keep the environment stable and healthy, one will find some if not all the following issues:

    • Developers are not deployment-aware
    • Developers do not think of exposing or exporting the right controls for allowing Ops to configure app dynamically (Unintentionally. They simply don’t know.)
    • Many configurations are embedded deep inside the app
    • Adding specific third parties, changing their versions is sometimes missed in Ops communications
    • Ops is unable to clearly articulate what they require from Dev, even when everyone is co-located
    • Continuous firefighting in Ops results in fast fixes, better debugging, but not proactive processes
    • Even if weekly meetings between Dev and Ops happen, they do not have a clear agenda or metrics

    While there are many more symptoms and their causes, and while there are multiple solutions to fix processes and problems, we have found particularly one solution that seems effective.

    We added a fourth question to daily scrum meeting. That fourth question is: “Are you blocking Ops?”

    This question is a direct question to each developer in daily scrum meetings. This question essentially means to ask:

    • Have you written in any code that requires a new configuration?
    • Have you written any code that requires change in existing configuration?
    • Have you written any code that is creating a new binary or file that needs to be installed in production?
    • Have you recompiled existing code with new third party libraries because older ones are unsupported, even though there may be no other change in functionality?
    • Have you hard coded any IP address or other configurations into the code which cannot be changed without rebuilding?
    • And so on…

    Ask the Fourth Question

    The above list of questions is developed based on the root cause analysis of app stability in Ops. This list will be different for different developers/teams. The scrum master communicates the appropriate actions to developers based on the answers to these questions. The scrum master also alerts the Ops team and gives them a heads up on what’s coming their way, and provides timely feedback to the developers.

    Is this question important enough to be the fourth question in scrum meetings?

    Or is this question covered as part of the “Any blockers?” already being asked? One can debate either way. The blocker question in scrum meetings is thought of in a limited Dev/QA scope, and that scope has not included the Ops team until now. Given that Ops team faces customers, given that the DevOps movement is about improving synchronization between teams, this Ops question needs to be a first-class citizen in daily scrum meetings.

    Lastly, it’s critical to to look at root cause analysis-related Ops problem as early as possible. The scrum master becomes more mature as he asks this question to his team while interacting with Ops on hearing them out. If teams are funded enough to have someone from Ops into the scrum meetings, then he/she can be the primary representative for Ops and can steer the delivery to be Ops-friendly.

    Irrespective of how one achieves it, taking care of configuration issues proactively in daily meetings, and thinking about Ops on a daily basis in Dev meetings is key to successful DevOps adoption. Configuration issues are not the only focus, but they do form a large chunk of Ops issues.

    From our metrics, customers have seen following benefits after adding the fourth question their scrum meetings:

    • Shipping code 7x more frequently
    • 45 percent fewer failures
    • 12x faster ability to recover earlier in the tool chain

    Please feel free to share your findings with us @ info@opexsoftware.com.

    Happy DevOps!

     

    About the Author/Sanju Burkule

    sanjuSanju Burkule, a pioneer, visionary and creator of Products in the DevOps Space @OpexSoftware. He writes about automation and strong, connected, integrated IT and tweets often about these topics.

     

     

  • DevOps Days returns to London

    DevOps Days returns to London

    On April 19, after a break of nearly two and a half years, the DevOps Days conference is returning to London. Expect great things: All DevOps Days events are extremely interactive, making use of the Open Space format, and this year’s event includes a world-class set of presenters ready to share their experiences and guidance.

    The emphasis of the conference is on DevOps in the Enterprise, with specific reference to the financial services industry. That said, the event is open to any delegate, and participation is encouraged for enterprises that may be finding DevOps to be a struggle.

    The conference kicks off with a talk from Joanne Molesky a global expert in improving IT at large enterprises and co-author of, “Lean Enterprise: How High Performance Organizations Innovated at Scale.” Molesky’s talk, “We Are Great at DevOps but the Enterprise is Failing,” will include legacy management practices in finance, IT governance and procurement, and how they inhibit our ability to extend the benefits. She will offer up many real-world examples challenge us to apply Lean thinking and principles to break down these management barricades and deliver even more value to our customers.

    Kris Saxton, organizer of the London Infracoders meetup group, is slated to present a discussion with the provocative title, “Bimodal IT and other Snake Oil.” Saxton will draw on experience from five projects (four within large retail or investment banks and one with a large re-insurer) to challenge the idea that reforming legacy estates while continuing to innovate shouldn’t be attempted. Saxton hopes to convince you that this thinking is a flawed oversimplification which, when adopted, results in lost opportunities, delays crucial decisions and ultimately cripples innovation.

    Expert in everything from platform-as-a-service to people and culture to Perl—with the proud claim that modern Macs still ship with some of his Perl code, Casey West plans to discuss platform applications. He will play close attention to the operations, and, in particular, the operability end of DevOps. In fact, he won’t just be talking about it: Instead using 100 percent open source software, he will be performing a demo on stage in what he promises to be an “interactive live explosion” of a modern platform that will highlight quite how accessible in 2016 they are now.

    Day two kicks off with Gene Kim, award-winning CTO, researcher and author of, “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.” With all his immense dedication to DevOps operations and IT security, when Gene offers to share, “My Top Five Lessons Learned: DevOps In Large, Complex Organizations,” you know it’s going to be extremely valuable. In Gene’s own words: “I can’t wait to share with the DevOpsDays community what I’ve learned, share the problem road map that’s been verbalized, and some of the things we’ve been working on to help people overcome those problems!”

    Gareth Rushgrove is perhaps most well-known for writing the Devops Weekly newsletter, which is now more than 5 years old, but his talents don’t stop there. He is also a senior software engineer at Puppet Labs; a former infrastructure, operations and information security specialist for the UK government; and an established conference speaker. In his talk, “Rate of Change, (Un)opinionated Platforms and Devops Coevolution,” he will share insights on how the adoption of practices such as DevOps, microservices and tools such as platform-as-a-service are interconnected. He will explore the power of co-evolution, which makes cause and effect bi-directional, making it possible to leverage what you do and how you do it to improve your organization as a whole. Be prepared!

    Philippe Guenet is an expert in DevOps, in particular transforming banks toward continuous delivery. With his talk, “From Lean in Automative to Lean IT,” Guenet plans to draw heavily from what other industries learned in the 20th century dating back to Henry Ford and, of course, Toyota. He will revisit fundamental Lean concepts and principles applicable to IT and explore how they can be applied to improve organizations. The topics will include Total Quality Management, organization optimization, simplification and improved time-to-market.

    This is of course all just a taster. Slated also are talks from Thiago Almeida (Microsoft), Michael Coté (Pivotal), Jeromy Carriere (Google), Clair Agutter, Benjamin Wooton, Oliver Wood and John Clapham. In addition a panel is scheduled that will include at least one chief operations and technology officer. Last but not least, on top of all of the great interaction in breaks and Open Spaces and great presenters, we promise a great social event the evening of April 19, including a few surprises.

    The last few tickets are available here, so get one before they are gone!

    See you there!

    About the Author/Mark Rendell

    me_profileMark is a DevOps and continuous delivery specialist at Accenture with 11+ years of experience in development, configuration management, infrastructure as code, continuous delivery, environment management, build, deployment and testing automation as well as tooling. Currently, he runs a team specializing in implementing continuous delivery for large-scale, transformation deliveries across multiple industries.  He is also a proud contributor to the DevOps Platform.

    Blog: https://markosrendell.wordpress.com/

    Twitter: @markosrendell

     

  • The DevOps Equation: Agility + Empathy = Quality

    The DevOps Equation: Agility + Empathy = Quality

    I had the chance to catch up with Jeff Sussna (@JeffSussna), founder and principal of Ingineering.IT, ahead of his keynote address on continuous design, scheduled for DevOps Days Atlanta, April 26-27.  He discussed the importance of designing for service, responding to the unexpected, and the importance of building empathy across teams.  

    Derek Weeks (@weekstweets): Today I’m really happy that we have Jeff Sussna joining us for the latest in this series. Jeff, why don’t you introduce yourself?

    Sussna: Sure. Thanks for having me. I’m an independent consultant. I’ve been around, since rocks were young or the 80s, whichever is older. My particular background is that I have built systems and led organizations across the entire development, QA and operations spectrum. So, I’m a little unique in that respect.

    My current focus is on helping IT organizations and digital businesses really bring together their ops and design thinking so they can not only go faster and not only be more resilient, but provide more customer value at the same time.

    Weeks: Let’s talk about that a little. How is continuous design—a subject you have written a lot about—coming into play? How can it help us be more resilient.

    Sussna: Just to explain a little about what we mean by continuous design. It’s basically bringing together the entire organization and a unifying design in operations. It’s taking what we are doing in DevOps and expanding it to include product, design and support. It lets the whole organization really have an agile conversation with a market and with its customers.

    The issue that we face is that we live in a service world now. The point is no longer delivering products like cars coming off an assembly line. It’s much more about responding to our customers and having relationships with them. This is happening in a world that is more and more complex while less and less is in your control. We can’t figure everything out in advance. We have to have the ability to respond to the unexpected. We have to be able to do that at every level.


    “This is happening in a world that is more and more complex, while less and less is in your control.”


    I will give you a couple of examples of things that are perfect illustrations of this:

    I recently wished a friend of mine happy birthday. The reason I did that was that because there was a notice on my calendar that said it was her birthday. It was wrong. It wasn’t her birthday. I had no idea where that notice came from. Did it come from Skype? Did it come from LinkedIn? Did it come from email? I had no idea.

    We face that in IT, too. Systems are becoming more complex. We develop microservices. We use public and private clouds at multiple levels. We’re no longer in a situation where we can control things. We have to be able to respond and respond at all levels.


    When we make design and operations unified and continuous, we are able to say, “Where did we think we were going? Where would we need to go? and what do we do next?”


    What I talk about in my book, in my workshops, and all my work is that this needs to happen at every level. It’s not just a matter of designing for failure inside the data center or inside the infrastructure. We also have to do it at other levels.

    Here is another example:

    I was trying to get paid recently by a large enterprise client in Europe. They were trying to wire money internationally and my bank wasn’t really set up for it and they were having problems. They just kind of gave up. I kept getting these messages saying, “Sorry, we tried paying you and it didn’t work.” That was the end of it.

    I started to get frustrated and my client within the enterprise started to get frustrated to the point where they sent a scathing email to our AP department which they CC’d me on, saying, “Why can’t we take the lead to get his guy paid?”

    That’s an approach that is based on resilience in the sense of assuming that something can go wrong and figuring out what he can do to fix it when it can go wrong. As opposed to the traditional approach where we’ve been trained to push certain buttons and take certain actions. If those actions don’t work the way they are supposed to, well, I guess that is someone else’s problem.

     

    https://www.youtube.com/watch?v=ISSMHmwIMtA

     

    Weeks: This brings to light some recent conversations I was having with Paula Thrasher at CSRA. She was talking about developing systems and how to make them more resilient. She is using pictures as part of her conversations with people to help visual their work. What work steps are there? What work is flowing? What is communicating with what?

    Are you taking similar approaches with continuous design in how you work with clients to pursue it?

    Sussna: Well, I try to introduce them to the principles and I also try to introduce them to a set of practices which actually comes out of Mark Burgess‘s work on promise theory. It is really a conceptual model for how we think about success and failure and how do we think about service. For example, Amazon helps you get everything you need in your life from shaving cream to batteries to snow tires to books as quickly and easily as possible. In order to keep that promise they make a whole variety of other promises. Some of which filter all the way down to a promise to be able to go over and make a change to production as quickly as possible.

    Once you take that mindset. The interesting thing about a promise and the reason we use that word is because it is very intuitive. We all know what a promise is. We all make promises to each other every day. On the other hand, when we think about it in terms of service, IT, or whatever it may seem a little odd. The thing about promises is that on the one hand we are making commitments to do things and on the other hand promises sometimes get broken. That is actually good because it forces us to think. If I promised to pay my vendor. What do i do if I’m not succeeding in paying my vendor?


    The thing about promises is that on the one hand we are making commitments to do things and on the other hand promises sometimes get broken. That is actually good because it forces us to think.


    It’s sort of flipping our mind. Again, we are doing this in DevOps world where you start thinking about things like mean time to repair (MTTR) instead of mean time between failure (MTBF). We shift from how do we create a situation to how do we create a situation from where nothing will ever break—which becomes less and less possible. When then need to shift to a situation where we can respond more and more quickly.

    We go to executives and say, “We don’t want to worry about how often stuff breaks anymore,” and they freak out. We have to teach them and we have to teach ourselves that we can’t prevent things from breaking. Instead, let’s be able to fix them so quickly that maybe no one notices. My focus is on really helping people shift their mindset from thinking about ourselves to thinking about who we are helping. And from thinking about what we are doing to thinking about how well it succeeds and how well it overcomes problems.


    We have to teach them and we have to teach ourselves that we can’t prevent things from breaking. Instead let’s be able to fix them so quickly that maybe no one notices.


    Weeks: You talked about delivering on promises as quickly as possible. How does continuous design account for doing things as quickly as possible but making sure the quality is there at the same time?

    Sussna: In two ways—one way is that when you do things in small increments you are able to find problems more quickly. This is classic; it comes out of lean. It comes out of integration. It’s pretty basic stuff for our community now that it’s much easier to find and fix a bug five minutes after you change a code than five weeks after you change the code.

    When you put things like Linux, agile, DevOps, and design all together, they all focus on incremental feedback loops where you discover the gap between what you think you’re accomplishing and what you’re actually accomplishing very quickly. Secondly, when you take this all to the way to the marketplace and you make it part of the relationship with your customer, you can reduce a tremendous amount of waste.

    The way to think about continuous design is: it’s all about steering. Think about a boat and you’re trying to sail across a bay. You can’t just point the boat at the other side and say go because the winds are changing and you’re going to be able to respond to things that you can’t predict. If you sail for 20 minutes then discover you had pointed in the wrong direction, you’re going to be way off-course. You’re then going to generate a tremendous amount of energy getting back on course.

    What continuous design says is just an extension of continuous integration and continuous delivery taken all the way though the business. When we steer continuously we steer efficiently.


    What continuous design is really just an extension of continuous integration and continuous delivery taken all the way though the business. When we steer continuously we steer efficiently.


    Weeks: I know you’re going to be keynoting DevOps Days Atlanta coming up in April. Chris Corriere, one of the guys on the organizing committee, tells me that you’re leading a full day workshop on continuous design before that. Maybe you could tell some of the people that are attending DevOps Days Atlanta why they should be there a day earlier for your workshop and what things you’re going to address during that time.

    Sussna: To some degree what I am going to address is everything we have been talking about for the last 10 minutes or so—relative to DevOps. I think the key thing is, and we need to think about it. We need to think about more than just going fast. One of the big misunderstandings that I see with DevOps is this notion that it is primarily about delivering application changes to production faster. That’s only half of it. We really need to think about from the perspective of providing service.


     One of the big misunderstandings that I see with DevOps is this notion that it is primarily about delivering application changes to production faster. That’s only half of it. We really need to think about from the perspective of providing service.


    You know what we are talking about is essentially developers and operations people providing better service to each other. As a result, they can deliver better service to the business.

    Where we talk about microservices. You know that word service is in there for a reason. If all you do is you kind of shatter your organization and shatter your architecture into a bunch of little pieces, you still have to figure out how those pieces come back together into something coherent. The answer to that is taking a really deep service-oriented approach. That is very much what my workshop is about. What does that word even mean and how do we do it?

    Weeks: You mentioned thinking about interactions as delivering better service to one another within the organization. How does empathy come into your thinking around DevOps and Continuous Design?

    Sussna: I’d like to say agility plus empathy equals quality. What I mean by that, in particularly empathy, I think it’s important to take a minute and clarify. When we talk about empathy we are not talking about wallowing in someone else’s pain. We’re not even necessarily talking about things on an emotion or feeling level. It’s very simply the ability to see things as if from an other’s perspective. I will give you a prime example of where that comes into play and how it’s important and how it helps.


    Agility + Empathy = Quality


    When I work with clients, I tend to look at their process using their organization from the product beginning to the support end. Often times I see support getting the short end of the stick. I see places where they get phone calls from customers with features that they don’t even know exist yet. They sometimes get talked down to by development or product or even operations. They say things like, “Well you’re just supposed to support the stuff. You’re the young people. You’re the less experienced people.”

    When we have the ability to see things from their perspective, we can think about how support is actually experiencing our delivery process. If we don’t think about them when we go through the process of delivering new features, they will be lost. Them being lost will show when customers call for help. That will significantly affect customer satisfaction and then even beyond.

    The reason we need to empathize with support is so we can empathize with the customer. Often times when we build things and we don’t think about the notion that our customers might have trouble with it. On the one hand we have the illusion that if we design it and we build it really well, they won’t have trouble. We can’t perfectly predict who they are, what they need or what they will encounter. There’s an inevitability to the idea that they are going to need some amount of help.

    Weeks: I think that is probably the best description of empathy within a DevOps environment that I have heard anyone talk about. Jeff it has been a great conversation. Thank you very much. I really appreciate it and we look forward to seeing you at DevOps Days Atlanta coming up.

    Sussna: Thanks. It has been great talking to you and I am really looking forward to the conference. It will be fun.

    If you want to learn more about continuous design from Jeff, be sure to join us at DevOps Days Atlanta. I will also be at the event giving an Ignite Talk, titled, “Real Heroes Draw Pictures,” focusing on lessons learned from the 31 DevOps and continuous delivery reference architectures I assembled.

  • ‘Don’t Make Me Code’: Affordances for Developers

    ‘Don’t Make Me Code’: Affordances for Developers

     

    Thanks for listening to “Don’t Make Me Code,” a Heavybit podcast that discusses developer experience design and the unique challenges of building developer-facing products. Don’t Make Me Code is hosted by Steve Boak, co-founder and CPO at Opsee, and David Dollar, co-founder and CEO at Convox.

    In this episode David and Steve are joined by Dustin Larimer from Keen IO. They discuss affordances—the visual cues or attributes of a thing that inform you how it works – for developers.

    This podcast is brought to you by Heavybit, a program that helps developer-focused companies take their product to market.

    [soundcloud url=”https://api.soundcloud.com/tracks/251905881?secret_token=s-mL7jb” params=”auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=false” width=”100%” height=”166″ iframe=”true” /]


    Steve: Welcome to “Don’t Make Me Code,” episode four. We’re calling this episode “Affordances for Developers,” and with us today is Dustin Larimer of Keen IO.

    And he wrote a really excellent blog post about how they refactored their code based on some documentation and we’re gonna get to that in a second. I think, first though, since we’re talking about affordances for developers, we need to start with what are affordances? Since not everyone may know or realize what they are. And so I don’t know if you want to kick that off, Dustin?

    Dustin: Yeah, sure. It’s a really cool concept. It comes more from product design and industrial design. We’re talking about the visual cues or properties or attributes of something that tell you how it works.

    Like a really great example, like we’ve got buttons and knobs and dials and levers all around us in our environments and when you glance at it you can kind of tell. Like, “Well, this flips up and down,” or, “I can push this in and push it out.” And so, with physical products, it’s possible to embed a lot of knowledge about how something works simply in how it looks and that’s the idea of affordance, is that the thing kind of teaches itself to you.

    You don’t have instruction manuals for coffee pots, unless it’s some kind of crazy new thing with coffee pot. Or, like, tea kettles or things like that. You know, if there is an instruction manual, it’s probably been poorly designed. Because it’s considered successful and you can just, kind of, figure it out. There’s not a huge learning curve to make it work.

    And this gets really interesting with software because we’re moving from design of physical things to the design of, pretty much, invisible things.

    Steve: Yeah, and in a digital space, we have things that we call affordances. Things that look like buttons or look like links and we’ve moved from these skeuomorphic designs to flat designs and a lot of that stuff, I would argue, is getting harder.

    We have to rely on patterns that we know and trust and I honestly can’t tell, sometimes, if that’s getting easier or harder as we get better at this.

    Dustin: Yeah, totally. Like early on things like tabs were kind of based on the little tab of a filing cabinet. Like filing sheets. Because people were familiar with those in the analog world and so it kind of transferred nicely into the screen world.

    You know, things like drag and drop were brand new back when, you know, operating systems became consumer-friendly, so how do you teach people about drag and drop? So the little mouse cursor was a hand that literally grabbed and let go of things. So, it’s kind of a transition and the skeuomorphic era was largely to introduce mainstream public to how these interfaces work.

    ‘Cause they were familiar with how they, you know, they’ve seen them before so if something looked like a big ugly dial on the screen, you knew it could turn. But then, I guess Apple’s a good lead for moving away from that with their big redesign a few years ago. They justified it by saying, “We’ve moved into a time where enough people have learned about interfaces and screens.” We have a larger shared competency around these things so we don’t really need to make things that painfully obvious and rooted in the analog world anymore.

    People have a little more capacity for figuring this stuff out and so they could move into a new world where there’s not really a lot of basis for how these things work. They just invent new patterns and it succeeds or it fails. So, a hamburger menu was one of those. Everybody agreed it worked and now they’re everywhere. To a detriment, I think. But things like that, these new languages, new vocabulary kind of goes mainstream. New words become very popular or fall out of favor.

    Steve: Yeah, and they come and go kind of like fashion. The hamburger menu, again, being a good example of that. I feel like there’s been a lot of backlash against that because of the cost of using it that we only realized after playing with the implementation of it for a while.

    Dustin: Yeah, it kind of also became kind of a crutch. People didn’t have to think so much about their information architecture because they could just hide it all behind a hamburger menu.

    And that was the mobile, a mobile invention that made it into desktop and larger screens and a lot of the usability testing around that and, sort of, analytics around things like that find that it’s not a great idea. You know, it’s not a very effective design feature to just paste in arbitrarily anywhere. You kind of have to keep thinking about things.

    David: It is interesting to think about though, because even if you do end up with a bad design, like if everybody comes on the same design, then there is some sort of value to that, right? Like, even if the hamburger menu is terrible, now that I’ve encountered my 550th one I kind of know what you expect of me here.

    Dustin: You figured that one out. Yeah, totally. That kind of becomes part of our shared vocabulary around these interfaces.

    Steve: 
    And in the digital world, it’s kind of one abstraction away from the physical world, but then dev tools feel like a whole other level beyond that. Because now there isn’t even a clicky UI, necessarily. There’s nothing to latch on to, visually.

    And that’s part of what made your story so interesting. And maybe we can start talking about that now. Like, what was the problem you were trying to solve and how you tackled it.

    Dustin: At Keen IO I’m on the Experience Team, I’m a product designer, I’ve been doing engineering, also, for a long time. Kind of both interchangeably and when I came to Keen IO I took over all of our Java Script tooling, data visualization, dashboards and all that stuff. And so, that was the first time I was ever really faced with having to shepherd along a product that’s made of code and designed for people. There’s a lot of interesting moving pieces to that. But the idea was that we could keep adding new features and tacking new things on and just calling it whatever we wanted to and sort of letting feature bloat run away and do that or we could try to consolidate and shrink the footprint.

    A big challenge is that when you’re designing a product you like to believe that it’s this really important crazy thing that people are going to spend a lot of time on and love, but then you have to recognize that you’re just one more vendor in their stack. You’re just one more thing to integrate so they can get on to doing what they really care about and you can’t afford to require people to learn about you. I think that’s going to become very competitive of developer facing companies is that time to value.

    How long does it take for me to figure out what the hell you’ve created and how to use it and how to plug it in. And so, looking at that, we started designing with that in mind first. Like, how quickly can somebody find the code, install it, wire it up with minimal impact, minimal footprint and start getting immediate feedback that it’s working. And so we can’t just keep tacking features on and expanding the docs forever, we have to start shrinking at some point. So, with Java Script SVK where I’ve put all of my time, that’s a huge factor, it’s our most popular SVK.

    Most of the data that goes in and out of Keen IO goes through that. Most of it’s coming from browser traffic and that sort of thing. So, its got to be super effective at people being able to learn it, install it, use it. And a big part of that is having confidence to know that the thing you’re using, or the way you’re using it is the right way. That’s another big one. Where a lot of things fall down. Where you’ve set it up and you’re running it, but you’re not totally sure if you did it right. The things, the names of methods, and the way that things are called. It’s sort of ambiguous and it leaves you wondering if you’re gonna get surprised one of these days, like you just did it wrong.

    Steve: And we’ve talked before on the show, too, about the importance of boiler plate code and CLI’s and examples. Like, all the things that you rely on. And so for your Java Script API, this was just all in one place, it was one giant document?

    Dustin: Yeah, it was one monolithic library. One big document. It did tracking, recording events, querying, running analysis and then visualizing it all in one monolithic pile and we started breaking those out into their own standalone libraries and we wanted each one of those to be very purposeful and effective and super easy to glance over and get it.

    So, another thing we’ve run into with SDKs is the method name to log and event or just save an event into Keen IO is called a lot of different things. Like cross different SDKs it’s add event, or record event, or push event or whatever. The common thread being events, we talked about events, which is another point of confusion sometimes.

    And so we thought, “Well, let’s just shore that up.” Let’s only say record events and let’s say it everywhere and that way when you see something that says record events you know with absolutely no hesitancy whatsoever that this is the thing you’re looking for, you’ve found the right method, this is the way to do it, it works in a predictable way.

    So it’s really consolidating our vocabulary and shrinking the footprint so that it makes our messaging more effective because it works across all these different languages and it helps us focus the message and give people confidence that they’ve found what they’re looking for.

    So they don’t have to spend hours hunting through docs and looking for examples and asking questions on stack overflow to get that assurance that, yes, you’ve done it right.

    Steve: I thought one of the interesting things about your approach, too, was the team that you put together. It sounded like they’re developers, designers, product managers, people from all disciplines in the company that got together to work on the problem. Can you talk more about how you approached it?

    Dustin: Totally. Yeah, so part of that is to do with Keen IO itself. So, we’ve always been hedged on generalists. Mission-driven generalists, specifically. Where we like to have people come in who might have some, in most cases do have some, really great specialty and competency in one important area, but have a lot of lateral interests across other areas and have spent time doing marketing or content strategy or design, in addition to engineering.

    We find people who were doing one thing in life and then became platform engineers. And so, they bring a lot of really amazing breadth to the perspective when we solve these types of problems. And so, on my team, experience team, we are part of engineering, but we’re the part that makes the API fit for human consumption. So we built apps and interfaces and tools, developer tools, point-and-click user tools, all that parts of the website that people interact with when they log in. So we own the developer experience and the, just general point-and-click user experience. We want people to find what they’re looking for and get value fast. And so, to do that, we have some amazing engineers on the team. Some really brilliant engineers.

    A lot of our stuff specializes in Java Script, ’cause that’s just the environment that we spend most of our time in, we build stuff in. But also people who have also done product management and design and content strategy and things like that in the past. So, we’re a very small team with a lot of very broad experience. And I think that important, to sort of keep breaking yourself out of the bubble where you might only see things inside of one paradigm and I think that’s where you get caught.

    Steve: Yeah, and you were talking about the need for consistency, both in the Lexicon, and in the function of the tools. And so, it sounds like you ended up figuring out a logic to break what was one big monolith into pieces. And there was some pretty logic, it sounded like, that came out of that. So can you talk about the results?

    Dustin: So the way we went about it first is we used the idea of breaking it out and having standalone SDKs for these three functions. For tracking, querying and visualization. We looked at it as a chance to, kind of, break away from some of the patterns of earlier stuff and try to build, like the ideal and then try to work our way back and make it somewhat backward-compatible. But we took the chance to really get our thinking clear first and to that.

    And that’s where the thesis and that blog post came from is if we were building physical product, we wouldn’t start building a tea kettle by bending aluminum and hammering something out in the shop. We would probably just get a big piece of paper and start sketching and start thinking through all of the different what-ifs and maybes and to really push the forum of that until we could remove all the things that weren’t necessary and we could focus on what mattered.

    We could get our thinking really, really clear. And so, we did that with these SDKs, also, where we realized that software is more or less invisible in that sense, but its form is its documentation. So we started out just sketching docs, just doing install guides, writing pseudo code for how to install and run this thing. Looking at all the different method names and functionalities people wanted and making sure it was consistent, you know?

    It’s like if you, you don’t want the situation where if you use this method, it gives you a callback with these arguments, and if you use another one, it gives you a callback with different arguments, or maybe no callback with no explanation whatsoever.

    So we wanted that consistency to become part of the vocabulary for how the thing functions. And by doing that, we came to a very, very clear spec of not only how the thing works, but how I use it. How I install it and run it and under what examples and situations. So we wrote a lot of example code. And all this was before we wrote a single line of code.

    The beauty in that is once you have that perfect clear picture of what you’re building with good rationale for why it needs to exist, putting the code together, actually took no time at all.

    Steve: Yeah, and it sounds like a lot of the code was already there and you figured out a really nice way to sensibly reorganize the documents and then you reorganized the code to fit that.

    Dustin: Yeah, yep. So we started by designing our thinking and designing the conceptual form of it and then filled in the rest. And really wrote a lot of tests before we even wrote the code itself. So that really changed the way that I thought about making software. Where, now, if I ever catch myself getting stuck on something new, I realize it’s because I’m looking at a very granular level and kind of pinning myself into a corner, so stop, drop it, back up, and start writing out pseudo code and docs for it and then it makes sense.

    You can follow the show on twitter @dontmakemecode and email Stephen and David at dmmc@heavybit.com to suggest a topic or guest for the show. You can also subscribe to all of our shows by copying this link into your favorite podcast app. Stay tuned for show-specific feeds!

  • Git: One Giant Leap Toward DevOps

    Git: One Giant Leap Toward DevOps

    If you work in an organization that has not yet embraced DevOps, then you might be wondering where to begin. Fortunately, there are many good answers. But I want to introduce one simple starting point that you probably haven’t heard: introducing Git to operations. By itself, Git might not seem like a meaningful way to overcome the barriers between dev and ops. Yet, Git provides some capabilities that mark the difference between the lowest performing IT organizations and the highest. And the difference in IT performance can translate into substantial business benefits. The highest performing IT organizations have noticed the following benefits:

    • They are about twice as likely to exceed profitability, market share and productivity goals.
    • Their employees reported far higher levels of job satisfaction.
    • They realize 50 percent higher market capitalization growth over a three-year period.

    To be clear, no amount of data will ever prove that use of Git leads directly to growth, profit, productivity and job satisfaction. What the Puppet Labs’ “State of DevOps Survey” does show is the incredible range between low and high performance in IT organizations. The rate of deployment has hovered at 30 times faster for high-performing organizations over the lowest. The 2014 survey showed 50 percent fewer deployment failures, while 2015 showed 60 percent fewer deployment failures, when comparing highest to lowest. The survey also indicates which practices separate the highest performers from the lowest. According to the State of DevOps Survey, the top two indicators of high performance are:

    • Version control of all production artifacts (surprisingly, a far greater predictor of performance than version control of source code); and
    • Peer review of production changes (in contrast to external change approval).

    It’s not that these two practices magically yield business benefits. Indeed, ITIL has long recognized management of production artifacts and review of production changes as best practices. But too many organizations still believe faster deployment and higher reliability are trade-offs. What is “magical” in high-performing organizations is they realize they can have both faster deployment and higher reliability by collaborating across organizational boundaries and automating intelligently. That means dev and ops sharing goals, measurements and tools. So one way in which these practices differ from typical ITIL practices is how dev and ops perform them collaboratively. More specifically, when dev and ops use the same tool for version control, they can better share techniques and workflows. For example, the peer review of changes can include both dev and ops. Thus, Git is a gateway to DevOps.

    Git has long provided those two capabilities for software developers: Developers use Git to version source code and developers use pull requests to perform peer review of source code. With a few tips about how to apply Git to configuration files, operations can use pull requests to perform peer review of configuration changes. Now with git-lfs, operations can also version production artifacts, which often are large, binary files that historically have not worked well with Git. More advanced concepts such as infrastructure as code and dedicated binary repositories, for example, build on version control, so even if the role of Git changes over time, the skills and collaborations remain valuable.

    In summary, the business case of Git for Ops is simple. The Puppet Labs survey shows meaningful upside for low performing IT organizations to become high performing. The survey shows high performing organizations are characterized by both rapid deployment and high reliability, not picking one over the other. Two practices that characterize high performing IT organizations are version control of production artifacts and peer review of changes. Git is now well-positioned to provide both capabilities. If you’re ready to start introducing operations to Git, don’t forget that Bitbucket has no limits to the number of private repositories.

     

    About the Author/Ian Buchanan

    ianWhile Ian Buchanan has broad and deep experience with both Java and .NET, he is best known as a champion of agile methods in large enterprises. He is currently the Developer Partisan for Developer Tools at Atlassian, where he focuses on the emerging DevOps culture and the application of Bitbucket and Bamboo for better continuous integration and continuous delivery. During his career, he has successfully managed enterprise software development tools in all phases of their life cycle, from cradle to grave. He has driven organization-wide process improvement with results of greater productivity, higher quality and improved customer satisfaction. He has built multinational agile teams that value self-direction and self-organization. When not speaking or coding, you are likely to find Ian indulging his passions in parsers, meta-programming and domain-specific languages.