devopsy.kinsta.cloud

Tag: Measuring

  • Improving DevOps Success with Well-Defined Metrics

    Improving DevOps Success with Well-Defined Metrics

    Collaboration within diverse project teams is often more difficult to achieve when there is no clearly visible aim, the planning is haphazard and the information shared is inadequate. However, these are crucial hurdles to overcome in any organization—and more so in product companies with globally distributed teams working on different parts of the product pipeline.

    Successful product development requires effective and constant communication between various teams through well-defined metrics, not only to reduce the engineering operations overhead but also to improve on the overall engineering deliverables.  These metrics, including  ROI or profitability, mean time to recover and deployment frequency provide a clearer picture to an engineering or business owner on how the Dev and Ops teams are working, their adherence to processes and guidelines and the technology they’re using.

    Every engineering owner likes to know if the technology investment they make is going to yield a significant ROI. It is no different with DevOps. It is important to identify the exact metrics by which the positive impact of DevOps can be measured.  The most effective way to illustrate these metrics in a way that would make sense to senior management is revenue impact.  Here are a few of them which have had proven success on internet of things (IoT) projects.

    Business metrics such as cycle time impacts incremental sales and customer satisfaction. Whereas other business metrics such as product investment and running costs impact profitability, DevOps implementation effectiveness can be measured by both. Cycle time is, in turn, derived from a number of product features delivered, sprint velocity and release planning metrics. The lower the cycle time, the more features get pushed and impacts the go-to-market. It also results in potential incremental revenue in terms of the value brought by reducing the cycle time.

    Another dimension to measure DevOps impact is engineering metrics. Companies today are worried about the cost of downtime in the operational phase as much as they worry about time it takes to bring the product to market. DevOps metrics are valid throughout the end-to-end implementation of a project.

    Build automation and code quality analysis, for example, can provide significant savings in build cycle time and build rollout time. Automating deployment process improves time to deploy and sprint velocity. Continuous testing improves mean time to resolve, and CloudOps monitoring tools can help reduce mean time to recover and uptime of any application. Further collaboration in automation through ChatOps tools can help improve mean time to respond. Once set in motion, all these matrices start improving the product quality, which leads to less rework. Intelligent automation also incurs direct productivity benefits.

    Organizations now measure these codes to release the metrics at the end of the day to get maximum ROI. A lot of these metrics also depend on the DevOps capability maturity. Organizations at the higher end of the maturity spectrum would tend to measure all of these metrics and try to optimize them. Also, the emphasis of these metrics vary with the domain and stage of the product life cycle being worked on. Home automation companies developing their IoT platform, for example, would emphasize build and deployment metrics, whereas companies in industrial automation with large infrastructure deployment would look at CloudOps monitoring metrics more closely. For telematics companies, test automation metrics become important when there is limited time and bandwidth to capture the data.

    DevOps is a cultural transformation and most of the time businesses measure the effectiveness in a very obvious way. However, a change in the way the organization operates should be looked in such a way that the metrics can generate long-term value. Thus, continuous measurement of the DevOps practice should also become one of the major focus areas of DevOps teams.

    About the Author / Urvashi Babaria

    Urvashi Babaria is a Product manager at eInfochips working in new age areas like IoT, DevOps and CloudOps. She is a techno commercial marketer with nine years of experience in product and project management, business analysis and transformation. She has keen interest in areas of data science including visualization, analytics and modeling. You can start a conversation with her at urvashi.babaria@einfochips.com

  • Using DevOps Metrics Effectively

    Using DevOps Metrics Effectively

    If you’re a DevOps leader, you have a very tough job. You have to radically transform processes and behaviors that have been ingrained in IT for decades. You have to adopt new technologies quickly while still delivering code quickly and cost-efficiently. And you have to do all this better than your peers at other companies. Otherwise, you’ll simply slow your company’s progress in the new digital marketplace while your better-performing competitors gain advantage.

    The good news is that the right metrics can help you gauge your success. Metrics can both tell you if you are traveling down the right path, or need to adjust, and help you prove the success of your DevOps initiatives to the rest of the business. But first you have to get them—and just as importantly, you have to use them.

    I wrote two previous blogs talking about how to get the metrics—one outlining what not to measure and another that delved into determining which metrics are right for your organization. In this blog, we will explore the latter part: how to use metrics to ensure ongoing DevOps success.

    Consider the Audience

    Measurements of any sort have two primary audiences: internal and external. The same metrics can be used and gathered for both, but the intent—and how they are communicated—is different.

    For internal audiences, you are measuring things to gain a better understanding of your own performance. It’s important to know what adjustments need to be made: where to double down and where to change course. Like a racing car pit crew monitoring hundreds of sensors on a vehicle, your DevOps team should be constantly looking at ways to extract the maximum performance and value from your efforts. Metrics will help you do this.

    External audiences are less concerned about the nuances and details and more concerned with the actual outcomes. Going back to the racing car analogy, external audiences care about lap times and top speeds; not about the operating temperature of the engine. It’s important to keep this difference in mind when you are communicating results to business leaders. Don’t bog them down with nuances and details; instead, show the results.

    Best Practices for Communication

    For both internal and external audiences, there are five best practices to consider when communicating metrics:

    1. Automation. Automation isn’t just important for efficiency and consistency in the software development life cycle, it is also useful for capturing and communicating DevOps metrics. Tools that automatically record and communicate your results will help you keep your finger on the pulse of your organization without getting bogged down in measuring and reporting.
    2. Evolution. The savviest DevOps leaders don’t marry themselves to a specific set of metrics. They intuitively understand that the metrics they use to evaluate DevOps progress will mature as the DevOps practice itself matures. The metrics you measure in the first six months may not be the ones you measure in year two.
    3. Action. This perhaps goes without saying, but metrics need to be communicated to both audiences listed above. It can be challenging communicating results when they are not what was expected, but the best way to communicate less-than-optimal metrics is to have an action plan to remediate any possible issues.
    4. Communication. As noted above, action and communication go hand in hand. Once you know what’s working and what’s not working, you should have a solid plan on next steps. When something’s working well, the plan should focus on how to replicate success; when something is not working as well, the actions should focus on future improvements.
    5. Self-examination. Metrics aren’t just indicators of what your team is doing right and doing wrong. They’re also useful diagnostics for leadership decisions and leadership style. So remember to take the responsibility and share the credit.

    When you marry the above best practices with the other measurement tips outlined in my previous blogs, you will not only be able to gather the right metrics, you’ll know what to do with them. Remember that, like DevOps itself, the practice of measuring DevOps success is something that can be started slow and expanded.

    Not every organization has the time or the resources to measure everything that should—or could—be measured. So, start slow, focus on the metrics that matter to your organization and use the best practices outlined above to guide you. For those who want to learn more about DevOps and DevOps metrics, my book, “DevOps for Digital Leaders,” explores these topics—and more—in a much more detail.

    Happy measuring!

    — Aruna Ravichandran

  • DevOps Contrarian

    DevOps Contrarian

    I admire and appreciate the ideas of DevOps as a result of living through two decades of working in IT and experiencing the pain of what I would call the “failed state of IT”. The failed state of IT being something inherently broken, something that couldn’t survive on its own, something that eats its budget and provides no value and something where the people working in such a system don’t even see the worth of the system and stop challenging it all together. In order to correct or prevent these “failed states” I often apply my experiences learned through DevOps and Lean processes and values.

    DevOps to me has been about recognizing and pulling yourself out of this “failed state” concept of IT and doing so with a little bit of empathy, a little bit of common sense and a lot of science. I don’t necessarily mean pure science in trying to come up with an underlying law that describes and predicts your system of work, but the foundations of science – The ability to question everything, the ability to collect data, analyze data, measure data and develop analysis thereof. By thinking “DevOps Science” you choose to develop models that describe your work, define your flow, show the value of your system and tailor it to your business needs and to research the best models to achieve your end goal and strive to continually improve those models. DevOps is about putting this science to work, and doing so with collected intelligence & experimentation behind it.

    Where I find myself being a DevOps contrarian of sorts is in the often over simplified representation of DevOps as an applied model. I don’t have any issue whatsoever with the concepts described below by themselves; they create amazing discussions and speak to very core foundations of DevOps, but they’re often only spoken of superficially.

    DevOps is not just about Culture.

    I often see people speak of DevOps as an applied “Cultural Invention”, an innovation found to be useful to a group of people in their behavior. While that is a great description from a 30 foot view as if you’re looking from the outside, it really doesn’t help one understand culture in any sense to help foster it. DevOps culture isn’t about copying what others do and hoping for success, it’s about developing it within realizing that its affected by both forces encouraging change and forces resisting change and finding the balance there of is that is important. You can’t simply force DevOps on an organization and call that culture. The real power of  DevOps as a cultural invention is being able to develop DevOps values that directly apply to your organization so that it can be passed down generation to generation and survive on its own.  Thinking scientifically, we learn from these shared examples of cultural invention and their successes by experimenting with them to see how they fit our organization as there is no universal law of culture but merely models we can research and identify with. It’s great to build upon the success for others, but the only way to validate those successes is to test them yourself!

    Another danger of thinking applied culture is the development of cultural optimums, I parallel this to what Garret Hardin coined “The Tragedy of the commons”, whereby individuals, acting independently and rationally according to each one’s self-interest, behave contrary to the whole group’s long-term best interests by depleting some common resource. I won’t claim to define what your resource is that you’re depleting, but it’s something to keep in your mind as you think of culture because it’s important to understand culture external of yourself as much as it is within. The value of culture isn’t in how you rationalize it for your own goals, but how it operates on its own and if that culture is appropriate for the good of the entire system (business). The strongest metaphor in DevOps for depleting the wrong resource is wasted resources not working towards your common goal. Are you actively engaged on the right tasks? Right automation? Right flows? Are you merely creating more work or are you adding value? DevOps isn’t something you apply methodologically, its something you experiment with methodologically.

    DevOps is not just about automation.

    Automation is the process for operating with minimal or reduced human intervention.  If you’re focused on automating processes are you still improving that process or did you think to yourself “I automated that, now I’m free to work on other stuff”? Automation can tempt you into doing “more work” if not kept under control and that creates a real danger of losing focus, having too much work in progress and actually hurting your overall performance.

    Often times the best value in the systems we call “Systems Automation“ – the Chef, CFEngine, Puppet, Ansible & Saltstack systems  is not necessarily their amazing capability to automate processes but their ability to document processes. Those cookbooks, manifests, scripts and DSL’s you’re using are absolutely the best documentation of your system you can have. They provide a great resource when done in such a way to easily describe the processes that they’re automating so that they’re well enough documented for improving them. You can’t automate that which you don’t understand and the best way to understand them is to make sure the processes that describe them are easily readable, documented, tested, and validated. They should be part of your experiment, not just testing for failure but testing for improvement.

    Through the lens of science, we can look at automation more as standardizing on a process and applying the best resource to that process.  Sometimes it is compute power instead of human power  but don’t fall into the trap that the process is said and done (Automated all the things!) and time to move on! Science is a never ending process by where you build upon your conclusions!

    One conclusion I and many others have come across is that people are the most flexible resource you have! It may be a better value to your business to have the flexibility of a human resource if the process you’re trying to build requires flexibility. Even when planning for automation, requiring human intervention is a perfectly valid result and a justification you can sell to your business that has been proven through experimentation & data. Don’t ride the automation bandwagon when your data shows otherwise!

    DevOps is not just about measuring.

    What are you doing with the data you measure? What value does measuring give your organization? When you have the flu and you take your own temperature, what are you measuring for? Do you have the cultural aptitude in your organization to accept the ramifications of what you measured? Can your organization stop working when it’s “sick”, take the time to heal and start back up after taking corrective or remediation actions?

    Measuring is extremely important, but measuring isn’t what DevOps is.  Measuring is a part of what makes your organization responsive, culturally adept, and accountable for itself. It’s what allows you to convey value, to compare itself against and strive for improvement. If you’re not measuring, what are you improving against? Inversely, If all you’re doing is measuring, what is the value of your measurements? What are they the basis of? I like to think of biological systems when I start thinking of computing systems where there is a cost of energy to do something. Are you utilizing that cost the best way you can by measuring what you are or should you evolve the system to spend that energy elsewhere? Are you collecting the right metrics to even comprehend your systems? Do your metrics allow for capturing exaptation and evolution thereof? When you experiment, measure, rinse & repeat you get patterns; measuring helps identify those patterns. Creativity, experimentation and the process of science allow you to act upon those patterns and create value from those patterns. Some people call that innovation.

    DevOps is not just about sharing.

    There are people who believe that unless you are open source, have a GitHub repo and publish your code, you’re not really sharing and you’re not DevOps. I certainly hope we’re not going down this path and certainly hope we can appeal to the broader ideas of sharing. Sharing is about the joint use of a resource and sharing is knowledge charity. Sharing goes hand in hand with culture in not just expressing your needs and sharing your knowledge, but having the ability to listen, feel empathy and accept what others share as well.  When it comes to collaboration & sharing it’s much easier to accomplish if everyone sees it through the lens of science, not only do you speak to DevOps values more rationally but you can even accept opposing views and ideas much easier because you have evidence to them and aren’t emotionally tied to their success, you’re tied the process that challenges their success as a way to improve them!

    DevOps through the lens of Science.

    The beauty of DevOps through the lens of science is that science is about challenging ideas,  accepting failure, learning from history/prior experiments and having the strength to not only say those successes and  failures are a valid output of your experiments but also come up with a new experiment or new hypotheses and do the process all over again. This helps define your culture, this helps setup what you should automate, this helps you validate what to measure and this develops what you should share.  Through the lens of science you learn through experimentation, testing & verification what works for you, what creates value, what brings you closer to your goal. When honestly done you also learn what doesn’t work for you, where you failed, and how you can use those lessons in failure to continuously improve.  A failed hypothesis after all is still good data!

    Why so much focus on the sciences? Personally, I enjoy astrophysics not because there is direct monetary value in understanding the cosmos around us but because of the philosophical nature of being, the beauty of knowing and the questions it gets you asking as the more you know, the more you wonder. That wonder, is what makes DevOps so great to me. That wonder is what drives continuous improvement. That wonder is what drives the “what next…”. That wonder is what pushes “question everything”. That wonder is always what pushes me to accept and understand the science we do know and trust it because it’s based upon the values that are respected by thousands of scientists all over the world through peer review and consensus. That foundation is not the assumption that we think we know it all, but that we know enough to describe something and we trust that description good is enough that we challenge you to break it.  I would highly recommend that any DevOps practitioner cross pollinate their technology passion with the sciences as there is nothing more exciting than realizing commonalities of biology, physics, astrophysics and the laws of nature within your own thoughts and patterns as they’re applied to DevOps.

    So be a DevOps contrarian; for its not accepting DevOps as the answer, it’s the challenge there of that will make it aspire to bigger and better things. It’s not that Culture, Automation, Measuring and Sharing is wrong by any means, but they’re just scratching the surface of what you need to understand to make DevOps “Work” for your organization.

    I’m not only a DevOps contrarian, but I’m an OS contrarian, a Developer contrarian, a Systems Contrarian. I’ve made the best of running Windows, Oracle Linux, Oracle Database, eBusiness Suite, Weblogic and much more. Not because I’ve personally identified or I am idealistically aligned with any of these tools & systems, but because they solve the needs of the business; they create value, they operate and perform admirably and we have proven their worth. It’s no utopian system by any stretch of the imagination, but I find great reward in the continuous experimentation of making these systems work and improving upon them in a very DevOps way.

    After practicing DevOps & Lean principals and having  over two decades of IT experience, I’ve realized the “failed states of IT” and people who thrive in them are people who stopped critical thinking, stopped challenging the status quo and stopped experimenting.  They stopped having a contrarian.