Author: Byron Miller

  • Parallel Evolution of DevOps Enterprise Patterns

    Parallel Evolution of DevOps Enterprise Patterns

    The DevOps evolution (Revolution!) has a great story; a story of winning the hearts and minds of people of all experiences who all anxiously strive to achieve their goals in amazing new and opportunistic ways. This story isn’t that of throwing everything away and starting over, nor is it a story of always having to start from scratch and do things from the ground up. It’s a story that reflects the hard work, critical thinking, scientific processes and deep analysis of your business needs with your goal in mind.  The concepts of the classic DevOps cloud business and the classic Enterprise all evolve in different ways but end up with the same evolutionary cognitive artifact, the focus on improvement and adding value through technology.

    When it comes to talking about Enterprise patterns vs the more spoken about “Cloud” patterns, we sort of speak to them as if their exclusively different, almost denying them an existence by choice.  One sees an “Enterprise” shop and almost cringes at the thought of them being DevOps out of an ideological view more than an evolutionary view.  However, when studying a “DevOps Enterprise” and “Cloud” company  from the ten thousand foot view we can see them as being much more similar.

    Cloud Patterns

    Develop -> Build -> Test -> Deploy

    Enterprise Patterns

    Develop -> Build -> Test -> Deploy

    Visually, this pattern while overly simplified is the same but how people get from develop to deploy may be entirely different.   In a developer centric “cloud pattern” you may be developing a web application to offer self service functionality to your end users but in a corporate environment you may be less developer centric as a business analyst adding features in a corporate resource planning application.  The end result is still creating value for your customer, developing functionality, building that functionality, testing that functionality and deploying that functionality and doing so in a very “DevOps Way”. Taking a closer look at the Enterprise through a slightly stronger magnifying glass, we would see something more like this for a typical enterprise deployment pattern:

    Storage -> Physical Host > Hypervisor ->  Virtual Host -> Database  <-> Application -> Customer

    Everything that composes this pattern could be made of of all sorts of parts from many vendors, open source projects, COTS systems and value add resellers. The components aren’t really what matters.  It’s the technology that an organization has acquired in doing business and the processes they have developed to meet the demands of their organization over their years of growing. It’s grown organically just as our concepts of IT have grown over there years and its done so creating and growing value too (at least, that’s the expectation). It’s evolved.  For better or worse, there is something we’ve encoded in these concepts and transmitted in an artifact we call “Enterprise”.

    The end result of applying DevOps to an Enterprise “Develop -> Build -> Test -> Deploy” pattern is to add value to this pattern, align this pattern to the business and to optimize the flow of this pattern. It’s really no different than applying these ideas to a “Cloud” business. One could optimize the Enterprise pattern of their Oracle Financials application by doing continuous integration just as cloud operations strive for continuous integration, but the CI you see in Oracle world  may be drastically different than that of typical Cloud Ops CI. Enterprise CI may be more of a “validation” experiment, a “Safe to fail” experiment. It may be more business process oriented – you may see people experimenting with BPEL or other tools to wrap business requirements and compliance around tasks/alerts/metrics/services/data.  The more we talk about Enterprise DevOps patterns the more we realize that we can benefit from these experiences in all DevOps patterns! There is a lot we can parallel in our views and see value from. Just because the “Enterprise” and the “Cloud” may appear differently, doesn’t mean they’re exclusively different by mandate or incapable of converging ideas.

    Looking more closely at the Cloud, we see this a mode of operations:

    Virtual Host -> Database  <-> Application -> Customer

    More frequently, depending on which type of cloud you seek out, you could simply see:

    Database  <-> Application -> Customer or simply just  Application -> Customer

    I feel it’s important to look at this evolution as parallel or even somewhat convergent (the differences being arbitrary if you ask me) because the end results are the same, that all-powerful “Goal”.    How one gets there, is how you tell your own story, but that story is your own evolution of services and what worked for you and your survival. The common hereditary artifacts of this separate but in the end parallel evolution of services is that they appear to build upon each other; sharing similar services, using APIs, using abstractions when needed, using automation when needed, building up “self-service” type systems/frameworks and doing so with the end user in mind and creating value (surviving!). To the customers in an Enterprise business, all they see is “application” or “service” just as in the cloud services organization your customer  sees “application” or “service”.  DevOps is the patterns that create/add value/optimize flow regardless of the service to the customer.

    Some people call the Enterprise applications and services we all know so well as the “Dinosaurs” of the computing age. Let me remind you, dinosaurs ruled the earth for 230 million years;  that’s 229+ million more years than humans! Dinosaurs weren’t stagnant by any means, they continually evolved and grew with staggering variations and abilities to survive.  If you stopped evolving, you went extinct.  Beyond the extinction of the dinosaurs, many species from the same era did survive and they survived not just because they had the winning combination of hereditary traits for survival but because they evolved through the changes that continued to happen regardless of their will to survive. Dinosaurs had an evolutionary dead end not because they did the wrong things or had the wrong design, but because couldn’t survive against the external forces that happened upon them.  The Cloud business could just as well be a dinosaur, soon to be extinct if that business fails to evolve for its own survival. Same for the Enterprise business.  We’re all evolved from dinosaurs in the end and if some have figured out how to survive through their cosmic collisions and still appear to outsiders as a metaphorical dinosaur, then that survival should be seen as having value in and of itself. The cognitive ability of reason should win, not an ideological opinion thereof.

    DevOps at “Big Giant and often cumbersome Corporation” may be different in scale/ideas, but the goal is just as much the same  and the experiences we can learn from each other, are just as much the same. Join my efforts in documenting DevOps in the enterprise, and help work on Enterprise DevOps patterns or DevOps Audit Defense concepts.  Both of these are open to participation and aren’t in any way specific to any technology or service other than sharing experiences that are common in the arbitrary “Classic Enterprise” or “Classic Corporate IT” shop.

    Through the parallel evolution of Cloud and Enterprise systems, convergence may be the end result, but it certainly isn’t the mandate. Keep evolving! Mother nature has done it for billions of years and she isn’t stopping anytime soon.

  • DevOps Contrarian

    DevOps Contrarian

    I admire and appreciate the ideas of DevOps as a result of living through two decades of working in IT and experiencing the pain of what I would call the “failed state of IT”. The failed state of IT being something inherently broken, something that couldn’t survive on its own, something that eats its budget and provides no value and something where the people working in such a system don’t even see the worth of the system and stop challenging it all together. In order to correct or prevent these “failed states” I often apply my experiences learned through DevOps and Lean processes and values.

    DevOps to me has been about recognizing and pulling yourself out of this “failed state” concept of IT and doing so with a little bit of empathy, a little bit of common sense and a lot of science. I don’t necessarily mean pure science in trying to come up with an underlying law that describes and predicts your system of work, but the foundations of science – The ability to question everything, the ability to collect data, analyze data, measure data and develop analysis thereof. By thinking “DevOps Science” you choose to develop models that describe your work, define your flow, show the value of your system and tailor it to your business needs and to research the best models to achieve your end goal and strive to continually improve those models. DevOps is about putting this science to work, and doing so with collected intelligence & experimentation behind it.

    Where I find myself being a DevOps contrarian of sorts is in the often over simplified representation of DevOps as an applied model. I don’t have any issue whatsoever with the concepts described below by themselves; they create amazing discussions and speak to very core foundations of DevOps, but they’re often only spoken of superficially.

    DevOps is not just about Culture.

    I often see people speak of DevOps as an applied “Cultural Invention”, an innovation found to be useful to a group of people in their behavior. While that is a great description from a 30 foot view as if you’re looking from the outside, it really doesn’t help one understand culture in any sense to help foster it. DevOps culture isn’t about copying what others do and hoping for success, it’s about developing it within realizing that its affected by both forces encouraging change and forces resisting change and finding the balance there of is that is important. You can’t simply force DevOps on an organization and call that culture. The real power of  DevOps as a cultural invention is being able to develop DevOps values that directly apply to your organization so that it can be passed down generation to generation and survive on its own.  Thinking scientifically, we learn from these shared examples of cultural invention and their successes by experimenting with them to see how they fit our organization as there is no universal law of culture but merely models we can research and identify with. It’s great to build upon the success for others, but the only way to validate those successes is to test them yourself!

    Another danger of thinking applied culture is the development of cultural optimums, I parallel this to what Garret Hardin coined “The Tragedy of the commons”, whereby individuals, acting independently and rationally according to each one’s self-interest, behave contrary to the whole group’s long-term best interests by depleting some common resource. I won’t claim to define what your resource is that you’re depleting, but it’s something to keep in your mind as you think of culture because it’s important to understand culture external of yourself as much as it is within. The value of culture isn’t in how you rationalize it for your own goals, but how it operates on its own and if that culture is appropriate for the good of the entire system (business). The strongest metaphor in DevOps for depleting the wrong resource is wasted resources not working towards your common goal. Are you actively engaged on the right tasks? Right automation? Right flows? Are you merely creating more work or are you adding value? DevOps isn’t something you apply methodologically, its something you experiment with methodologically.

    DevOps is not just about automation.

    Automation is the process for operating with minimal or reduced human intervention.  If you’re focused on automating processes are you still improving that process or did you think to yourself “I automated that, now I’m free to work on other stuff”? Automation can tempt you into doing “more work” if not kept under control and that creates a real danger of losing focus, having too much work in progress and actually hurting your overall performance.

    Often times the best value in the systems we call “Systems Automation“ – the Chef, CFEngine, Puppet, Ansible & Saltstack systems  is not necessarily their amazing capability to automate processes but their ability to document processes. Those cookbooks, manifests, scripts and DSL’s you’re using are absolutely the best documentation of your system you can have. They provide a great resource when done in such a way to easily describe the processes that they’re automating so that they’re well enough documented for improving them. You can’t automate that which you don’t understand and the best way to understand them is to make sure the processes that describe them are easily readable, documented, tested, and validated. They should be part of your experiment, not just testing for failure but testing for improvement.

    Through the lens of science, we can look at automation more as standardizing on a process and applying the best resource to that process.  Sometimes it is compute power instead of human power  but don’t fall into the trap that the process is said and done (Automated all the things!) and time to move on! Science is a never ending process by where you build upon your conclusions!

    One conclusion I and many others have come across is that people are the most flexible resource you have! It may be a better value to your business to have the flexibility of a human resource if the process you’re trying to build requires flexibility. Even when planning for automation, requiring human intervention is a perfectly valid result and a justification you can sell to your business that has been proven through experimentation & data. Don’t ride the automation bandwagon when your data shows otherwise!

    DevOps is not just about measuring.

    What are you doing with the data you measure? What value does measuring give your organization? When you have the flu and you take your own temperature, what are you measuring for? Do you have the cultural aptitude in your organization to accept the ramifications of what you measured? Can your organization stop working when it’s “sick”, take the time to heal and start back up after taking corrective or remediation actions?

    Measuring is extremely important, but measuring isn’t what DevOps is.  Measuring is a part of what makes your organization responsive, culturally adept, and accountable for itself. It’s what allows you to convey value, to compare itself against and strive for improvement. If you’re not measuring, what are you improving against? Inversely, If all you’re doing is measuring, what is the value of your measurements? What are they the basis of? I like to think of biological systems when I start thinking of computing systems where there is a cost of energy to do something. Are you utilizing that cost the best way you can by measuring what you are or should you evolve the system to spend that energy elsewhere? Are you collecting the right metrics to even comprehend your systems? Do your metrics allow for capturing exaptation and evolution thereof? When you experiment, measure, rinse & repeat you get patterns; measuring helps identify those patterns. Creativity, experimentation and the process of science allow you to act upon those patterns and create value from those patterns. Some people call that innovation.

    DevOps is not just about sharing.

    There are people who believe that unless you are open source, have a GitHub repo and publish your code, you’re not really sharing and you’re not DevOps. I certainly hope we’re not going down this path and certainly hope we can appeal to the broader ideas of sharing. Sharing is about the joint use of a resource and sharing is knowledge charity. Sharing goes hand in hand with culture in not just expressing your needs and sharing your knowledge, but having the ability to listen, feel empathy and accept what others share as well.  When it comes to collaboration & sharing it’s much easier to accomplish if everyone sees it through the lens of science, not only do you speak to DevOps values more rationally but you can even accept opposing views and ideas much easier because you have evidence to them and aren’t emotionally tied to their success, you’re tied the process that challenges their success as a way to improve them!

    DevOps through the lens of Science.

    The beauty of DevOps through the lens of science is that science is about challenging ideas,  accepting failure, learning from history/prior experiments and having the strength to not only say those successes and  failures are a valid output of your experiments but also come up with a new experiment or new hypotheses and do the process all over again. This helps define your culture, this helps setup what you should automate, this helps you validate what to measure and this develops what you should share.  Through the lens of science you learn through experimentation, testing & verification what works for you, what creates value, what brings you closer to your goal. When honestly done you also learn what doesn’t work for you, where you failed, and how you can use those lessons in failure to continuously improve.  A failed hypothesis after all is still good data!

    Why so much focus on the sciences? Personally, I enjoy astrophysics not because there is direct monetary value in understanding the cosmos around us but because of the philosophical nature of being, the beauty of knowing and the questions it gets you asking as the more you know, the more you wonder. That wonder, is what makes DevOps so great to me. That wonder is what drives continuous improvement. That wonder is what drives the “what next…”. That wonder is what pushes “question everything”. That wonder is always what pushes me to accept and understand the science we do know and trust it because it’s based upon the values that are respected by thousands of scientists all over the world through peer review and consensus. That foundation is not the assumption that we think we know it all, but that we know enough to describe something and we trust that description good is enough that we challenge you to break it.  I would highly recommend that any DevOps practitioner cross pollinate their technology passion with the sciences as there is nothing more exciting than realizing commonalities of biology, physics, astrophysics and the laws of nature within your own thoughts and patterns as they’re applied to DevOps.

    So be a DevOps contrarian; for its not accepting DevOps as the answer, it’s the challenge there of that will make it aspire to bigger and better things. It’s not that Culture, Automation, Measuring and Sharing is wrong by any means, but they’re just scratching the surface of what you need to understand to make DevOps “Work” for your organization.

    I’m not only a DevOps contrarian, but I’m an OS contrarian, a Developer contrarian, a Systems Contrarian. I’ve made the best of running Windows, Oracle Linux, Oracle Database, eBusiness Suite, Weblogic and much more. Not because I’ve personally identified or I am idealistically aligned with any of these tools & systems, but because they solve the needs of the business; they create value, they operate and perform admirably and we have proven their worth. It’s no utopian system by any stretch of the imagination, but I find great reward in the continuous experimentation of making these systems work and improving upon them in a very DevOps way.

    After practicing DevOps & Lean principals and having  over two decades of IT experience, I’ve realized the “failed states of IT” and people who thrive in them are people who stopped critical thinking, stopped challenging the status quo and stopped experimenting.  They stopped having a contrarian.