devopsy.kinsta.cloud

Category: Enterprise DevOps

  • DevOps: State of the Art in 2017, and Why CDOs Should Care

    DevOps: State of the Art in 2017, and Why CDOs Should Care

    Nine years after the term “DevOps” was coined, the following two questions seem still open:

    From the perspective of the industrialization of work labor also in the software industry, the answer to the first question appears very clear to me: DevOps is the third (digital) revolution of software and IT services, where CI/CD was equivalent of the second industrial revolution as elaborated in more detail in this article. Chances are high that further transformation can be expected to merge it with the fourth industrial revolution, as stated here: “DevOps will underpin the 4th industrial revolution.” That is also why chief digital officers (CDOs) should be in contact with DevOps teams in their organizations.

    Now, I see two important things have happened end of 2016: First, the establishment of DevOps Express, as 14 leading enterprise vendors have recognized the importance of interoperability of their tools for leveraging DevOps stack. The demand for this—the second important thing—has been (in my opinion) totally confirmed by CA’s infographic going viral, which maps a broad selection of tools around the software development life cycle. Adding the lifecycle view has been definitely an improvement beyond an overview of tools as proposed in the Periodic Table from XebiaLabs. Still, we are not done. Let me explain why.

    The next challenge I see is there is yet no agreement within the community what is—or could be—a reference DevOps toolchain (meta) model or process (meta) model. Is it:

    • “Plan, Develop, Test, Release, Operate,” as suggested by CA, or maybe
    • “Plan, Create, Verify, Package, Release, Configure, Monitor,” (Wikipedia) or
    • “Requirements, Development, Build, Testing, Deployment, Execution” (Len Bass et al.)?
    Source: Wikipedia, DevOps toolchain

    In the seminal book, “Continuous Delivery,” this life cycle was suggested as “Commit stage, automated acceptance testing, automated capacity testing, manual testing, release.” While we talk so much about culture in DevOps, we should recognize the need to align on a high-level view on the DevOps life cycle that is well-understood and accepted by most stakeholders.

    To have a track of all this and attract other collaborators, I have created an open format (SVG) infographic and published it as a GitHub project. Dear colleagues at the DevOps Express consortium, this is your call! Let’s join forces and accomplish this together.

    Next Steps

    After we will have (I hope so) a community-accepted DevOps meta-model toolchain setup and all important tools in place on the ecosystem, many more interesting questions will appear:

    • Which of these DevOps tools leverage open APIs to reduce risk of vendor lock-in?
    • Which tools belong to which generation of systems, leveraging optimization of human labor?

    Why DevOps is also CDO’s Business

    The fourth industrial revolution builds on digital process data, right? So if we aggregate all that data spread across and produced by the DevOps tools, we could get useful and holistic analytics and insights about IT in much faster time than we can imagine today.

     

    About the Author / Peter Muryshkin

    1Peter studied M.Sc. in Applied Computer Science and works in management consulting roles focusing on complex analysis and customer satisfaction, like IT Business Analyst and Change Manager. His current scientific research focus is digital transformation applied to the field of software engineering in large distributed organizations. Connect with him on LinkedIn.

     

     

     

  • The Perfect Combination: Imperative Orchestration, Declarative Automation

    The Perfect Combination: Imperative Orchestration, Declarative Automation

    There are two fundamental approaches of how to design automation processes, declarative (also often known as model) and imperative (also often known as workflow or procedural).

    The purpose of this article is to explain the principles of each, explore a little of how they work, try to highlight some of the strengths and weaknesses and explore how they can be used together. I don’t think one size fits all—based on your requirements and some of what is explored below you may use a combination of the two, even if some of the tooling you use leans toward one way or the other.

    Declarative/Model-Based Automation

    The fundamental of the declarative model is that of desired state. The principle is that we declare, using a model, what a system should look like. The model is “data driven,” which allows data, in the form of attributes or variables, to be injected into the model at runtime to bring it to a desired state. A declarative process should not require the user to be aware of the current state; it will usually bring it to a required state using a concept known as idempotent.

    Idempotent

    What this basically means is that if you deploy version 10 of a component to a development environment and it is currently at version 8, changes 9 and 10 will be applied. If you deploy the same release to a test environment where version 5 is installed, changes 6 through 10 will be applied, and if you deploy it to a production system where it has never been deployed, it will apply changes 1 through 10. Each of the deployment processes brings it to the same state regardless of where they were initially. The user, therefore, does not need to be aware of the current state.

    Maintaining State

    Once we have a model for our infrastructure or applications, any future changes required are made to the model. This means that as a model changes over time—and may have a current, past or future state. This raises the question of how can a model be applied to a target system, how can it know what changes to make? This is not always easy to answer and is not the same for every platform and runtime you are managing.

    State Management

    It’s hard to cover a broad subject like this without making some generalizations—and there are some in this section. There are broadly three methods used to keep an environment or system and its desired state models in line:

    Maintain an inventory of what has been deployed. This where we maintain the state of what has been deployed, and what is in the desired state of a release, and only apply the difference.

    For example:

    1. My release or ‘desired state’ contains database updates: 1.sql, 2.sql, 3.sql, 4.sql, 5.sql
    2. My example test system has had 1.sql, 2.sql, 3.sql already deploy (recorded in my inventory)
    3. Automation determines to only run 4.sql and 5.sql

    Use Case: This type of state management is particularly useful for things such as databases or systems that requires updates from a propriety API where the state of the target cannot always be easily determined.

    Validate/compare the desired state with what has been been deployed. As a simple illustration, we may want to determine if important_file.xml from my desired state deployment exists on the target system and whether it looks the same as the one in my package.

    1. If it doesn’t exist – create it / copy it, etc.
    2. If it does, determine whether it is different
    3. If it is different, how do I update it (converge, copy over, etc.)?

    Use Case: Probably the most widely used implementation is to simply manage the files that many cloud-native runtimes use to store configuration definitions, such as ssh keys for a server or xml files to manage a java runtime such as tomcat. This also can be used to maintain runtimes that are managed via an API; for instance, What is my heap or config item for my WebSphere Server? Is this what it should be? If it is doing nothing, set it to correct desired state value.

    Just make it so. There are occasions when we don’t really care about what exists. The most obvious example here is anything that is stateless, such as a container. There are probably very few example of why you would want to run any automation within a container to change its state; you would just want to instantiate a new one that has the new configuration you require.

    Imperative/Procedural/Workflow-Based Automation

    The fundamental of what often is referred to as workflow, procedural or imperative approach is that a series of actions are executed in a specific order to achieve an outcome. For an application deployment, this is where the process of “how the application needs to be deployed” is defined, and a series of steps in the workflow are executed to deploy the entire application.

    A standard example might include:

    • Some pre-install/validation steps
    • Some install/update steps
    • Finally, some validation to verify what we have automated has worked as expected.

    The often-cited criticism of this approach is that we end up with lots of separate workflows that layer changes onto our infrastructure and applications—and the relationship between these procedures are not maintained. The user, therefore, needs to be aware of the current state of a target before they know which workflow to execute. That means the principle of desired state/idempotent is hard to maintain—and each of the workflows are tightly coupled to the applications.

    The reality here is that the situation is not black and white. Puppet is an example of what is seen as a Declarative automation tool, while Chef is said to be Imperative. Do they both support concepts of desired state and are idempotent? The answer is, of course, Yes. Is it possible to use a workflow tool to design a tightly coupled release that is not idempotent? The answer is, again, yes.

    What are the Benefits of Workflows?

    The benefit of using a workflow is that we are able to define relationship and dependencies for our units of automation—and orchestrate them together. In the example above, we can create a workflow to perform pre-install, installation and post-install steps, perhaps adding additional conditional steps if it’s in a particular environment such as production (disable a monitoring system, additional security configurations, etc.).

    Procedural workflows also allow us to, for example, deploy component A & B on Server1, then deploy component C or Server2 and then continue to deploy D on Server1. This give us much greater control in orchestrating multi-component releases.

    It is often suggested that the choice is between Imperative- and Declarative-Based Deployment. However, these two approaches are not mutually exclusive.

    Workflow for Applications

    One of the issues in the world of DevOps is the lack of a consistent lexicon or terminology. Just ask someone who works in the industry what their definition of an environment is or what the term application means to them. My guess is you will get differing answers.

    To me, an application is made up of components and a release is generally in the form of an application that is created from multiple versioned components. Now, depending on the organization you work in, you may or may not recognize the definition I have given—and if everything in your world is a stateless loosely coupled microservice on a cloud native platform, then this may be the case. If you work  in a more traditional enterprise organization, you will more likely recognize the requirement for multi-component release orchestration, requiring procedural or workflow processes to handle the coordination and dependencies between the different components.

    Model for Components

    When it comes to deploying a specific component, I think it’s hard to argue against the benefits of using a Declarative or model-based approach. The concept of something being Idempotent—I can deploy any version to any target system if it has never been deployed to before, if we want to introduce a new change to an already existing environment or even if we want to roll a unit of automation back—is hard to argue against for its consistency and simplicity.

    Imperative Orchestration & Declarative Automation

    The future of IT Operations is doubtless based on things such as stateless microservices running in fungible containers, using Kubernetes watchers to trigger event-driven automation, much as the future of retail is for Amazon to deliver milk to my IoT-connected refrigerator using a drone.

    That being said, most of us still live in the present and have the need, for some time to come, at least, to address some of the automation and orchestration challenges that exist in today’s IT landscape. With this in mind, a combination of Declarative (or model-driven) units of automation, coordinated by Imperative (or workflow-based) Orchestration can be used to achieve this. An imperative/workflow-based orchestrator also will allow you to executive not only declarative automation, but also autonomous imperative units of automation should you need to.

    My recommendation is that an application workflow defines the order and dependencies of components being deployed. The Declarative Model for a component determines what action needs to be taken to bring a target system into compliance with the model.

    It is hard to underestimate the importance of desired state—it has changed the way we look at system and application updates and drastically improved the visibility, audit and control of IT systems. Any progressive organization looking to move to a continuous delivery/deployment model or implement things such as autoscaling or “deploy and destroy” for development or testing purposes will benefit from the use of declarative/model-based automation.

    However, a model-based approach does not answer all the difficult questions around orchestration. This, in my opinion (for the moment, at least) is best achieved using Imperative, Procedural or Workflows processes.

    About the Author / Dave Sayers

    Dave Sayers is a co-founder and technical services director of MidVision, a provider of Release Automation and Continuous Delivery Software. He works closely with customers and partners on DevOps and Automation initiaitves throughout Europe and North America and is responsible for devising and managing the partner and channel strategy. Connect with him on LinkedIn.

  • Strategies to Build Trust in DevOps

    Strategies to Build Trust in DevOps

    I’ve been a member of the DevOps community a long time (literally since before it was ever even called “DevOps”). I was a pre-publication reviewer for both “The Phoenix Project” and the recently published “The DevOps Handbook,” and I’m always interested in seeing the outcomes successes software teams have and the failures they overcome, including trust.

    While DevOps has proven to be effective in organizations of all sizes—startups and enterprises alike—there are still many who have not yet begun their DevOps journey, so I thought I might share some of my own experiences while introducing DevOps into an enterprise IT group.

    When I was the VP of Operations at Big Fish Games, I had the responsibility for production operations at Big Fish’s three production data centers in Seattle, Virginia and Luxembourg. I also had responsibility for internal corporate IT and information security. Our production operations supported around a million game downloads per day. Games were localized into 11 different languages. We processed essentially every major currency, and right before I left Big Fish, we started taking Bitcoin as well. From an IT perspective, Big Fish Games was a global eCommerce company selling digital widgets on the internet. We had around 800 employees in our five offices in Seattle, Ireland, Oakland, Luxembourg and Vancouver, BC.

    My first attempt to introduce DevOps at Big Fish failed miserably due to cultural misunderstandings and a fundamental lack of trust between the development and operations teams.

    You absolutely need some minimum baseline level of trust in your organization if you are to have any chance at implementing DevOps. Trust is important because you are blurring traditional boundaries between development and operations responsibilities. Including development in production operations can be scary for operations teams. It just feels wrong from a historical ITIL-type perspective, and operations teams may know development as “the people who always break things.” At the same time, it can be scary for development to involve operations in the development process because they may know operations as “the team that always says ‘No.’”

    Building Trust

    Successfully merging the priorities and goals of development and operations teams to create one cohesive DevOps effort can fail due to cultural misunderstandings and, oftentimes, a fundamental lack of trust. I used three specific strategies to overcome these challenges.

    Lose the Term ‘DevOps’

    At Big Fish Games, application development teams equated DevOps with developers carrying pagers: an idea that they really did not like. They were highly resistant to this. Every time I brought up DevOps, they thought I was engaged in a secret plot to make the developers carry pagers. What I found interesting about this was that I had no intention of asking developers to carry pagers.

    The term “DevOps” covers a large range of continuous improvement techniques. Some organizations have developers carry pagers, and it works for them, but this is certainly not a prerequisite for DevOps. What was worse, I had no idea they were thinking this because they were afraid to bring it up for discussion. We failed in our first attempts to implement DevOps because the trust level between development and operations teams was so low that we couldn’t even tell each other what we were thinking.

    I realized two things from this experience:

    • Getting rid of a term that scared both development and operations teams would eliminate an unnecessary distraction preventing these teams from coming together to focus on common goals.
    • We needed to do something about the trust issues between development and operations before we could succeed with DevOps.

    Once I realized this, I immediately stopped using the word “DevOps.” Instead, I talked about continuous improvement and specific continuous improvement methodologies that we wanted to implement, and how these new adoptions would result in shared, organizational-wide growth.

    Focus on Methods That Benefit Other Teams

    When deciding which of these DevOps, a.k.a. “continuous improvement,” techniques we wanted to implement, I specifically chose ones that would benefit development teams as opposed to ones benefiting operations or other teams. The development teams noticed this over time. They could see I was doing this and that I (the person who ran operations) was going out of my way to help them. This helped build trust. This wasn’t a one-time thing. We did this consistently.

    Here’s another simple failure example: I noticed while managing release efforts that release schedules for the various development teams ranged from a couple weeks to a couple months—too long for a hyper-competitive industry like gaming. I asked the development teams if they would like our release engineering team to support daily code releases or even multiple code releases per day. I thought this was an innocuous question. In fact, I thought I was being a little clever by playing dumb here. I assumed I would get an enthusiastic high-five along with a, “Yes, we’d love that! How soon can you make that happen?!”

    I was surprised when I received a violent “No! We don’t want that!” This confused me for a while until I realized that the development team thought I was in cahoots with the product management team to put the squeeze on them. In my mind, I was offering to make their release process more lightweight so they could break up their work into smaller, more frequent releases with less stress for everyone, but I didn’t understand their fears and problems. I wasn’t looking at it from their perspective and here was the trust issue rearing up again. They misinterpreted my intentions as nefarious.

    I pictured them being able to release code whenever they want, which seemed like something they would like. In their minds, they pictured a product manager constantly randomizing them with demands for new features to be released right there while the product manager was leaning over their shoulder. This was not the kind of problem you could resolve with logic. The solutions and working procedures I was proposing were so far outside the development team’s experience that they couldn’t extrapolate through simple logic from their current state to the world I was describing.

    Maximize Daily Interactions Between Development and Operations Teams

    It’s easy to distrust someone you don’t know personally or you don’t have to see every day. It’s a lot more difficult to hate on someone you know and work with on a daily basis.

    One of the DevOps techniques that I implemented early on was embedding a dedicated operations liaison into each development team. This liaison acted as an advocate for that development team back into operations and they helped smaller development teams navigate their work through a large, complex, centralized operations organization. The operations liaison attended daily development team standups, group meetings and project meetings. This program was a huge success. Development teams saw how their operations liaison helped expedite their work through the operations organization and advocated for their development team’s interests inside operations.

    After the development teams got used to working with their operations liaison on a daily basis, we pushed a bit farther and got the development teams to agree to include operations in architectural discussions at the beginning of application design. At first, our development teams were skeptical that operations could provide useful input here, but the important thing is we had developed enough mutual trust that they were not fearful of including operations in the discussions. They were skeptical that we would add any value, but they weren’t afraid that we would cause them problems. This was a big win.

    Although it felt a little weird to our development teams to include operations in design discussions, they quickly saw how obtaining input from operations early on, before a product was built, resulted in better quality and fewer headaches for everyone when we launched into production. I also made sure that we put our most senior and diplomatic operations team members into this role.

    In conclusion, it is worth repeating that trust is a prerequisite for any organization to succeed with DevOps. This is due not only to problems arising from fuzzy definitions of what DevOps is and is not, but also the seemingly dangerous idea of blurring the boundaries between development and operations organizations. Whatever your definition of DevOps is, fostering an increased level of trust between development and operations teams as a prerequisite for your DevOps journey will ensure that even greater trust—and the positive business impact that comes with it—are your outcomes as well.

    Good luck!

    About the Author / Paul Farrall

    Paul Farrall is the VP of Operations for Skytap. He was most recently vice president of Operations at Big Fish Games where he led the rollout of global infrastructure for the Big Fish cloud gaming platform and implementation of the Big Fish DevOps program. Prior to Big Fish, Paul was the Director of Operations and Chief Information Security Officer at Intelius, a Bellevue e-commerce company. Previous companies also include AT&T Wireless in Bothell, Washington, and GetThere.com in Palo Alto, California (now Sabre). Connect with him on LinkedIn and Twitter.

     

  • Increasing Your ‘Bionic Footprint’ with DevOps

    Increasing Your ‘Bionic Footprint’ with DevOps

    “The Six Million Dollar Man” was a 1970s TV series. In the show, NASA astronaut Colonel Steve Austin is severely injured in a crash and is “rebuilt” with bionic features that enhance his strength, speed and vision beyond normal human capabilities. The main tagline was, “We can rebuild him, we have the technology.”

    I recently was watching an old episode and it got me thinking about how the show is analogous to the genesis of DevOps—specifically, how individuals with an operations background have to “rebuild themselves” to keep up with the speed and direction of the technology industry.

    Over the years, there have been several decrees of “Unix SysAdmins are going away.” I have always firmly disagreed with that assertion. Rather, I believe that SysAdmins have to continually evolve their engineering skills to stay relevant and, more importantly, gainfully employed. It was also becoming increasingly apparent that the manual tasks of deploying infrastructure, keeping it patched, monitoring and the myriad other things a SysAdmin performs every day could not keep up with the demands of the increasing pace of the digital world and the onset of cloud computing. Infrastructure is moving rapidly to being automated and programmatically deployed and the talent needs to adjust accordingly to stay relevant.

    With many enterprises adopting the agile methodology and moving toward a continuous integration/continuous delivery (CI/CD) pipeline, the culture of DevOps became a reality. Due to the increased velocity, there became a clear need for the core tenets of DevOps culture, which include culture, automation, measurement and sharing (CAMS). In Six Million Dollar Man terms, this was simply “Better … Stronger … Faster.”

    Embracing DevOps through CAMS is similar to the superhuman enhancements that composed the rebuild of Steve Austin, which took him from man to bionic man. Here are the four ways that DevOps can increase your enterprise’s “bionic footprint”:

    Improve Culture

    Even Austin’s bionic features only got him so far; the bionic man had to rely on his team to complete his dangerous missions unscathed. The same is true in the enterprise: Building a collaborative culture is crucial for success. Culture is paramount in maintaining overall velocity without adversely affecting quality. Developers and operations teams need to function in concert to ensure a resilient application and infrastructure.

    Increase Velocity via Automation

    Automation greatly increases velocity, akin to how Austin’s bionic legs allowed him to run at high speeds and make great leaps. Automating processes saves time, money and lets your team focus on more complex tasks—a win-win for productivity gains.

    Measurement

    Steve Austin had a bionic left eye that enabled him to see in the dark as well as detect heat. This provided him precise measurements of his surroundings in any situation, enabling him to gain the upper hand against adversaries. Similarly, measurement gives insight into all aspects of the software development life cycle, application performance and other key metrics. Without insight and transparency into measurement, your team is essentially working blind.

    Sharing

    Austin had a bionic right arm that had the strength equivalent of a bulldozer. Strength and resiliency are increased by teams that work and communicate effectively made possible by DevOps.

    To summarize, I strongly believe that the mantra of “Better … Stronger … Faster” should be that of every DevOps team moving forward. DevOps’ parallel of CAMS upgrades your team’s capabilities to perform more productively. There is no end game; the objective is about continuous improvement, velocity and sustainable growth. Enterprises that want to undergo digital transformation need to adopt this cultural approach to increase their “bionic footprint” to succeed.

    About the Author / Mike D. Kail

    Mike D. Kail is chief innovation officer at Cybric. Prior to CYBRIC, Mike was Yahoo’s chief information officer and senior vice president of Infrastructure, where he led the IT and global data center functions for the company. Prior to joining Yahoo, Mike served as vice president of IT operations at Netflix, where he was responsible for employee technology and various cloud engineering components. He has been widely recognized for his insightful industry commentary on social media Twitter, and was recently named by the Huffington Post as one of the “Top 100 Most Social CIOs on Twitter.” Connect with him on LinkedIn and Twitter.

  • How I Bet on DevOps – and Won

    How I Bet on DevOps – and Won

    One evening in late 2015, after the kids were in bed, I sat down next to my wife and said, “I’d like to do something irresponsible. I want to gamble some of our savings on DevOps. I’d like to buy some stock.”

    Nicole Forsgren’s 2014 presentation, “DevOps and the Bottom Line,” opened my eyes to the business impact of DevOps. Then the 2015 State of DevOps Report rolled out with the following tucked into a footnote:

    The publicly traded companies that had high-performing IT teams had 50 percent higher market capitalization growth over three years than those with low-performing organizations.

    I believed the data. It matched my intuition and experience.

    DevOps and the Market

    Companies doing DevOps win in the market. In a world where digital transformation has made competition between companies increasingly dependent on their ability to differentiate through technology, the ones who do best in tech are likely to win. As I heard an executive at an airline put it, “Anyone can spend tens of millions on a new Boeing or Airbus jet. There’s no sustained differentiation there. But a great booking experience … a great mobile app … that can win loyalty.”

    So I found myself thinking that I sit in the middle of a DevOps community and know which companies are bragging about their DevOps successes. I truly believe it helps the business win.

    The Bet, and Winning It

    So that brings me back to that evening in December and the proposal I took to my wife.

    1. This was gambling. We’d risk only a fraction of our savings.
    2. We would buy and hold for one year, then re-evaluate.
    3. I would do no research on the companies we’d invest in outside of their DevOps practices. We would assume other public information would be priced in by professionals acting in an efficient enough market.
    4. Investments would be in companies “doing,” not “selling,” DevOps. No vendors.

    After getting the green-light, I bought a dozen stocks based mostly on the “who talks DevOps at conferences a lot” criteria: Amazon, Bank of America, Capital One, Cisco Systems, Etsy, Live Nation Entertainment, Macy’s, Netflix, Nordstrom, Sherwin-Williams, Target and Verizon Communications.

    So let’s talk about a few of these companies. Amazon and Cisco arguably violate the “no vendors” rule. Amazon I’ve excepted, as the company has a good reputation for DevOps internally, and many of the other retailers included are motivated to DevOps by fear of Amazon. They’re included as a hedge against Amazon killing all retail. In fact, Amazon was my only retailer to beat the S&P. Over the past three years, I found Cisco’s talks of its transformation at conferences (including IBM’s) compelling. Further, the hardware-to-network functions virtualization (NFV) transformation they’re going through seems similar enough to a retailer’s brick-and-mortar to e-comm to warrant inclusion.

    My employer, IBM, is not included because it is a vendor and as an employee its performance already has a big effect on my life. Etsy and Netflix are my “unicorn” contingent. Etsy crushed it for the year (+38 percent). Netflix underperformed (+6 percent) and was a major drag through the first half.

    Retail and e-commerce was generally a weak segment with Live Nation (Ticketmaster), Macy’s, Nordstrom, Sherwin Williams and Target all underperforming the S&P 500. Compared to a retail index fund (XRT), all but Nordstrom and Target came out ahead, though. It was a rough year for the segment, but those investing DevOps seemed to outperform their competition.

    All in all, the approach beat the S&P 500 by about half for the year, albeit with considerably more volatility.

    Good Sentiment, Sketchy Investment Strategy

    That said, a deeper look at this approach makes me think it was more luck than skill. While I believe that high-performing IT yields business wins, I didn’t actually know which companies have high-performing IT. The DORA team has the best data. I don’t. A big financial institution is likely to have more techies than Facebook. Do a handful of them presenting at conferences really say much about their thousands of colleagues? I have doubts.

    DevOps transformations, like Agile transformation, are hard to do at scale. It’s the scale that counts, and will generate the business differentiation. Some transformations led by brilliant champions will end up failing. Mid-year, I was at one of our DevOps networking events talking with someone from one of the companies in this list and suggested that they must be nailing DevOps given what I heard at conferences. The answer was, “Not really … we’re kind of starting over and the people championing DevOps have all left.” I’m not surprised that company is one of my underperformers.

    Sophisticated investors will point out approaches for managing exposure to segments and better isolate DevOps effects from overall market moves. Not researching and using those is another flaw in the approach.

    And yet, if picking stocks at random would yield average results, but DevOps helps company win, and people talking about DevOps likely represent companies with more DevOps than others, it’s not shocking to see this strategy happen to get lucky.

    What Will I Do Next?

    My focus will remain on ensuring the IBM UrbanCode business embraces DevOps practices, mindsets and approaches in every way and supporting IBM’s overall transformation. While transforming myself, I’ll remained focused on helping my clients with their own transformations.  Check out some DevOps success stories.

    As for investing, I’m not a gambler at heart and may simply close down this experiment while I’m ahead and pick a nice index fund. Or, I may refine the picks to only take those that are publicly talking about how they spreading these practices to thousands of practitioners.

    Disclaimer: I’m a techie, not a financial advisor. I recommend improving your own business with DevOps. I do not have a recommendation for how you should invest. Past performance is not indicative of future results. This experiment could have gone badly, and for a while in the first quarter, it looked as though it would. Please don’t take risks you can’t afford based on the experience of a rank amateur investor like myself. This experiment was mine and mine alone. I speak for myself, not my employer.

    About the Author/ Eric Minick

    Eric Minick is a technical evangelist. He joined IBM through its acquisition of UrbanCode. He has spent the last 10 years helping organizations large and small adopt continuous integration, delivery and, now, DevOps. He is a frequent speaker on the topic, and co-authored “Application Release and Deployment For Dummies.” Prior to consulting, Eric was a developer, test automation engineer and support engineer, and has contributed to multiple generations of UrbanCode products. Follow him on Twitter at @ericminick.

    — Eric Minick

  • Forecasting Culture: Sunny with Accumulating Collaboration

    Forecasting Culture: Sunny with Accumulating Collaboration

    Bryson Koehler believes the most important enabling factor for DevOps success is culture.

    Koehler is CTO of IBM’s newly formed Watson and Cloud Platform group. Prior to this role, he served as executive vice president, distinguished engineer and CIO/CTO of The Weather Company (TWC), an IBM business since 2016. He has a record of successfully leading significant business transformations and underlying cultural transformations, which he maintains in tech companies must be incited first at the developer level. As such, while at TWC Koehler built a DevOps culture and practice that encourages transparency, accountability and collaboration, and it is his mission to replicate the successful model in the transformation of IBM’s culture.

    Since I experienced some of the TWC culture firsthand, I asked if he would be willing to share his thoughts on enabling a high-performing culture as well as maintaining that culture during periods of rapid growth or change.

    He started the interview with a theme clearly central to his core ideology as a transformational leader, and one of TWC’s key factors for success:

    Culturally, the goal is to find people who really love what they do. I want people to wake up in the morning and love what they do—not tolerate it, not like it, but love it. We work very hard to find people who love what we do and love the mission we are on. If you work very hard to assemble a team that loves what they do, the culture will take care of itself as long as you enable it … it’s about enabling them, with as much freedom and flexibility as possible, while still maintaining some level of consistency and control. It’s a balancing act and at TWC we worked very hard to give the right tools that enable people to work on any device, anywhere, at any time. The tools were an enabler; it was the culture that really invented itself based on the fact that we hired people who love what they do.

    Koehler’s commitment to finding the right people with the skills and passion consistent with his company’s mission put the base focus of enabling DevOps practices where it needs to be—with the people.

    For those that have read some of my other blogs, one of the things I’ve written about before is the mistake of choosing tools without first addressing the critical underlying aspects of DevOps including, a solid agile foundation and the cultural changes required to facilitate DevOps practices. Needless to say, Koehler’s comments about tools being an enabler really resonated with me because it was clear he still views the tooling as very important but puts the real emphasis on having the right people. The other key message that really resonated with me during the interview was his commitment to accountability and transparency:

    … from a leadership perspective, it’s around making sure there is clear accountability. There is single-person accountability for work. If you walk into a meeting and you don’t know who the single decision maker is in the meeting, my recommendation to the team is to leave. Don’t have the meeting, because it’s a waste of time. There should be one person that everyone knows is the decision maker, whatever the topic. If we don’t know that, then it’s my job to make sure I make that crystal clear. You can’t walk into a meeting with five people all thinking they own the decision when also working to create a culture with high velocity, high throughput and high passion towards meeting objectives and getting stuff done. You hire people who love what they do, you enable them with great tools, you have crystal-clear accountability and decision-making so that things can get decided quickly and that we know who is accountable for making sure that decision was successful, and then constantly work against the goals of the business. You also have to be very willing to adjust your tools, very willing to adjust your people, and very willing to adjust your decision-making. We must be constantly willing to look at ourselves and figure out how we improve.

    Koehler’s willingness to continuously look inward and adjust what is not working demonstrates another key success factor for high-performing DevOps teams. Having an executive driving that culture of continuous improvement is critical for DevOps success.

    He also is dedicated to core agile principles that allow for flexibility and responsiveness while maintaining some degree of focus and clear prioritization. With the rapid TWC growth, we also discussed how he maintained and adapted the existing culture in response to that growth:

    It’s really critical for me as a leader to stay very in tune with how these teams work and function, calling out any bad behavior and correcting it quickly. I’m just as willing to roll up my sleeves and get dirty as the next guy, and I expect everyone else to do the same. Kingdom building is not something that will get you very far in how we work. If you do that [roll up your sleeves], then as part of your culture, everyone puts down their guard and realizes that nobody is any more or less important than anyone else. We are all here to accomplish the same thing.

    Common goals and people that love what they do! It’s a great mix and Koehler has some fascinating perspectives on driving cultural transformation in support of DevOps practices, not only for the efforts he successfully led at TWC but for the efforts he will continue to lead within IBM. His commitment to culture can best be wrapped up by one of his final comments:

    None of our great technical ideas has any chance of seeing the light of day if we don’t get the culture right.

    Thank you again to @brysonkoehler for the interview and insight into TWC’s amazing culture and your vision for IBM!

    About the Author / Bryson Koehler and Shelbee Smith-Eigenbrode

    Bryson Koehler is CTO, IBM Watson & IBM Cloud – GM & Distinguished Engineer at IBM Connect with Bryson on LinkedIn and Twitter

     

    Shelbee Smith-Eigenbrode is a Senior Software Engineer/IT Architect at IBM and has been in IT for 19 years.  She has worked across varying roles spanning the software development lifecycle.  Her experience across functional roles as well as performing that work deeply inside traditional silos has contributed to her passion for embracing the DevOps culture and transforming the way teams are delivering technologies by adopting DevOps practices.  She is a member of the IBM Academy of Technology as well as an IBM Master Inventor committed to driving innovation and technical advancement. Connect with me on Twitter @Shelbee_SE.

     

     

  • DevOps in 2017: From Building to Executing

    DevOps in 2017: From Building to Executing

    In 2016, DevOps grew in popularity and use, new tools were introduced and enterprises across industries realized the influence and benefits derived from agile development. But there are even more developments to look forward to as enterprises engage DevOps teams further.

    Scaling-out solutions, providing a baseline for a new cultural approach to problems and continuous improvement will continue to be priorities. But there are a few items that we’ll likely see come to the forefront in the new year. In fact, 2017 should be an even more exciting year for DevOps.

    Here are my top four predictions for what we can expect this year:

    The next phase of DevOps will focus on security

    Security was a perennial news event throughout 2016, and we can expect the same in 2017. However, DevOps teams likely will be challenged to build good security practices into code as a part of application development, rather than refining security post-facto. This will lead to DevOps, QA and operations bringing security teams deeper into the DevOps environment, if not outright incorporating them as a part of the team.

    The foundations for this development were put in place in early 2016 as the DevSecOps model gained traction in cybersecurity circles throughout the year. In 2017, we’ll see DevSecOps teams build on these foundations. After all, every company is at risk for a data breach.

    Containerization will go mainstream

    The convenience of containerization makes it one of the most desirable technologies for DevOps teams, and next year we’ll see it gain even greater use. It’s easy to understand why: Containerization solutions, such as Docker, grant teams the ability to provide a consistent environment from development to production. We may even see containerization move to non-production environments over the course of 2017 due to its portability across multicloud platforms. And that portability is critical not just for production purposes but for security as well.

    DevOps will extend into pay-as-you-go

    DevOps will, almost by necessity, develop an on-demand model in 2017. Companies are realizing the stark demand for DevOps skills and they will be meted out according to internal needs and external demands.

    In particular, we’ll likely see a lot of opportunity for DevOps in cloud implementation, since enterprises leverage solutions across cloud providers. With cloud provider “lock-in” still a real concern for many enterprises, any provider capable of operating cross-cloud development—thereby allowing customers to switch to a low-cost cloud provider—will be in high demand. This is good news for customers, as it allows them to finally benefit from the elastic nature of cloud pricing models.

    DevOps will become pervasive across the enterprise

    Finally, enterprises will use DevOps across the enterprise. Thus far, large enterprises have been experimenting with DevOps in discreet projects, but they have started to plan for employing DevOps principles at enterprise-level, and we will see this roll out next year.

    Watching the Shift

    Thus far, DevOps has been focused on adopting and orchestrating technology and tools to build automation pipelines. But moving ahead, we’ll witness DevOps change its focus from building to executing. With this industry development and the shift within DevOps to security and containerization at an enterprise level, we’ll begin the true potential of DevOps.

    About the Author / Tan Moorthy

    Tan Moorthy is Head of Global Services for Application Development and Management at Infosys, a next-generation IT services provider. Connect with him on LinkedIn.

     

  • Compuware Survey Connects Mainframes, DevOps

    Compuware Survey Connects Mainframes, DevOps

    Mainframes are still necessary; integrating them with modern hardware and software is tricky; and DevOps is the answer. Those are the takeaways from a survey out this week that was commissioned by Compuware.

    The survey, which was released Dec. 14, was conducted by Forrester. It asked professionals how often their organizations rely on mainframe systems and what they do to overcome the challenges associated with using legacy mainframes within modern computing environments.

    The survey suggests that mainframes are not going anywhere soon: 57 percent of respondents said new business initiatives “always” involve the mainframe, according to Compuware. Even though mainframe systems are now decades old, they still have a place in modern IT.

    But bridging the gap between mainframes and more modern infrastructure and software delivery practices is tough, according to the survey. This is due not only to what Compuware calls “organizational bureaucracy” and “insufficient funding,” but also the difficulty of integrating mainframes with other types of technology that were designed in a different era.

    The survey shows that most professionals take one of three approaches to solving that challenge:

    • Move applications off of mainframes. The downside to this strategy is that mainframe applications may not run as fast or as securely on other hardware.
    • Avoid using the mainframe at all. This, too, is not always ideal, according to Compuware, because the mainframe is better at handling some types of workloads.
    • “Modernize” mainframes, which entails a lot of investment and effort.

    According to Forrester’s conclusion in the study, a better approach is to apply DevOps techniques to mainframe development. That means that the mainframe technology itself does not have to change or be hybridized; only the approach to mainframe app delivery changes. “Those companies embracing DevOps deliver faster and at a higher quality, all while fostering collaboration,” Forrester concluded.

    These are all things that a Compuware-sponsored study is likely to say, of course. Compuware specializes in DevOps solutions for the mainframe.

    Still, the survey is a good reminder that mainframe is alive and well—and DevOps teams should not leave it out of the picture.

    — Chris Tozzi

  • 12 Tips for Managing Your Multi-Cloud Environment

    12 Tips for Managing Your Multi-Cloud Environment

    No single cloud vendor has the perfect solution for your unique business situation. The only way to maximize the benefits of cloud services to your business is by dealing with several cloud providers and monitoring all of them simultaneously. That’s the IT equivalent of singing an aria while standing on your head and scrambling eggs. (Not really, but it can seem that way.)

    Here are 12 things to keep in mind when planning your company’s multi-cloud strategy:

    • Map your entire network, and then identify where the cloud fits in it. Get a clear picture of the cloud’s role in your overall system-management strategy to avoid having to backfill gaps in the data needs of business managers and other customers. Different lines of business will be best served by different cloud providers.
    • Work with vendors that respond quickly and positively to the unexpected. There will always be unforeseen situations when you’re stitching together multiple cloud services. Your partners should report regularly on the state of their services, enhancements, and new features.
    • Automate low-level monitoring and maintenance tasks. Two primary reasons for adopting cloud services are cost savings and more efficient operations. One way to achieve both is by reducing the amount of human oversight required to keep your systems humming.

    The potential complexity of multi-cloud management is shown in this flowchart encompassing the many elements that need to be stitched together precisely. Many of these low-level operations must be automated to optimize your strategy. Source: Carl P. Lehrmann, 451 Research

    • Follow your users. Pay close attention to how the “consumers” of your apps and systems are actually putting the tools to use. They are the best indicators of which processes are the biggest productivity boosters. You can also learn about any new services the users discover that help them get their work done faster, more efficiently and/or less expensively. This lets you get a jump on the kinds of tools the users are most likely to need in the future.
    • Double down on cost controls. You can’t control what you can’t see. Dealing with multiple vendors complicates the process for requisitioning new services. To avoid unnecessary impediments to quick implementation of useful new services, devise a purchasing process that is as flexible as the services themselves. However, not long after the service honeymoon period, evaluate whether each new service is proving to be worth the cost and whether there are better and/or less-expensive alternatives.
    • Application life cycle continues to dictate cloud migration. Data centers may be shrinking, but they are definitely not disappearing. For reasons related to security, compliance, insurance, and any number of other factors, some systems are staying in house for the foreseeable future. You certainly want to avoid attempting to lift and shift an in-house app to a cloud service, particularly one that is only a couple years old and thus hasn’t yet paid for itself. Once an older system is ready for a reinvention, it’s easier to make a business case for going cloud-native.
    • Take advantage of the power of pre-emptable virtual machines. Because they persist for no longer than 24 hours, pre-emptable VMs are a good choice for batch processing and fault-tolerant deployments, according to ComputerWeekly’s Matthew Cooter in a November 2016 article. Pre-emptable VMs are as much as 80 percent less expensive than Google’s regular instances, and their fixed price help companies plan and manage their costs.
    • Adopt an integrated data center management system designed for virtual environments. This is the only way to ensure your server, storage, network, security, operations and applications teams are working as a single unit toward common goals, according to Jay Litkey, president and founder of cloud service provider Embotics, in a November 21, 2016, article in Virtual Strategy magazine. Any organization structured around traditional physical infrastructure boundaries is destined to struggle with the shift to hybrid and multi-cloud environments.
    • Identify the apps in your organization that are best suited to cloud-native implementation. Traditional apps are monolithic, run as VMs, use a scale-up architecture and are generally more difficult to develop, deploy and maintain. By contrast, cloud-native apps are more modular and service-oriented, comprised of collections of containers and services, based on a scale-out architecture, and easy to automate, move, and scale.

    Focus on the apps in your organization that will benefit most from a cloud-native approach, and wait on those that are better served by deployment on a traditional data-center platform. Source: TechTarget

    • Match your apps to the services that play to their strengths. Big-name services such as VMware and AWS have bifurcated to meet the diverse needs of their customers. For example, VMware’s vSphere-based Enterprise Hybrid Cloud is intended for traditional apps, and the company’s Native Hybrid Cloud is designed for modular, microservice-based systems. Similarly, AWS’s Elastic Compute Cloud, Elastic Block Store, EFS file system and VPC private networks mimic in-house data centers, while the company also supports higher-level services such as NoSQL databases, BI processing, Hadoop-like clusters, message queues, push notification services, media transcoders and search engines.
    • Automate policy across cloud environments. The only efficient approach to use of multiple cloud services is to have a single standard for policies that are applied automatically to each environment. The policies cover such areas as virtual servers, workloads, data storage, traffic flows, compliance/regulation, reporting and security. It’s also easier to apply updates and changes when you have a single configuration across different infrastructures because the changes propagate seamlessly from environment to environment.
    • Vendor lock-in becomes a thing of the past. When you rely solely on one public cloud service, it can be difficult to migrate virtual machines and other resources to another service. Among the factors to consider are comparable read/write loads, data transfer needs and network latency. By relying instead on your custom combination of cloud services, all of which share policies and configurations, you find yourself managing the change instead of the change managing you.

    — Brian Wheeler

  • DevOps Survey Says! State of DevOps, Reliance on Scripting and IT Automation

    DevOps Survey Says! State of DevOps, Reliance on Scripting and IT Automation

    What our surveys told us about the state of DevOps, Reliance on Scripting and IT Automation

    DevOps Survey Results: Scripting & IT Automation

    A lot of IT Operations teams including application owners, configuration managers, middleware specialists and database administrators are managing infrastructure configuration change with manual processes, spreadsheets and scripts. They are spending a lot of time writing and maintaining a lot of scripts. Even with all those scripts, spreadsheets and processes and the resources they consume, teams are still not getting the visibility, control and error-proofing that they require.

    That is the message from five recent surveys conducted by Orcaconfig among IT industry pros. Perhaps these results should not be that surprising. But several years into the DevOps and IT automation movements, there is still plenty of room for improvement.

    About one-third of respondents said they spent more than half of their time writing and maintaining scripts.

     

    In complex application environments with multiple relationships and dependencies between applications, databases, middleware and operating systems, “changing this can easily (and unwittingly) change that.” Gartner refers to this as a domino effect. So, scripting to deploy changes becomes more and more tricky and thus, requires more and more time to do, test, redo and retest.

     

    		  what_percentage_of_time_is_spent_writing_or_maintaining_scripts_-_survey_from_orcaconfig-com_2

    Because of the risks involved with release or configuration-related scripting errors in production environments, some organizations hand these typical operations responsibilities to developers. The question is worth asking: Is release and configuration-released scripting the best use of Dev or Ops’s time?

     how_do_you_maintain_scripts_when_the_author_is_no_longer_available_-_survey_from_orcaconfig-com_3 With all these scripts holding key configurations together, are organizations at risk of “losing the recipe” if key employees leave? Apparently so. In our surveys, 30 percent of respondents admitted, “It’s a problem, we don’t really have a solution in place,” when asked about maintaining the spaghetti-works of scripts if some of those key employees suddenly leave the organization. Nearly half mentioned that they rely on transfer of information (TOI) before a key employee leaves.

     

    Employees exit companies for a number of reasons and some are in a bigger hurry than others to move on. Does TOI provide all the safeguards needed for the enterprise?

    Scripting might consume a great deal of IT resources and introduce a considerable risk with the loss of key talent. But does it at least accomplish the functionality that teams need? The next few charts provide insight to how well scripting achieves the visibility and error-proofed automation that teams need.

    Ecosystem Visibility

    Configuration Drift Detection

    For performance, security and audit reasons, organizations have an ongoing need to maintain configuration compliance in their application ecosystems. So detecting configuration drift is essential. Automatically detecting configuration drift is the goal. Despite complaints about the method, approximately half of respondents relied on scripting to accomplish drift detection.

    		  how_does_your_organization_detect_configuration_drift_in_complex_application_database_middleware_environements_-_survey_from_orcaconfig-com_2

     

     how_do_you_know_which_databases_are_being_used_by_which_applications_across_your_ecosystem_-_survey_from_orcaconfig_2 Mapping Databases to Applications:

    Sometimes IT operations are asked simple sounding questions such as, “What are all the applications that use this database?” and vice versa, “Which databases support this application?” Once again, nearly one-third of respondents admitted, “We really don’t know which databases are being used by which applications. Wish we did.” Another 36 percent relied on scripting or other manual processes to get that answer.

     

    Is it reasonable (in 2017!) to still rely on scripting to get basic information about interrelationships between applications, databases and middleware? Is it acceptable to operators, managers and auditors simply do without this information? Will Visio and similar static files always provide up-to-date information that IT Operations needs?

    Automation & Error-Proofing

    Environment-Aware Releases
    When moving an application through pre-production stages and into production environments, Release Managers want to ensure that the new release will just work … as planned … drama free.  This requires among other things that

    1. the configurations are up to date and production-ready, and
    2. that those business-critical applications are promoted simultaneously with their production-level configurations.
    		  how_does_your_organization_achieve_environment_aware_releases_

     

    Thirty-eight percent of respondents said they relied on scripting and other manual processes to try to accomplish environment-aware releases.  To ensure ‘everyone is on the same page’ many organizations use multiple planning meetings, sharing and updating spreadsheets and SharePoint files with (hopefully) the latest configuration information. Even among this group, respondents commented that scripting just does not work well for them.

    scripting_does_not_work_well_for_environment_aware_releases_-_survey_from_orcaconfig

    Despite the security, compliance and performance implications of configuration and release problems, 15 percent of survey respondents admitted they “really don’t achieve environment-aware releases.”

    Scripting isn’t Free; IT Ops Needs More

    Scripting is quite naturally the go-to method for many application owners and IT Ops teams. The cost of scripting or even finding and fixing a scripting-based error hardly ever appears clearly in budgetary line items. In the moment, it can feel like it is free. But over time, maintaining scripts and homegrown tools can build a hidden but costly mountain of technical debt. It might well be worth these costs and risks if scripting effectively resolved ongoing operations needs such as ecosystem visibility and drama-free deployments. However, time and again our respondents said they lacked basic capabilities such as configuration drift detection, mapping applications to databases and middleware or ensuring environment-aware software releases.

    — Scott Turner