devopsy.kinsta.cloud

Category: Features

  • Q&A with Mark Carrizosa of Soha Systems on secure transition to DevOps and cloud computing

    Q&A with Mark Carrizosa of Soha Systems on secure transition to DevOps and cloud computing

    We recently spoke with Mark Carrizosa, vice president of security at secure application delivery service provider Soha Systems about the trends he sees with enterprises moving to cloud services. Prior to recently joining Soha Systems, Carrizosa held the position of principal security architect at Walmart where he developed and implemented the company’s global e-commerce security architecture framework. Prior to Walmart, Carrizosa held security positions at companies including Wells Fargo, where he analyzed the company’s infrastructure and application compliance to improve the security of both customer-facing and internal systems. Before that, Carrizosa held positions at Freeport-McMoran and PetSmart. This interview has been edited for brevity.

    devopsy.kinsta.cloud: With the rapid move to DevOps and cloud, are there opportunities to improve automation and for enterprises to do more with less? Or is DevOps moving so fast that there are additional challenges being created that outweigh some of the benefits?

    Carrizosa: In my opinion, and in my experience, particularly over the last few years in dealing with predominately DevOps shops, the challenges that we faced were not necessarily “can we automate something,” rather it’s “should we automate something.” There are absolutely some things that should not be left entirely out of the automation stream, but should have controls setup as a requirement at a staging gate. For example, you can automate the dynamic scanning your application for vulnerabilities, but what you also need is somebody to actually look at those applications and manually verify and make sense of the reports. These are key components in deploying and delivering secure applications.

    There are also ways to expedite processes by integrating certain security frameworks into your workflow, providing the right training, and also creating a shift in your culture. And DevOps, for the most part, is a good thing. It allows organizations to rapidly come to market.

    It’s also common in DevOps to have contentions between security and IT. The security group says “No, you can’t do this, it’s going to hurt us.” While the engineers say “We need this to push this out.” How well organizations manage these relationships is a mixed bag. But there is a formula for doing it successfully, and it is all about ingraining security into the culture, into the operations, so that you come out with a secure product and not have to remediate it once it’s done.

    devopsy.kinsta.cloud: When you’re constantly iterating and changing your infrastructure and deploying new apps and updates, how does that affect aspects of security that can’t be automated, such as threat modeling applications before they are built or changes are put in?

    Carrizosa: Threat modeling, while a new term, it’s not a new concept. If you think about risk management, typically audits are done at a point in time. But very good information assurance programs not only audit a point in time, but they implement a continuous model of checks and balances. So should threat modeling.

    Threat modeling needs to happen at inception to ensure that the concept is sound and the methodology is acceptable from a security perspective. But there should also be some levels of checks and balances to ensure that the baseline, or security posture, is maintained throughout the entire change management cycle from inception all the way to delivery.

    These are some of the things that can be embedded within DevOps and Agile. Microsoft has a very good, secure lifecycle management methodology that incorporates various elements of security checks throughout the entire development lifecycle. It’s something like this that needs to be included within these operations to ensure that every step you take has some level of checks and balances, whether it be automated or manual, to ensure that the process of an application or an environment is up to what organizations are requiring.

    devopsy.kinsta.cloud: There’s still considerable interest out there when it comes to how companies are meeting the challenges of moving to cloud services. Are there common threads you see out there when it comes to the challenges enterprises face when moving to cloud services?

    Carrizosa: Yes, absolutely, and one of the key factors is that organizations are looking for ways to securely move to the cloud, or move to a more shared compute environment while still maintaining the control and the reduction of risk that they are used to internally. I spent the majority of my career dealing with these topics from an enterprise level. Most recently at Walmart E-commerce, but I’ve also done the same for PetSmart, Wells Fargo, and some leading industrial companies throughout the industry.

    Some of the challenges that I faced have to do with public exposure. You have to deal with migrating from on-premises to cloud, and dealing with BYOD enterprise users. There are a number of potentially sensitive customer-facing applications or data sets that organizations are a little hesitant to move to, particularly at large organizations.

    Those large organizations are used to doing things a certain way. Time and time again I’ve heard the phrase, “but we’ve always done it this way.” That in essence could be some of the downfall, or some of the hesitance with these organizations in moving to the cloud, because they are not absolutely sure that they will maintain the control and the effectiveness for environments that they can’t manage.

    That’s what Soha is about. Soha allows these organizations to make that next step, to be able to take advantage of these cloud benefits, or even take advantage of some of the cost overhead and the security controls that they’re typically having to address at a high rate of cost and a great deal of effort.

    devopsy.kinsta.cloud: Broadly, considering a lot of the credit card and other significant breaches that have occurred in the past decade, do you think the sense of control enterprises have about how well their security and governance efforts are doing is illusionary?

    Carrizosa: Well, there’s that old adage of, “it’s never ‘if,’ it’s always ‘when.’ These recent breaches, for instance, are not necessarily based on the types of controls that organizations have, but a lot of times legacy controls or legacy approaches have always seemed to be good enough. And until there is some sort of catalyst to effect change, organizations aren’t necessarily going to stick their necks out. You have CIOs saying, “We do this, we do this, and we do this,” because that’s what they’ve always known. It’s time to now shift to a more open model. With that comes many risks, but in order to do so, there needs to be a fundamental change in the methodology of how organizations are tackling security.

  • Will DevOps Kill Outsourcing?: Part 3

    Will DevOps Kill Outsourcing?: Part 3

    Part 3: What’s the differentiator? Understanding how to balance build vs. buy

    (Check out Part 1 and Part 2 of the series Will DevOps Kill Outsourcing?)

    The DevOps movement has spurred a lot of organizations to reevaluate their IT outsourcing strategies. But the reality is that there will be no cataclysmic shift of insourcing. Sure, there will be plenty stand-out case studies and successes among rapid first adopters in the swing away from massive outsourcing. But like all of the biggest sea changes in IT, things will happen gradually when we look across the industry.

    “At the moment, it isn’t necessarily  that enterprises are bringing development back in-house – instead what  we’ve seen is that larger organizations attempt to adopt DevOps and stand  up new teams for new applications in-house, rather than contracting those functions out as they might have done in the past,” says Dan Cornell, CTO of Denim Group. “So, over time the center of gravity will be shifting back toward in-house development. But that will take time.”

    And that won’t even mean that over time DevOps will completely sound the death knell over outsourcing. Most experts believe it’ll always play a part in the IT ecosystem.

    As Alex Henthorn-Iwane of QualiSystems says, he’s observed among customers that even when they bring a lot of functions back in house in order to support DevOps, these companies are still keeping major development processes outsourced. They’re just finding a way to fold them into new processes.

    “So far, it would seem that the pieces of the puzzle that are insourced are the strategic glue pieces,” he says. “There is still a lot of dependence on outsourced, contracted or vendor-delivered services and technology in our customer base.”

    In order to better fit outsourcing engagements into a DevOps-enabled enterprise, organizations will have to get smarter about how they construct their relationships with outside support. It’ll need to be done with flexibility top of mind, says Jeff Gallimore, founding partner at Excella Consulting.

    “They’ll contemplate how that support will integrate with the rest of their staff and work with those who can and do collaborate with their staff. Contracts will be more flexible and accommodating of change,” he says. “That could mean smaller, lightweight, short-term contracts or contracts that provide simple, trust-based change mechanisms.”

    It’ll also require organizations to think carefully about the balance between building, buying and outsourcing the technology that they’re delivering.

    “A  rule of thumb in IT is to insource the most valuable core competencies of the company,” says Andrew Storms, vice president of services for New Context, who explains that’s easier said than done.

    The problem is that as the pendulum shifts to insourcing many people are “poorly assessing which software is the differentiator,” says Theo Schlossnagle, founder of Circonus. There’s a point at which it doesn’t make sense to build something, otherwise everyone would be building it all themselves instead of using off-the-shelf software.  In his mind, the majority of IT effort today is spent integrating, not innovating.

    “When you conflate integration with innovation you lose,” he says. “The goal should be to buy over build when possible, but innovate in-house precisely those things that are core to your business that can change how you compete in your market.”

     

  • The Art of DevOps

    The Art of DevOps

    This is the third in a series of posts on DevOps. The first written by my colleague Lee Reid was titled The Simple Math of DevOps. The second The Calculus of DevOps was written by IBM client, and my friend Carmen DeArdo of Nationwide. Lee introduced mathematically improving delivery throughput by leveraging DevOps to improve Trust. And Carmen used Calculus to imply that the business value of IT can be improved by adopting DevOps to increase the frequency of releases. In my post I would like to visit the art of DevOps , or the art of adopting DevOps by picking up right where Carmen left off – by looking at the culture side of adopting DevOps.

    Lee and Carmen have already alluded to some basic premises behind DevOps as an approach – reducing batch sizeand improving collaboration. These result in reducing complexity, increasing throughput, and improving trust. Let’s talk about how to get there.

    As I have worked with customers adopting DevOps, across company sizes, across industries and countries, there are four common threads that make DevOps adoption successful, leveraging all three aspects of DevOps adoption – culture, process and automation. These are:

    1. Reduce batch size: The most effective way to manage risk and quality, while increasing speed is to reduce the batch size in each iteration or ‘sprint’. This is a mind shift to release smaller, more frequent new versions. Where it is not possible to release new versions that frequently to customers, release small batches to a pre-production area. The deliverable in pre-production is tested and made customer-ready, and then released to the customer at formal release dates. As Lee and Carmen have both discussed, reducing batch size is a pre-requisite to improving both trust and business value.
    2. Shift-left Operational engagement: Get operations engaged right from the beginning of a project/initiative. This has been a core principle of DevOps. Do not leave Ops in a silo that one throws code over the wall to, but have them engaged in the development process, right from requirement inception stage. This allows Ops to have visibility into what is coming from Dev; allowing them to deliver the right production-like environments, as and when needed to Dev and Test teams. It also allows Ops to shift operational concerns left, left into the minds of developers, making them more Ops conscious and engaged in what happens after they hand-off to Ops.
    3. Continuous funding: This is a significant change in an organizations funding model to enable continuous availability and engagement of SMEs and business functions, previously not engaged on a continuous basis – like security, legal, marketing, etc. Carmen in his post mentioned several examples of ‘waiting for…’ work, someone else to do something and environments. Another ‘waiting for’ in enterprises is for funding. Funding is typically provided in a waterfall manner, with specific hard dates (Months, quarters, fiscal year) and gates, not suitable for a Continuous Delivery model. Funding too should be continuous.
    4. Create a ‘Product Management’ team: This team includes a Designer, Development lead, operations lead and an Architect at the minimum (4-in-box). They own the ‘product’ through its entire lifecycle, and beyond transient projects. They are responsible for long-term thinking, design thinking, evolutionary architectural thinking, and overall ownership of the product, from concept to end-of-life.
    5. Setting up a DevOps Center of Excellence (CoE): This center of excellence is not an administrative organization or a ‘tools/enablement group’, but a place where DevOps adoptees come to learn from each other and share expertise and lessons learnt. As organizations adopt and scale DevOps adoption, this CoE also becomes a source of DevOps coaches, helping teams and programs adopt DevOps, and own the organizations DevOps framework – their own flavor of DevOps.

    Adopting DevOps is an art. It is not as simple as hiring a consultant who can show how to improve processes; it is not just buying and adopting tools to automate manual tasks; and it is not going through ‘fall-back-into-the-arms-of-the-person-behind-you’ exercises to build trust. It is a combination of all three – process improvement, organizational and cultural change, and automation with tools to replace manual processes. Adopting these in any enterprise, outside of startups, requires overcoming organizational inertia. By changing culture not by edict, but by showing the need and value of change. By building trust, not through meetings, but with improved communication and transparency. By improving business value, not by measuring mandated KPIs, but by improving organizational agility and throughput. Overcoming organizational inertia, by delivering towards a common goal for the business, irrespective of where in the organization one is, and what role one has. By developing a culture where everyone takes responsibility of delivering value to the business.

    Watch the webcast series hosted by devopsy.kinsta.cloud in partnership with IBM or download a copy of DevOps for Dummies book for more information.

    About the Author/ Sanjeev Sharma

    SanjeevSanjeev Sharma, CTO and Distinguished Engineer – DevOps Technical Sales and Adoption, IBM Cloud Unit, is a 20-year veteran of the software industry with expertise in DevOps, Mobile Development and UX, Lean and Agile Transformation, Application Lifecycle Management and Software Supply Chains. He is a DevOps Thought Leader at IBM and speaks regularly at conferences. He has written several papers and is the author of the DevOps for Dummies book.

     

    Contact Sanjeev on linkedintwitter

  • DevOps killing outsourcing? Another point of view

    DevOps killing outsourcing? Another point of view

    After reviewing the articles (here and here) on DevOps driving the end to outsourcing I felt it would be constructive to provide an alternative point of view.  The premise of the article is that because DevOps requires close collaboration between development and operations that it would be difficult if not impossible to effectively implement DevOps in a heavily outsourced environment.  Additionally the productivity advantages of DevOps would help eliminate the need for outsourcing.  Therefore, organizations wishing to see the advantages of DevOps should move away from outsourcing.  While I can see the advantages of that approach, I also think that companies that have to live with outsourcing can’t ignore the benefits from implementing DevOps principles with a well-designed deployment pipeline.

    From my perspective, companies looking to implement DevOps first need to start by understanding their architecture because it has such a large influence on how to effectively implement DevOps.  If they have loosely coupled architectures that enable small teams to independently develop, qualify, and release code then they should look to the world of unicorns for how to implement DevOps.  They should look at integrating operational people onto the development team and work to evolve the culture by blurring the lines and responsibilities between the two groups.  This feels like a fairly straightforward process that many unicorns have proved to be very successful.  For these types of organizations, I tend to agree with the articles recommendations that DevOps would work better without outsourcing.

    The problem is that most large traditional organizations looking to implement DevOps don’t have the benefit of an architecture that enables small nimble teams to work independently.  They typically have very tightly coupled legacy architectures that require 100-1000s of engineers to work closely together to develop, qualify, and deploy complex integrated systems.  Ideally, they would re-architect their applications so they could see the benefits of nimble independent teams.  The reality though is that this is just not practical for most traditional organizations in the short-term.  Additionally, they can’t wait for the re-architecture before they start realizing the advantages that DevOps principles can provide with a well-designed deployment pipeline.

    For these organizations implementing DevOps principles (the ability to release code to the customer on a more frequent basis while maintaining or improving stability and quality) is more about creating a well-designed deployment pipeline that builds up a more stable enterprise systems on a regular basis so it is much easier to release the code on a more frequent basis.  This is done by creating a deployment pipeline that integrates the code across the enterprise system on a much more frequent basis with automated testing to ensure that new functionality is not breaking existing code and the code quality is kept much closer to release quality.

    From my perspective this approach to DevOps is rather different from the more unicorn type approach described in this article.  It though does address the biggest opportunity for improvement that does exist in more large traditional organizations which is coordinating the work across teams.  In these cases the working code in the deployment pipeline is the forcing function used to coordinate work and ensure alignment across the organization.  If the code from different teams won’t work together or it won’t work in production, the organization is forced to fix those issues immediately before too much code is written that will not work together in a production environment.  Addressing these issues early and often in a deployment pipeline is one of the most important things large traditional organizations can and should be doing to improve the effectiveness of their development and deployment processes.

    These benefits are even more pronounced when working in organizations that have outsource development across multiple companies around the world.  Ensuring the different vendors don’t go off track and create code that won’t work together in production is critical and a well-designed deployment pipeline is a critical tool for coordinating development across different outsourced organizations in traditional tightly coupled systems.  For these types of outsourced organization, DevOps principles while different are almost more important than they are for companies that do all their development internally.

    About the Author/ Gary Gruver

    gary gruverGary Gruver is an executive with a proven track record of leading Large-Scale Agile transformations and co-author of “A Practical Approach to Large-Scale Agile Development” and the new book, “Leading the Transformation: Applying Agile and DevOps Principles at Scale.” He is currently working to help organizations with their transformations after successfully leading two large transitions at HP and Macy’s.  As the Director of Engineering at HP, he led a large organization (400+ developers) on a journey from waterfall to agile development. At Macy’s.com, as the VP of QE, Release, and Operations, he led the transition to Continuous Delivery.  Reach him on twitter at: @GRUVERGary

  • Will DevOps Kill IT Outsourcing?: Part 2

    Will DevOps Kill IT Outsourcing?: Part 2

    Part 2: Continuous delivery tool chain efficiency > Outsourcing savings

    (Check out Part 1 of the series Will DevOps Kill IT Outsourcing?)

    Experts on digital disruption and DevOps practices are vocal in their belief that DevOps and traditional outsourcing models don’t mix well.

    “Adopting DevOps requires organizations to erase the barriers between development and operations teams,” says Dan Cornell, CTO of Denim Group. “This is hard enough to do within a single organization and nearly impossible to do across organization boundaries.”

    If push came to shove and an organization needed to justify an either-or decision for DevOps over outsourcing, the argument would likely revolve largely around business value. Continuous delivery makes it possible to move to market more quickly, develop products better aligned to customer sentiment and kick the pants out of competitors who don’t do the same. That was the impetus behind the shift at GE Capital. Last year the firm’s CIO Jim Fowler said he was looking to significantly reduce IT outsourcing and shift toward more Agile teams. Earlier this summer Matt Merchant, former global managing director of DevOps for GE Capital gave devopsy.kinsta.cloud a snapshot into how this is driving innovation. (Ed. Note: Merchant suddenly passed on this summer. Our interview was just prior to this tragic event.) He explained how the company was able shift customer-facing mobile app development from a six- to eight-week release cycle to twice weekly.

    But the decision to dump outsourcing in favor of supporting DevOps practices within a more centralized IT group doesn’t just have topline revenue and innovation benefits. The upside presents itself in the bottom line as well.

    In many ways, we can chalk that up to the fact that the automation and cloud technology that facilitates continuous delivery is light years ahead of where it was when belt-tightening outsourcing deals crescendoed following the 2008 financial crisis. As things stand, the cloud has enabled IT to decouple infrastructure outsourcing from operational and process outsourcing.

    “Companies today realize they can achieve many of the cost efficiencies that outsourcing was originally intended to achieve by leveraging cloud technologies,” says Thomas Enochs, vice president of customer success at Chef . “There is a general trend towards leveraging cloud services rather than cheap labor to achieve the scale, speed, and flexibility they desire.”
    The fact is that many an outsourcing deal was born in a day when the tools of development were slow to deploy and expensive.

    “DevOps practices and modern software tools have drastically reduced the time and cost required to develop custom applications,” says Bernard Sanders, CTO of CloudBolt Software. “In the early 2000s, the step of just getting the servers and infrastructure set up for a new app could be a multi-week or -month chore.”

    Virtualization vendors, cloud providers and self-service automation have all made it possible for developers to get easy and affordable access to the infrastructure they need when they need it, Sanders says. And then there’s the advances in deployment pipeline automation that are seriously streamlining hand-offs over the course of the software development lifecycle. It’s all adding up to added efficiency and reduced cost of development and deployment.

    “In the future, the overhead required to build custom apps will be driven to near zero, and enterprises will be able to in-source this development, retaining this function that has become key to enterprises’ brand identity, product offerings, and customer experience,” Sanders says.

    Even now, though, the DevOps toolchain is completely blowing up a lot of the financial models that initially drove IT departments to aggressively outsource. While many outsourcing deals continue to be renewed without reviewing old assumptions, this is starting to change.

    “We have seen cases where outsourced functions like automating test or other processes has been brought back in house because equipped with the right approach and tooling, the internal team could achieve better results, but also because the automation work was a key building-block to a longer-term DevOps plan,” says Alex Henthorn-Iwane of QualiSystems.

  • Part 2: Meeting relentless mobile app demand

    Part 2: Meeting relentless mobile app demand

    As we discussed in  Meeting the Relentless Mobile App Demand of the Enterprise, the enterprise thirst for mobile apps will far outstretch what enterprise internal development teams will be in a position to produce.

    Developers will need every edge provided to them in order to help their enterprise success. And there’s no doubt that IT teams need to be agile and move as quickly as possible and this is as true with mobile development than anywhere. By not being able to meet the increasing demand for enterprise mobile apps, enterprises can’t be as effective and productive as they need to be. That creates substantial business opportunity costs.

    Here are some ways I think enterprises can improve the mobile app development efforts, in addition to what Gartner suggested, detailed in our part 1:

    Adopt agility. Enterprise IT is changing rapidly. The tight collaboration with development and operations teams, drive for agility, continuous development and other approaches that are increasingly mainstream are pushing applications out into production, and updating existing ones, more quickly than ever was before possible. Not only do successful DevOps organizations have short app backlogs, they are more innovative. Our special guest contributor Matt Selheimer wrote am insightful series that culminated with DevOps with purpose: ACT-ing with greater agility that details how to be more agile through DevOps practices.

    The more agile your organization, the more swiftly they can re-adjust and move always be producing the apps that matter and those that will drive the greatest business impact.

    Leverage APIs. Instead of building apps from scratch and creating more data silos, take a page from Gartner’s advice on mode 1 which is building infrastructure and APIs. Most existing large enterprises have enormous data and application and database silos. One effective way, when possible, to unleash that data and reduce silos is through APIs. APIs are the most straightforward way to build the capabilities the business needs through    extending the utility of existing applications.

    Leverage the cloud to the hilt. Whenever possible, it’s time to go cloud first. It’s time to gate legacy on-premises systems and maintain them until they can be displaced. This will include public, private, and hybrid cloud architectures – but the more rapidly legacy on-premises systems can be retired the more agile IT can be.

    Our post, Three tips for maximizing efficiency in the age of the hybrid cloud environment, provides insight into how to manage hybrid cloud environments.

    Choose cloud and development platforms carefully. There are a number of other critical platform questions, too: Can the data be provided wherever the user needs it? Does it help alleviate security, privacy, and regulatory compliance demands? And can the platform help with role-based identity management and provide for auditable access control?  All of these things being provided as utility help expedite application development and clear your app backlog.

    The more people in your organization who can develop the better. This could come in the form of training non-developers to build some of the app capabilities they need in cloud platforms and with very high level coding tools. It’s easier now than ever before for anyone to create business process workflows with hardly any code.

    Be MVP lean. The Minimum Viable Product (MVP) is a viable way to identify mobile apps that are in need and will likely provide the most immediate value. Simply defined, the MVP consists of what is only necessary to get the task done and little more. If the app is successful, it can be improved upon and iterated based on user feedback and business need.

    One way DevOps helps to achieve this is through splicing development time so many apps can be worked on incrementally, and the MVP concept is ideal for this type of workflow. It makes it easy to re-prioritize apps and new incremental work on existing ones based on pressing business priorities.

    I’m sure there are other ways to help make mobile app development efforts both effective and strategic to the enterprise rather than fire-fighting app demand. If you have any ideas, we’d like to hear them in the comments below. What strategies, tools, techniques are your organization using to increase mobile app development efficiency?

  • Will DevOps Kill IT Outsourcing?: Part 1

    Will DevOps Kill IT Outsourcing?: Part 1

    Part 1: DevOps and outsourcing like oil and water

    After nearly two decades of steady IT outsourcing for the sake of savings and efficiency gains, the shine seems to have worn off from this once glittering phenomenon. And according to many experts, DevOps and continuous delivery practices are playing a big part in many enterprises’ plans to reverse course on outsourcing key functions in their technology stack.

    “DevOps transformations emphasize collaboration, which is often at odds with outsourcing arrangements,” says Jeff Gallimore, founding partner at Excella Consulting. “Outsourcing arrangements often reinforce stovepipes and silos rather than break them down. And silos are speed killers and barriers to improvement.”

    According to a report out earlier this spring by outsourcing research firm Information Services Group (ISG), average outsourcing contract values saw their worst first quarter in a decade, dropping by 27 percent year-over-year. Big deals over $30 million were particularly hit hard, declining by 25 percent in both number and value.

    Even without the tidal wave of Agile and DevOps practices seeping into enterprise IT over the last few years, many organizations are already predisposed to take a second look at their outsourcing contracts, says Gallimore, who sees time and again that many businesses aren’t getting the value they thought they would from their outsourcing deals.

    “I hear many more negative experiences organizations have had with outsourcing arrangements than I do positive ones,” he says. “Poor quality. Higher-than-expected management overhead required. Inflexibility. Process challenges. All are reasons cited for why the arrangements didn’t live up to expectations.”

    The anecdotal stories are also piling up of big name enterprises pulling more of their IT resources back in house after years of outsourcing. For example, General Motors made waves three years ago when its CIO Randy Mott said he would completely invert the outsourcing ratio the car company had by 2017. The firm continues to make headway in changing from 90 percent outsourced IT to 90 percent insourced, with many of the positions advertised these days focused on DevOps and innovation-related roles. Similarly, David Smoley, CIO of pharmaceuticals firm AstraZeneca, went public with his goal to shift the company from 70 percent dependence on outsourcing to 30 percent, along with a commitment to Agile and lean development.

    As organizations increasingly witness the business boosts gained through a more collaborative DevOps delivery model, they are seeking ways to adopt continuous delivery practices across their entire software delivery lifecycle. But they’re finding that these practices are often wholly incompatible with the outsourcing arrangements they’ve developed, says says Jonathan Braunhut, chief scientist at KEMP Technologies. As he explains, many organizations will need to “unwind” these arrangements if they want DevOps to work well because in many cases the contracts and SLAs that govern them are inextricably tied with waterfall development.

    “Outsourcing in an enterprise setting is often implemented in a way that advances outdated waterfall-like practices and processes that are antithetical to continuous delivery,” Braunhut says. “The very wall between dev and ops that the DevOps movement seeks to eliminate can become codified in outsourcing agreements, as a reasonable means of achieving an enforceable and accountable contract.”

    More specifically, outsourcing tends to fly in the face of DevOps’ lean ethos, which is to “minimize handoffs and waste and increase quality by thinking about the overall system and flow,” says Gallimore, who believes that traditional contractual arrangements simply can’t operate with the kind of agility necessary to establish speedy and iterative cycles of continuous delivery.

    Perhaps even more fundamentally flawed in the outsourcing relationship is the idea of ownership. As many organizations finally recognize the value that IT brings to the table—more likely now to serve as a revenue generator than a cost center—it doesn’t make as much sense to give up so much ownership over the technology and tech-enabled services that could potentially transform the business.

    “To be most effective, enterprises need to own the transformation, and it has to be a truly collaborative effort across disciplines–business leadership, engineering, system administration, security & compliance,” says Thomas Enochs, vice president of customer success at Chef. “Collaboration and transformation are difficult to achieve with outside third parties or vendors. They need to own and drive the change themselves, and not be dependent on others.”

  • DevOps Dozen – Nominations are now open

    DevOps Dozen – Nominations are now open

    To succeed in todays speed of business, app-centric world the old ways of doing business just don’t work anymore. IT and development teams are being pulled, pushed, stretched and changed into new combinations and configurations. Greater cross-functional cooperation is a must, a culture of cooperation and a new generation of tools that facilitate these are changes are sweeping in like a Tsunami.

    DevOps has appeared like a superhero fighting a villain to combat these issues. Featuring a one-two punch of continuous delivery and continuous integration with a finishing blow of automation, DevOps enables easier and closer collaboration between development and operation teams.

    Whether they be cloud-native “unicorns”, traditional enterprise “horses” or born on the run startups, DevOps is becoming the “way they do things.”

    The field has matured enough where we think it is time to recognize some of the best and brightest.  Our first venture with this is our “DevOps Dozen”  The DevOps Dozen will feature the top 12 vendors in the DevOps space. I know 12 isn’t a lot given the amount of companies in the mix, but we had to start somewhere.

    So how do we get to a dozen from such a large and worthy group? We at devopsy.kinsta.cloud have compiled a list of what we think are the best known DevOps companies. Some of these companies have their product or service listed as well to help identify the face to the name. We have broken them down into several categories to hopefully make it easer to select (tip of the hat to XebiaLabs and their Periodic Table of DevOps for the help with that).

    The DevOps Dozen selections will be made by you our readers in two phases. First we will open the whole list as well as any write ins for those we may have missed. This nomination voting will be open until August 12th. At that point we will take the top 24 company/products who garnered the most nominations. These will be the finalists and we will have a 2nd round of voting for final Dozen.

    Feel free to tweet about the DevOps Dozen using the hashtags: #DevOpsDozen and #DO12

    Please click here to vote for the finalist nominations

    or fill in below

    Create your own user feedback survey

  • The Calculus of DevOps

    The Calculus of DevOps

    This is the second in a series of blogs, partnering with Lee Reid, who wrote the first installment “The Simple Math of DevOps” and Sanjeev Sharma both of who are outstanding DevOps Thought Leaders from IBM.

    I am picking up on Lee’s mathematical DevOps model to apply a calculus perspective, a “calculus of DevOps” if you will, (OK, not quite core Big Bang Theory Calculus but rather approaching it from the aspects of engineering queuing theory).

    One of the cultural challenges that DevOps poses is, that some aspects of it run counter to engineering principles that many of us learned with respect to capacity and response time.   When I worked at Bell Labs, we would apply queuing theory to determine how to best package information to be sent across the network. A basic premise revolved around a tradeoff between throughput and response time. On one hand, the more messages that were packed into the packet, the more the payload would improve capacity because the overhead of processing the packet of information would be spread across the messages.   However, waiting to accumulate enough call processing information meant that the packets would be transmitted less frequently, increasing the response time.

    A simple example of this would be making trips to the grocery store. If one wanted to spend the least amount of resources (time, gas, car wear, etc.) on shopping, then one would make less frequent trips and then buy a larger amount of food. So if the Smith family made one trip a month and spent 2 hours, then their average time per week spent on shopping would be 30 minutes.   If the Jones family went every week and spent an hour, then they would be spending twice the time and four times the gas on grocery shopping. However if the Smith family run out of an item (say cereal), they would have to wait, on average 2 weeks before this was replenished. The Jones family average on any item that they ran out of would be 3.5 days in comparison.

    So when the 2014 DevOps report was published[1] which states that areas that deploy more frequently get customer feedback more quickly, that is to be expected. But what’s not expected is that these organizations also have higher productivity.   So exactly what is happening here that seems to be in a basic conflict to the principles of throughput and response time?

    Keeping up with the JONESES

    If we think of a pre- DevOps state for a medium to large enterprise, it may be typical for applications to only be released (deployed into production) a few times a year to provide business functionality. (I assume there are more frequent releases for “unplanned work” like defects and other changes or operational work). For the sake of this discussion, let’s call this the Smith Application. So from a business perspective, the business wants to pack as much work into the Smith releases as possible given they only have a few cracks a year at getting new business functionality into production.

    SMITH APPLICATION CHARACTERISTICS:

    • Deliver Business Value 3 times a year (every 4 months)
    • Cost is C1
    • Business Value (BV) delivered is V1
    • Average time from concept (e.g. Story) being ready to be pulled from the backlog to the time it is delivered to the customer is 2 months plus any additional wait times (e.g. prioritization)
    • BV delivered per year is 3*V1
    • Cost per year is 3*C1

    Now let’s transport over to the Jones Planet where the application is released into production with new business capabilities at the end of every 2 week iteration.   In this case the business is more concerned with providing a constant demand flow into IT than they are with debating over whether a given feature/story is going to be included in this particular release.   Instead there is a constant prioritization (or continuous planning) process that ensures the highest value work is being fed into the IT pipeline.

    JONES APPLICATION CHARACTERISTICS:

    • Deliver Business Value 26 times a year (every 2 weeks)
    • Cost is C2
    • BV delivered is V2
    • Average time from concept (e.g. Story) being ready to be pulled from the backlog to the time it is delivered to the customer is 1 week plus any additional wait times
    • BV delivered per year is 26*V2
    • Cost per year is 26*C2

    Based on the previous engineering analysis, we would expect the following:

    1. SMITH annual cost (3C1) < JONES Annual Cost (26C2)
    2. SMITH annual BV (3V1) > JONES annual BV (26V2)

    So the tradeoff (we would believe) JONES is making by delivering more frequently is that they are spending more money and are delivering less business value on an annual basis.

    However the data from the previously referenced DevOps report is just the opposite. In fact the findings are that enterprises that deploy smaller units of work more frequently have much higher quality and much higher productivity (Capacity). So what’s going on here?

    A NEW PARADIGM

    If you go up to any IT team (even an agile one) and ask them why they don’t deploy business value more frequently, they will generally refer to the following types of wait states of waste that they encounter:

    • Waiting for work (lack of consistent flow)
    • Waiting for somebody else to do something (dependencies from other teams)
    • Waiting for environments (contention)

    One major root cause of this is the dependencies that arise from large enterprises packing a lot of work into infrequent releases. From a planning perspective, the process is elongated due to prioritization discussions because being on the wrong side of the line can mean delay amounting to months, in delivering a product to a customer.   Ask a business portfolio leader what their highest priority is for IT and I am guessing they can tell you in a matter of seconds. However, most large organizations can spend several months a year planning their annual portfolio of work for IT. Some organizations spend more than 50% of their new project build time prior to a single story entering an agile development team’s backlog!

    From a development perspective, packing a lot of work into releases means many more dependencies that have to be managed across initiatives and impacted applications. From a testing perspective, downstream environments (ST, UAT) have to manage much more information in terms of application versions and especially test data that complicate and slow down the process. The complexity of test data increase is proportional to the number of pairwise relationships that exist within the datasets required for testing.   As such, the testing complexity will increase non-linearly (perhaps proportional to the square of the dependent systems) and from a release and change perspective; there is a lot of coordination necessary given how much is being changed at one time.

    All of this work to manage all of the dependencies found in medium to large enterprises completely eliminates any potential savings one might get from batching work into a single delivery. In addition, the possibility of having to pull some work out of a release (due to quality or business readiness or some other reason) sometimes leads to complicated source code management (feature branches in some cases) that further complicate the process of merging frequently into the trunk. These types of practices also serve to drive up technical debt. So looking at the problem from this angle shows why results we are seeing make perfect sense.

    Productivity is inversely proportional to the number of dependencies in a release

    As Lee related to in the initial blog, a lot of an organization’s success depends on trust. And one factor of trust is clarity which speaks to understanding the changes that each release is bringing to the IT and customer environments. So a factor that is working against trust is the notion of delivering more things less frequently and better controlling change somehow, is a good notion.

    Trust is inversely proportional to the number of dependencies in a release

    There is another hidden component of waste associated with “big bang” initiatives. Go back to our grocery example for a moment. Chances are that the Smith monthly excursion results in buying some stuff they end up not using. Perishable items are not spread out very well when they aren’t purchased more frequently. Likewise in IT planning, we tend to put a bunch of stuff in our IT Delivery basket that we don’t know for sure that we will need and we end up with a lot of wasted effort for features that are hardly used or not needed at all. Some enterprise estimates are that 50-65% of what they put into large releases does not actually result in true value to the business. Doing more frequent releases allows a constant discovery process to better guide the usage of IT resources on what is truly of value.

    Waste is proportional to the size of a release.  

    In other words:

    Yield (Value per IT investment) is proportional to the frequency of releases.

    REVISTING THE MATH OF DELIVERY TIME

    So if we revisit Lee’s model

    Lee Reid Blog equationand apply the “Calculus of Release” as we stated above, there are increased costs associated with having larger and less frequent releases in medium/large enterprises.

    • Planning cost TPLAN
    • Development complexity TDESIGN + TDEVELOP + TBUILD
    • Testing Complexity TTEST
    • Release and Change Management Complexity TRELEASE + TDEPLOY + TEVALUATE

    TRUST % – Even if everything was proportional between the JONES and SMITH models, the component of trust that impacts clarity, consistency and collaboration, is much lower for large (SMITH) releases.  Given that TRUST has a multiplicative impact, this alone can account for the results found in the studies.

    CULTURAL CHALLENGE

    While clearly one needs to be sensitive to the changes being introduced into the business, having the ability for the business stakeholders to pull capability when they want it and having it available more frequently is not only good for customers but also has extreme benefits from an IT perspective in terms of increased productivity and reduction of technical debt. The challenge is to convince leaders in both the business and IT areas on the merits of doing more continuous planning and frequent deliveries.   Unicorns may have led the way in this type of thinking but it is actually the horses that will benefit most from this type of culture shift because it is the horses[2] that have the most to gain in mitigating the dependency factor which can undermine quality, productivity and cycle times.

    Read more about the topic in the next blog of this series The Art of DevOps by Sanjeev Sharma.

    [1] “2014 State of DevOps Reports”, Puppet Labs, IT Revolution, https://puppetlabs.com/sites/default/files/2014-state-of-devops-report.pdf

    [2] “The Phoenix Project”, Gene Kim, Kevin Behr, George Spafford, http://www.amazon.com/The-Phoenix-Project-Helping-Business/dp/0988262592

     

    About the Author

    CarmenCarmen DeArdo is the Director of Application Development Tools and Technologies at Nationwide Insurance. Carmen is responsible for driving continuous delivery utilizing DevOps, Lean and Agile techniques across Mobile, Distributed and Mainframe and other technologies. This includes recommendations and implementation of technologies used across the development life cycle (e.g. IBM Rational tool suite, open source technologies).

  • Inside Look at State of DevOps Report: Q&A With Puppet Labs

    Inside Look at State of DevOps Report: Q&A With Puppet Labs

    Last week Puppet Labs released the analysis behind its annual study of DevOps and continuous delivery practices in the 2015 State of DevOps Report. Now in its fourth year, the report offers an in-depth look at the findings of the team on what it means to be a high-performance IT organization—the cultural characteristics these organizations share, the lean engineering practices they engage in and the technical architectures they depend upon.  Like last year,  however, the report tends to be a bit of a black box when it comes to actual statistical findings. While some figures do make it through, for example the oft-flogged statistic of high-performance organizations deploying 30 times as fast with 200 times the lead time of low performers stands pat, the report itself is somewhat oblique about the statistical basis upon which the designators of low-, medium-, and high-performance were determined.

    devopsy.kinsta.cloud spoke to Nigel Kersten, CIO for Puppet, and Alanna Brown, senior product marketing manager for Puppet,  about the report to get a little more detail, to have them explain the process behind the analysis, and to offer up some of the most important facets of their findings over the last few years.

    devopsy.kinsta.cloud: Can you tell our readers a little bit about the high points from the report?

    Kersten: Big finding from 2013 was the orgs that used Devops’ practices deployed code 30 times more often and had 50 percent fewer failures.  In 2014 what was quite fascinating was that we actually found a link between these practices and overall organizational performance, not just IT performance. Higher-performing IT organizations are twice as likely to exceed their own profitability, market share and productivity goals.

    If we fast-forward to today, the high-performing organizations using Devops practices are more stable and reliable than they’ve ever been.  They’re recovering 168 times faster from their failures and have 60 times fewer failures due to their changes.  So we’re not actually seeing throughput increase from the year before, but what we are seeing is the quality and speed has definitely increased.  So people are producing changes that are of higher quality and that require fewer rollbacks.

    And so what we’re concluding from the data that we’ve seen this year in the report is that there’s been a really huge focus to the left of the assembly line from production. So if we envisage the developer work flow application delivery, essentially we start with the developer laptop like on the far left, and on the far right is the application actually existing in production.  So from the point between the developer’s laptop and the actual release, we’re seeing organizations focus much more on that stage, putting more testing in the hands of the developers, and all of these practices not just being focused around the release but the pre-production stage of the application life cycle having a really huge impact upon quality.

    devopsy.kinsta.cloud: Why doesn’t Puppet publish the actual data from the survey? You have great analysis here, but there’s very little statistical proof behind it—why don’t you provide that?

    Brown: Well so we have a ton of data, and part of the reason we don’t issue the data is because you wouldn’t be able to actually get to the same results without doing a ton of data cleansing and statistical analysis.  So our analysis is actually fairly rigorous, and some of that analysis doesn’t really lend itself very well to graphs and charts and visuals and things like that because it’s all based on correlations and predictive regressions.  So that’s part of the reason.

    Kersten: I think in many ways it’s analogous to releasing software to the world. We have all of these tools inhouse.  We have all of these Excel spreadsheets we’ve put together.  We have all of these you know pivot tables and different bits of cluster analysis, but they only – they’re not particularly usable for anyone else right now.

    As much as we would love to work out how to polish all of that up and share it so it’s one big data set along with all of the tools, our biggest worry is honestly being that people would actually draw sort of misleading and incorrect conclusions from the data because of the level of analysis that was required.

    devopsy.kinsta.cloud: Can you tell us, then how you came to the determination of what’s a high-, medium- or low-performance organization? Because if you look at some of these numbers, you say they’ve got thirty times more deployment frequency. But of course they do because they’re high performers. So how does that work out statistically?

    Brown: No, no, you’re absolutely right, and I will agree that IT performance metric is a bit circular.  However, those are the actual metrics that are important to an IT organization, so they care about deploy frequency, deployment lead time, their mean time to recover and change failure rate.  So we feel like that’s a pretty good approximation or a proxy for IT performance, and it’s better you add at doing those things that is correlated to higher performance overall.

    Kersten: One thing, too, is that even though there’s a whole spectrum in terms of how fast people are able to actually deliver and ship code, what actually came out really strongly out of the survey this year and in previous years is that the high performance (orgs) definitely cluster in terms of data points. It’s not just a smooth spectrum, but there’s this really obvious clusters of high performance that leap out that all have a very high correlation of the kinds of practices they use.  So it is a little bit circular but it’s also it’s not that we’re seeing this smooth spectrum from zero to 100.  We’re seeing the masses and then the high performers, and they divide into quite clear clusters.

    Brown: Yeah, and they’re not arbitrary clusters; they’re actually statistical.  So the low IT performers share all the same characteristics and the high IT performers share all the same characteristics, so they’re very clear.

    devopsy.kinsta.cloud: OK, so let’s talk about some of those common characteristics between these groups if you’ve got statistical bases for these.  Tell me a little bit about the low performers and we’ll kind of go on up the spectrum and we can talk about the characteristics they share.

    Brown: So for IT performance, deployment pain was actually the number one most highest-correlated construct. And deployment pain consists of a couple of questions.

    Like how painful are your deployments?  Are they very disruptive events, etc?  So there’s three questions that we ask around deployment pain.

    The next thing that’s highly correlated with IT performance is version control. We use the version control for all production artifacts.  Following that is culture, and the model we use for culture comes from a guy named Ron Westrom who is a sociologist.  And he created a typology of cultures, and you can see the table in the report itself.  So he has a pathological culture, bureaucratic, and then generative.  So the more generative your culture is, the more likely you are to have higher IT performance.

    Kersten: And I think this is what was so fascinating for us when we first started adopting Westrum last year We actually adopted Westrum as part of the model in that this pathological versus generative cultures, this is the language our practitioners in this space use to talk about you know what’s so great about working there: “My ideas are valued. I get to create ideas.  I can create impact.” Similarly around what’s horrible about working at this other place.  It maps to the pathological and generative descriptions of organizations perfectly really.

    devopsy.kinsta.cloud: So what would you say is the key characteristic in that first leap from low performance to medium performance, based on your data?

    Brown: So the lower performers in terms of IT performance will be doing less automation, less testing.  They haven’t automated their deployments, for example.

    Kersten: I think one way I at least think about is that there are these characteristics that are necessary but none of them are sufficient on their own, and so you see people who say just adopted version control.  They’re definitely heading ahead of the pack of the people who haven’t adopted any of these tools.

    But there’s this transformative leap once you actually adopt version control and continuous delivery and the highly-automated conflict management system, that those things together are greater than the sum of their parts.  That is at least how the data seems to us, ’cause we don’t actually have this smooth little spectrum between low performers and high performers who just adopted one of those technologies.

    We definitely see improvement from those people, but it’s when they adopt all of them in total that they suddenly take this sort of order-of-magnitude leap.