Category: Features

  • What CIOs need to know about building deployment quality

    What CIOs need to know about building deployment quality

    Application Release Automation, or ARA, is the process of packaging and deploying applications (or updates to applications) in an automated, repeatable and consistent way.

    In today’s market landscape software has become a major differentiator for many businesses, across all industries and verticals. In a world of apps, Internet of Things and customers’ expectations to be always connected, – just about every company is now becoming a software company. Software development is now critical to the success of your product, and its delivery process needs to be fine-tuned and streamlined to ensure that you get quality software in the hands of your end users as quickly as possible.

    To understand how ARA can help you achieve higher quality software releases (which contribute to your bottom line), you must first understand the challenges you might face if your business is still using manual deployments, or other archaic methods. If that’s your case, though, you’re not alone: a survey we ran last year found that over 75 percent of deployments are still done manually, despite the many problems with this model, which is extremely brittle and error prone. This creates not only many failed deployments, but loss of hundreds of men hours trying to troubleshoot those failures. This is certainly painful in the mission critical production environment, but carries great cost long before we reach production. Even in the lower environments (such as testing or pre-production), 66 percent of respondents report its very time consuming just to troubleshoot why the failure occurred. As we promote code to be deployed in production- the stakes are higher. Costs of failed deployments, delayed releases and loss of men hours are exponentially aggravated when you add possible fines on top of that, cost of SLAs violations, customer churn, and loss of business. There’s no question the implications of failed deployments put a major dent in your bottom line, and with some companies – failed deployments have literally put them out of business.

    A prime example of how a failed release can have huge impacts on a business is the case of Knight Capital. Knight Capital WAS a financial firm. In August of 2012, they manually pushed out a new release to their software when the markets were down over the weekend. Monday morning came and when the market opened, their new software kicked in. The results were disastrous. They accumulated (extremely quickly and erroneously) a $9 billion [H1] position in the market – represented by the movement of 397 million stocks in just 45 minutes. Two hours later, when Knight was still chasing the culprit in their release process and trying to rollback to a previous version of the software, the New York Stock Exchange said, “We’re done waiting for you,” cut their hardline to the market and Knight couldn’t make any trades.

    The effects were disastrous and swift. Knight Capital endured a pre-tax loss of $440 million. Within weeks, Knight Capital was sold for pennies on the dollar – a failed release literally killed the company. While this is certainly a worst-case scenario, it is a great example of how it is necessary to understand that once you start talking about production environments, there is a lot of regulatory and best practices that have to be followed or big problems can occur.

    Why are manual deployments so problematic?

    First of all, manual deployments are error prone because each deployment has to be done by an actual human. Even the most veteran of developers or ops engineers make mistakes, and when you start talking in the hundreds (or thousands) of deployments, across complex stacks and configurations, an error is almost inevitable. While no system can fully-eliminate the potential for an issue, you can imagine that in a deployment process where many of the steps are automated – and therefore consistent across releases – your chances of an error are reduced dramatically.

    With manual deployments, it’s also difficult to achieve true, fine-grained, visibility and auditability into the entire deployment process of all the application’s components and their corresponding infrastructure or configuration. Once a deployment is pushed live, the only person who really knows how, exactly, the update was promoted from Testing to Production is the individual (or individuals) who performed the release. And as good as they might be in keeping notes, they’ll never be as comprehensive, or bulletproof, as an automated log that track every single component in the multi-stages deployment process.

    Manual deployments also are not adept at negotiating what is called the “Delivery Gap.” Essentially the delivery gap is the chasm that exists between three different factions of the software team. On one side there is the development team, which has a bunch of tools and practices they like to use. On the other side is Operations, which has their own set of tools and methods/requirements. Then, finally, there is the infrastructure, which spans across both Development and Operations and places requirements on both parties. So you have essentially three different players, using different tools, with difference practices and requirements, and negotiating between them is a tricky and difficult process. Ensuring there’s no configuration drift between infrastructure or app components across the different team, and ensuring consistency in deployment processes has proven difficult for humans to perform, particularly as each process includes hundreds – if not more – of artifacts, configuration files, dependencies and deployment requirements. For an automated process, however, consistency and repeatability are the default state.

    So now that we’ve established the pitfalls of manual deployment practices, let’s talk about how Application Release Automation can solve some of these issues and support higher quality software releases.

    We’ll start with the aforementioned “delivery gap.” Application Release Automation is a DevOps solution that bridges the gap between the Development teams and Operations teams. It does so by creating a solution that encompasses Continuous Integration, Configuration Management and Deployment practices that respect infrastructure, as well as operation’s needs and regulatory requirements. It achieves this because you can build these considerations into the automated process, which is recorded, logged and tracked by default. Rather than each deployment being a unique ‘snoflake’, software deployments become routine and predictable. It’s not on one person to mentally “flip the switch” and jump back and forth between groups to make sure everyone are satisfied. Rather, it is an automated process that takes everyone’s interests and requirements into account, and then makes sure everything fits within the infrastructure and configuration requirements.

    Furthermore, ARA is a “model-driven” approach that allows you to reuse existing templates, configurations and processes over and over to make your deployments consistent, and require minimal overhead for the initial set up of the process. This means you can deploy any version to any environment quickly, with push-button deployments. This not only increases the speed of deploying across different environments throughout your process, up until the final release to Production, but also allows for repeatability in a way manual deployments simply can’t. This is especially important when it comes to testing. Imagine if the framework of every update you deployed was identical – how much easier would that make testing? And how much easier would your tests be if they were all ran continuously and automatically? That is what ARA can offer.

    Additionally, with consistency, your deployments are very predictable. You know what the run times will be, allowing you to plan your development processes more accurately and efficiently. This means you can allocate time and resources in a way that maximizes your output – which allows your team members to spend more time on their core competencies and improving your product, versus having to manage the deployment process or the infrastructure.

    Along the same line, one of the key tenets of ARA is packaging. When you manually deploy updates, bits and pieces that comprise your application are being deployed (“moving”) at different times, creating a “noisy” process, with lots of dependencies, that’s difficult to manage. With ARA, everything is wrapped up in a tidy “package.” This effectively creates a “bill of materials” for your release and all dependencies and related configurations, which allows you to easily indicate what components of the application need to be deployed where, and in what order.

    Application Release Automation also solves the issue of having such a critical process to the company be dependent on what is often just one person – a “star student” in your team who “holds the keys” to the project and only he has the tribal knowledge and expertise to perform or debug a deployment. With ARA, there is an implicit transparency to the project that gives any team member the ability to access the system and deploy the software (pending the right security permissions, of course).

    Lastly, with ARA, you have complete visibility of the process from start to finish. If something goes awry, you have a much better view of where the error originated from, making recovering much faster and easier, and the damage much less significant.

    Ultimately, a successful ARA strategy eliminates the need to build and maintain custom scripts for application deployments, while simultaneously reducing configuration errors and downtime. It allows strategic allocation of resources, giving users the most “bang for their buck” from their developers and other IT professionals. Because of all these factors, Application Release Automation is a hugely effective strategy for increasing both the quality and speed of your deployments.

    About the Author/Anders Wallgren

    leadership-anders-wallgrenAnders Wallgren is Chief Technology Officer of Electric Cloud. Anders brings with him more than 15 years of in-depth experience designing and building commercial software. Prior to joining Electric Cloud, Anders held executive positions at Aceva, Archistra, and Impresse. Anders also held management positions at Macromedia (MACR), Common Ground Software and Verity (VRTY), where he played critical technical leadership roles in delivering award winning technologies such as Macromedia’s Director 7 and various Shockwave products.  Anders holds a B.SC from MIT.  Connect with Anders on LinkedIn or Twitter.

  • Bonitasoft makes business process management more agile with open source

    Bonitasoft makes business process management more agile with open source

    Organizations embrace DevOps for a variety of reasons, but one of the core value propositions is speed. DevOps enables business to build, test, deploy and update applications more quickly than without DevOps. With so many quickly-moving parts, though, organizations also need tools to help keep track of it all like a business process management platform.

    Bonitasoft developed its business process management platform—Bonita BPM 7—to facilitate the creation of engaging, process-based applications. It was also important, however, that the applications be able to adapt in real-time to changing business requirements. The latest release is an open source version that represents a strategic shift for Bonitasoft—a somewhat inevitable evolution to leverage the same practices and principles in designing and maintaining the platform as the platform is intended to manage.

    “We’ve embarked on a new mission to create solutions that empower developers to build applications that support continuous change,” Bonitasoft CEO and co-founder Miguel Valdes Faura said in a press release statement. “To win in today’s competitive climate, you need to be able to adapt to change quickly and continually improve your business processes and the applications that support them. Bonita BPM 7 is the first solution leveraging the power of open source and BPM process-based applications that meets the demands of today’s business requirements.”

    Bonita BPM 7 is built on a popular and widely used open source BPM solution. Developers can create processes in Bonita BPM 7 using the BPMN 2.0 standard, and can seamlessly connect with existing third-party systems such as CRM and ERP systems. Bonita BPM 7 builds on the open source foundation, though, with more extensible frameworks, and a powerful connector builder tool to enable developers to freely code their own graphical components and REST APIs.

    The “secret sauce” of BPM 7 is the way Bonitasoft segregates business logic from the data and user interfaces. The separation makes it easier to introduce changes in real-time. Developers can perform maintenance or push updates to the user interface independently of the business workflow and without taking the application offline.

    According to Bonitasoft, the key benefits of Bonita BPM 7 include the following:

    • Speed – Developers can build applications faster, improving developer productivity
    • Freedom – Developers can easily build what business really wants, and companies can maintain their applications easier and more affordably with live updates
    • Adoption – Development teams can empower their users with more accessible, engaging and usable applications, boosting competitive advantage

    “Bonita BPM 7 represents a real convergence between the world of the ‘BPM app’ and traditional enterprise software,” Valdes Faura said. “It gives developers the freedom to easily code and extend the system whenever they hit the limit with its native features. Plus, the platform is built for change, so developers and business users can apply changes, updates and improvements over time – creating what we call ‘living applications’ that live and breathe and grow along with your business.”

    As developers develop faster and DevOps enables organizations to push and update applications more rapidly the need for tools and platforms to monitor and manage things grows. By leveraging open source software Bonita BPM 7 follows a successful strategy already employed by companies like Github and CloudBees. If you’re looking for a business process management platform you should check out Bonita BPM 7.

  • Say hello to Container Journal

    Say hello to Container Journal

    Today we launched a sister site to devopsy.kinsta.cloud – Container Journal. The url for the site is http://www.containerjournal.com. The site is dedicated to spotlighting everything about containers and computing (as opposed to containers like in the Container Store). This includes not only the containers themselves, but everything that goes along with it.  Topics wil include container management, data management for containers, container security, networking for containers – in short the entire container ecosystem.

    Right now most of the content on Container Journal was container related content that was originally on devopsy.kinsta.cloud. However, over time Container Journal will have more and more of its own original content that will not appear here devopsy.kinsta.cloud. So you will need to check Container Journal to stay up on the latest container related content.  A good way of doing that is to sign up for the Container Journal newsletter.

    In addition to articles Container Journal will feature webinars on container related topics, as well as other special features including ebooks and surveys. Many of the outstanding authors who contribute to devopsy.kinsta.cloud will also be writing for Container Journal as well.

    We are looking for good container related content though. If you would like to contribute to Container Journal, write me at editor@devops.com with your request.  We are also seeking sponsors, partners and advertisers for Container Journal as well.  In a relatively short time, we think Container Journal will be largest media site with a dedicated collection of container related content.  This is your chance to be part of it. Sponsor info can be had at sponsor@devops.com.

    Also be sure to check out the Daily Poll on Container Journal. We will be asking some great questions with rapid feedback on what others are saying.

    Like any new venture, I am sure there will be growing pains with Container Journal. Please keep me posted if you notice anything.

  • Top 5 Anti-ESB Arguments for DevOps Teams.

    Top 5 Anti-ESB Arguments for DevOps Teams.

    Unix and Mac. Developers are always in combat. My last verbal combat was a “discussions” around ESB and Microservices. They are fun because both supporters tend to have very strong opinions about what they think is best and both want to fight each other. I recently got involved in one of those discussion about ESB vs. Microservices and the ESB guy throw lots of arguments at me. “ESB is magic” got almost burnt into my mind until I started to investigate a bit: “How to fight ESB discussions in combat?”. Here are my top 5 findings that may help in your next verbal ESB combat situation.

    (more…)

  • “Show Me The Money!” – Delivering DevOps Value

    “Show Me The Money!” – Delivering DevOps Value

    HM Heath Solutions delivers new business applications in the healthcare industry to 35 million customers.  In order to embark on their DevOps journey to improve software delivery, Valerie Scott, their Manager of Shared Services Division, had to show senior management an ROI on a successful project, before they would invest fully. On June 2, Val Scott along with Rick Slade of IBM joined Alan Shimel of devopsy.kinsta.cloud in a webcast to discuss this interesting client case study.  During their discussion, they illustrated in vivid detail how HM Health Solutions showed success in DevOps to their senior executives, for them to buy in and sponsor the project.

    Val and Rick’s webinar parallels many similar discussions I’ve had with senior executives.  To net it out, the bottom line is, as it was so clearly and unequivocally articulated in this amusing clip from the movie Jerry Maguire: “Show Me The Money!”

    cuba gooding jr show me the money jerry maguire tom cruise show me the money jerry maguire
    When I meet with senior LOB, Finance or IT executives — they shout a similar refrain:

    “All I want to do is buy a business outcome.  I want to invest in an IT project, and have it pay back.  Unfortunately for many application development (or other types of enterprise IT) projects, that pay back is illusory, and ROI difficult to measure.”

    — Jane or John Executive, representing nearly every IT executive or CFO I have ever met

    Unfortunately, many struggle to quantify, monetize and demonstrate the financial return on investing in IT projects.  Yet as described in this blog, it is the primary responsibility and outcome of management; leading the economic performance of their department or unit.  Show me the money!  Fortunately, many clients experience positive returns, both operationally and economically.  Ten (with links to more) client case studies with returns are highlighted in this blog, and summarized on this page.

    As you undoubtedly have experienced yourself, senior executives respond to clear business value and benefits articulated as financial returns, as ROI.  But how do you do that for DevOps, a set of technical practices and associated culture change?  As Val and Rick described during their informative webcast, comparing productivity, time-to-market, speed of delivery, and quality, both before and after a DevOps pilot, are crucial.  These operational metrics will be improved by adopting DevOps.  However, to win the hearts and minds of senior executives, you have to monetize the results, in other words – show them the money!  Technical techniques and practices such as those Val and Rick described to Alan, shifting left, test driven development, and automating application build, deployment, and test are absolutely necessary, but they are insufficient to win the argument with senior executives.  The gains they produce are necessary, and impressive.  The operational results have been stellar thus far at HM Health Solutions, per Val:

    • a 67% reduction in resource demands on the release team & process,
    • a 75% reduction in relative hours to repair defects per environment,
    • an 82% reduction in application deployment time.

    Yet as impressive as these operational results are, they are insufficient.  As Val described, arguments about business value, technical value, or improvements to developer or operations staff productivity, alone did not persuade the management.  The magic word is an economic and financial one: “investment”.  Making a strategic investment to improve delivery speed and quality of business applications was the key to winning the hearts and minds of the senior executives.

    In addition to the technical practices described above, Alan asked Val to describe the tools HM Health Solutions adopted.  A continuous delivery solution with application release automation software from IBM UrbanCode was implemented, including application build, deploy, and release.

    As a way of summarizing the webcast for the audience, Alan asked Rick and Val for their two key takeaways; the two most important messages and things to remember.  Rick’s:

    1. Knowing which metrics are critical to your organization.  There are many benefits to adopting DevOps, and many potential hard numbers that are straightforward to capture and measure.  Deciding which metrics have the most impact and are most persuasive to your stakeholders is critical to making the case to invest in DevOps.
    2. Implementing tools and automation: automated testing, and linking that to build and deployment processes and automation.

    Val’s:  Rick’s two takeaways of course!   And one more…

    1. DevOps is new in the industry, and is not yet well understood or common practice in enterprise IT or among its leaders.  Understand your audience, and become good at explaining it in straightforward and clear business and financial terms to senior executives.  Especially persuasive are other client proof points, and the results of pilots in your own enterprise IT environment.  Make sure pilot results show that the solution works and delivers timely business value, and that the related facts and metrics clearly quantify the results.

    Read this blog to know more and Check out other customer proof points/ case studies

    This was a very informative, fast paced, and information rich webcast.  I hope that you find the replay useful in continuing your DevOps adoption journey.

    If you’d like to explore more about management responsibilities and know-how for establishing an ROI for DevOps and app release & deployment automation, see this ROI web page, and read this ROI blog for additional resources on method, approach, making the case, and calculating the ROI.  And please don’t hesitate to reach out to me on twitter: @paspung.

    I look forward to hearing your success stories in making the ROI case to senior executives, and delivering DevOps value to your organization!

    peter signature, small

    Peter Spung, DevOps Deploy and UrbanCode Integration Executuve, IBM Cloud, @paspung

  • Apply DevOps automation to APIs with Akana

    Apply DevOps automation to APIs with Akana

    One of the core values of DevOps is automation. The ability to automate routine tasks and have triggers that automatically initiate other tasks takes care of all of the routine functions so developers and IT personnel can focus on bigger issues. One thing that has also emerged as driving force for DevOps—and IT in general—is APIs. Organizations have different platforms and tools from different vendors and they need everything to integrate smoothly together. The APIs enable that synchronicity but add another layer of complexity to develop and manage the APIs as well. That’s where Akana comes in.

    Akana enables organizatiions to quickly deliver scalable applications, connect and integrate applications, and share data as APIs. The Akana API Management Platform was created to give developers and IT personnel the tools necessary to effectively develop and manage APIs. Akana recently introduced new capabilities in its API Management Platform to streamline API management and reduce the time required to update APIs.

    One problem organizations face is that DevOps disrupts the traditional workflow and creates new bottlenecks. DevOps is sort of a natural evolution of Agile development—taking the principles of Agile beyond developers to the IT infrastructure and the company as a whole so that everyone can keep up with the faster pace of development. The rapid deployment of apps, however, can have a cascade effect that impacts the API lifecycle.

    As new apps are deployed the API definitions can change. Those changes have to be propagated to the API portal. The API owner has to update associated documentation. In most cases this process is a tedious manual exercise. The new Akana API Management Platform seeks to change that by bringing DevOps-style automation to the API lifecycle management process.

    The new API Management Platform integrates with continuous integration platforms like Jenkins and TeamCity. It has provisions for tracking necessary approvals to push updates to the API portal or API gateway. It also includes support for standard API descriptor document types like Swagger, RAML and WADL.

    “APIs have fast become the de-facto standard for digital enterprises to connect applications and services with digital end-points like mobile apps and internet-of-things,” said Brent Carlson, Senior Vice President, Technology, at Akana. “With this new capability, enterprises can deliver continuous innovation with speed and agility, ensuring that new updates and capabilities are automatically, efficiently and securely delivered to their developers and partners, in a timely fashion and without manual intervention.”

    No matter how streamlined you get you’ll eventually hit a bottleneck. As more of the development and deployment process gets automated that bottleneck gets moved farther down the chain, but then new platforms and tools emerge to address evolving issues and fix the new bottlenecks. For some organizations that have already embraced DevOps APIs may be the new bottleneck, but a tool like the Akana API Management Platform may alleviate that problem.

    It will be interesting to see where the next bottleneck strikes. At some point as DevOps evolves and every element is streamlined and automated the only bottlenecks will be organic—meaning the human beings involved in the process.

  • There is only one DevOps Connect

    There is only one DevOps Connect

    They say that imitation is the sincerest form of flattery. If that is true I guess the folks over Informa Telecoms and Media Ltd are very, very sincere. Why else would they change the name of their DevOps Conference from “DevOps Summit” (which name they “borrowed” from the Sys-Con folks) to DevOps Connect Europe.  No matter that we here at devopsy.kinsta.cloud have already held several events in Europe and London specifically with our DevOps Connect name. No matter that we have a website and own the domain http://www.devopsconnect.com.  Why shouldn’t they just use our name and employ the “colorful” technique of registering a hyphenated play on the domain name? I guess that alone should tell you all you need to know about this company and the events they produce.

    Now of course we can go hire some International copyright attorneys and litigate first usage and ask for injunctions and damages. But why bother? I am a big believer in letting the market be the judge, jury and executioner.  I am perfectly fine with using devopsy.kinsta.cloud to get out the word about these kinds of business practices and letting the market act.

    The event these people want to put on will probably be no better than the several DevOps Summit events they put on previously. I have heard from exhibitors, speakers and attendees of those events that they were less than successful. Mind you I haven’t attended any, just reporting on what I was told. Ultimately though if they were successful they wouldn’t be trying to rebrand using someone else’s name.  Hard to argue with that logic.

    I am asking our readers (and there are many of you in London,  as it is our single biggest city for viewers) to let this company know that copying others business names will not stand. The best way to do this is to not attend, don’t participate and get the word out that this is not the way business is done.

    Don’t be fooled by the attempt to piggyback our brand and success, we have nothing to do with that event. In fact we will be holding several DevOps Connect events in London and elsewhere in Europe in the fall.  Details will be announced shortly but there will be lots of opportunities to participate.

    Some of you may ask, “you haven’t given us the link to this event?” You’re right, I have not and for good reason. I have already given this company and their event more ink than they are probably due.

    So be aware of imitators, remember there is only one DevOps Connect and those are the events put on by us at devopsy.kinsta.cloud. We hope to see you at them and will be sure to let everyone know when we do. Until then don’t reward imitators.

  • Bimodal IT and Remodeling Traditional IT for Greater Agility

    Bimodal IT and Remodeling Traditional IT for Greater Agility

    Bimodal IT is a Gartner concept created to help CIO’s understand that IT organizations need to support both traditional and agile modes of IT solution delivery and operation in order to make wise choices about infrastructure, process, people, and tools.  Much attention has been given and rightfully so to the disruptive and innovative “Mode 2” teams building on the practices established by web-scale giants like Amazon, and Google.  However, in enterprises there still exists a sprawling majority of “Mode 1”, traditional, industrialized applications run by teams that must focus on risk aversion, security and compliance.  While these will likely not match the innovative velocity of agile IT teams and applications, traditional IT teams are still under pressure to get faster, more agile, innovative, productive and cost-efficient.  Yet, the Mode 2 playbook can’t be applied wholesale to Mode 1 transformation work because it often assumes a greenfield situation—kind of like a new home construction.  By contrast, Mode 1 modernization is more like renovating an existing home that must proceed in steps that address the present realities of applications, legacy infrastructure, business mandates, organizational interactions, and technical skills.

    Understanding Bimodal IT

    The Bimodal IT concepts is meant to distinguish between applications and teams that are traditional in their focus, and those that are agile.  In her blog post on Bimodal IT, Gartner analyst Lydia Leong explains Bimodal IT thusly:

    “Traditional IT is focused on “doing IT right”, with a strong emphasis on efficiency and safety, approval-based governance and price-for-performance. Agile IT is focused on “doing IT fast”, supporting prototyping and iterative development, rapid delivery, continuous and process-based governance, and value to the business (being business-centric and close to the customer).”

    The reason that Gartner promotes the idea of Bimodal IT to CIO’s is to help them see that it can be dangerous trying to optimize processes, human resources, infrastructure, and tools to cover such different modes, and as a result, optimizing to neither.  Rather, the right move according Gartner is to allow agile teams room to develop a brand new set of capabilities that can then help transform the whole organization. Lydia continues in her blog:

    “We’ve found that organizations are most successful when they have two modes of IT — with different people, processes, and tools supporting each. You can make traditional IT more agile — but you cannot simply add a little agility to it to get full-on agile IT. Rather, that requires fundamental transformation. At some point in time, the agile IT mode becomes strategic and begins to modernize and transform the rest of IT, but it’s actually good to allow the agile-mode team to discover transformative new approaches without being burdened by the existing legacy.”

    The Contrast Between Agile and Traditional IT

    Traditional IT and Agile IT are different in a number of ways.  Perhaps an effective way to explain the difference is to picture what these applications and teams look like.

    The classic agile IT initiative in the enterprise is around a mobile, e-commerce or web-centric application that is aimed at moving into new markets, improving engagement with new customer segments, or disrupting a current market.  The application is fully virtualized, and the preferred infrastructure is a public cloud such as AWS.  The team is conceived as a unified, DevOps team of line of business product managers, developers, testers, and operations folks working collaboratively on an agile development methodology. Automation is built deeply into the process, with infrastructure as code tools used to manage public cloud virtual machines and related infrastructure components via well-documented RESTful API’s, plus other DevOps toolchain elements automating and managing most processes, including continuous integration, continuous delivery and continuous deployment of code into production.

    Traditional IT stands in pretty stark contrast.  Let’s consider an industrialized, mission-critical application that has evolved slowly for years.  The application (which is usually composed of anywhere from 5 to 10 or more components) is likely running on a mix of legacy (think RISC UNIX), dedicated (x86-based) and virtualized servers (VMWare and perhaps Hyper-V) running in a private or hosted private datacenter, using a mix of traditional and software-defined storage and connected by Ethernet and Fibrechannel switches.  The application components themselves may be legacy code, making them difficult to quickly port to more modern infrastructure.  For example, I was talking recently with a colleague who told me about his sister who still writes code in Fortran for a major bank.    With such a large, complex, long-lived and mission-critical application suite, the IT organization maintaining it and the developers creating updates and upgrades are highly concerned with quality, reliability and predictable performance.  The nature of the overall team working on an industrialized application is a far cry from the sort of “two pizza” DevOps teams on the agile IT side.  Rather, it’s more of a loose consortium of teams such as developers, testers, security and compliance, plus outsourced contractors and vendors.  The development process is waterfall-based, managed by an application lifecycle management (ALM) suite of some sort.   Automation is scant, and in particular it takes a long time to pull together infrastructure for those dev, test and other teams to perform their jobs.  Despite that, a production-like environment for testing, security, compliance, etc. is considered necessary to minimize downtime and other quality risks.  As a result, release cycles can stretch into many months.

    Modernizing Traditional IT

    When my wife and I remodeled our 1908 Craftsman-style house about 15 years ago, we discovered that an earlier renovation had actually cut out a third of a key supporting beam, leaving one part of the house literally hanging on a beam that was only attached to one side of the house.  Needless to say, we were a bit alarmed to find that out.  When dealing with mission-critical applications and infrastructure, you can’t be careless.  As mentioned earlier, simply taking the playbook for a greenfield, agile IT application and trying to apply wholesale it to a legacy, industrialized application usually won’t work.  Making traditional IT more agile requires a step-wise renovation—a risk-mitigated path that leads towards agile and DevOps practices, but without a great leap into the dark.  The steps need to establish the building blocks to agile/DevOps practice without disrupting the application, without introducing downtime, and while delivering solid returns along the way.  Successful legacy application and service ‘remodeling’ efforts observed in highly traditional organizations such as telecommunications and cable operators provides good ideas on how to take those steps:

    • Identify or Procure Automation Skills:  The initial steps to remodeling Traditional IT are about incrementally improving current practice with automation and at the same time making it ready for agile and DevOps.  This means that to be successful, you need to get some automation skills into the mix.  A team that can’t take hold of its own destiny in modernizing a Traditional IT application is not going to have a sustainable change, and that means identifying or procuring some automation expertise and protecting that person’s time to focus on the renovation rather than on firefighting.
    • Private Cloud IaaS:  One of the most serious obstacles to agility is the painful amount of time and effort it takes to allocate and provision infrastructure for developers, testers, etc. and the lack of standardized, consistent infrastructure environments, due to manual processes.  This is exactly the issue that drives some development teams to go shadow IT and start developing on AWS.  Often times, these development teams then discover that integrating that software into the production environment is plagued with translation issues.  Nonetheless, the first order of business then is for the infrastructure team to establish IaaS that can serve up full production-like infrastructure environments to devs, testers and other constituents.  This isn’t trivial when you’re dealing with a combination of legacy servers, bare metal x86, VMs, and perhaps networking, security and other devices.

    The IaaS delivers a number of key improvements.  First of all, it speeds up access because developers, security, compliance and other personnel can now get access to infrastructure in the space of minutes rather than the space of days or weeks.  More than just speed, IaaS enables asynchronous productivity by various teams to happen 24×7, so that quality progresses in an even fashion, unimpeded by office hours.  This is especially helpful if there are global teams, contractors or vendors involved.  IaaS also provides standardization and consistency across all efforts.  Finally, IaaS saves costs.  Significant amounts of operations personnel time that was devoted to painstaking manual infrastructure assembly can be repurposed to higher value activities, and infrastructure utilization rates can easily double or triple almost immediately, which reduces the need to purchase more capital equipment.

    A word here about GUI’s.  One of the key notes of dissonance that often occurs is when people hear developers say that they don’t want GUI’s, they want API’s.  That in my view is absolutely valid, but more so for Mode 2 Agile IT teams.  It turns out that IaaS GUI portals can be rather helpful in providing an easily understood reference point to multiple teams that may have highly varying understandings of the end-to-end picture of the application and its infrastructure.  For example, if an IaaS self-service portal provides a full visualization of the end-to-end infrastructure involved, a developer who might assume that latency is always near zero might become more aware that there is a WAN in the middle of the end-to-end infrastructure.

    • Test as a Service:  Many development and testing teams have taken some stab at test automation.  However, test automation tends to live and be maintained at the individual level.  The natural follow-on to IaaS is to build a standardized regression or certification of automated tests that can be offered from the IaaS portal—test as a service (TaaS).  This allows teams to access production like environments, do their development work, then easily launch a comprehensive certification test.  Tests naturally should report their results into the ALM tool so that individuals and teams can review results in near real-time to keep the process moving.  The benefits are similar to IaaS:  increased velocity, standardization, productivity and efficiency.
    • DevOps Evolution:  With a combination of IaaS and TaaS, the stage is now set to deploy a DevOps toolchain and start doing cross-functional technical collaboration, and agile methodologies.  For example, the IaaS and TaaS can be launched by a unified continuous integration server upon software development check-ins from any team.  Of course, DevOps doesn’t just mean tools and automation, but the process of coming around collaborative devtest infrastructure and common certification tests to build a common, agile development practice is one of the steps towards a more broadly responsive cycle.

    Conclusion:

    Traditional IT may not be the sexiest beast in the technology forest, but it’s often mission-critical.  Gartner’s Bimodal IT concept encourages CIO’s to let go and let Mode 2 Agile IT teams do their thing to create new culture and practices.  Translating that to Traditional IT requires a more step-wise approach.  Progressing from manual processes to IaaS to systematized, self-service test automation as the foundation for DevOps evolution is a sound way to proceed.  With these steps. Traditional IT can “do IT right” and do IT faster.

    About the Author/ Alex Henthorn-Iwane

    AlexAlex Henthorn-Iwane joined QualiSystems in February 2013 and is responsible for worldwide marketing and public relations. Prior to joining QualiSystems, Alex was vice president of marketing at Packet Design, Inc., a provider of network management software, and has 20+ years of experience in senior management, marketing, and technical roles at networking and security startups.

    Through his roles at QualiSystems, Packet Design, CoSine Communications, Corona Networks and Lucent Technologies he has acquired expertise in cloud computing and the opportunities presented through virtualization. He has written for Embedded Computing, Virtual Strategy Magazine, Datamation, SDN Central, Datacenter Knowledge and InformationWeek.

    Connect with him on LinkedIn, Twitter, or SlideShare.

  • OCP is a victory for CoreOS and Containers in general

    OCP is a victory for CoreOS and Containers in general

    A broad coalition of major tech companies have joined forces to create the Open Container Project—or OCP. The move to develop and maintain common standards around software containers vindicates the CoreOS split from Docker and reunites the two in a common cause that will make all container platforms and technologies stronger.

    A press release announcing the Open Container Project explains the overall vision for OCP. “Housed under the Linux Foundation, the OCP’s mission is to enable users and companies to continue to innovate and develop container-based solutions, with confidence that their pre-existing development efforts will be protected and without industry fragmentation.”

    Docker and CoreOS are at the heart of OCP but they’re by no means alone. A veritable “Who’s Who” of DevOps and container technologies has enlisted to support OCP, including Amazon Web Services, Cisco, EMC, Google, HP, IBM, Linux Foundation, Microsoft, Red Hat, and VMware.

    At one point Docker had a Standard Container Manifesto that spelled out some basic attributes it felt all containers should have. The Standard Container Manifesto called for containers to operate in a standard fashion, be content agnostic, infrastructure agnostic, and designed for automation among other things. When Docker removed the manifesto and seemed to shift to a more proprietary strategy CoreOS split off and launched its own container technology to continue the vision of a container standard.

    “An open, well-defined specification is required for the overall success and adoption of containers. At CoreOS, when we started the App Container spec (appc) our goal was to have a well-designed software container specification. With the OCP, we have now come together with Docker and many industry giants in an effort to unify this goal.”

    The launch of OCP and the laundry list of companies backing it show that CoreOS was not alone in its pursuit of the vision for a container standard. “We are excited to support this effort to bring the industry together to agree on a core standard for containers, which we feel will benefit all customers no matter what cloud provider they use,” said Jason Zander, CVP, Microsoft Azure. “Through our close work with Docker over the last year we have demonstrated our commitment to openness and welcome this opportunity to help create an open platform for customers.”

    “Customers need an open, interoperable future for software containers and we are happy to see the industry coming together to establish a common standard,” explained Kit Colbert, vice president and CTO, Cloud-Native Applications, VMware. “Our participation in the Open Container Project reinforces our goal to help customers innovate more rapidly and benefit from containers as they adopt the technology in their environments. We look forward to working with the community on container portability and security which are areas of focus for enterprise IT.”

    According to the OCP press release Docker will donate the code for its software container format and runtime—as well as the associated specifications—to the project. The leadership team from the Application Container spec (appc) will also contribute technical leadership and support for OCP.

    Ben Golub, CEO of Docker, summed up by acknowledging the effort and support for OCP. “We’d like to thank the many people in the industry who came together to support this important initiative. Our thanks to the Linux Foundation for the incredible work they have done in helping to pull this broad group together. Given the criticality of the intersection of containers and operating systems, we’d also like to thank Red Hat and Microsoft for their early leadership on this initiative. Our sincere appreciation to IBM and Intel for their ongoing encouragement of open governance. Finally, we’d like to recognize the members of the appc community, including Google and CoreOS, for their willingness to join all of us to constructively address these critical issues.”

    Now that all of the major players are united in a common cause to define and maintain an industry standard for containers it will be much easier for developers and organizations to embrace container technologies without reservation. The confidence that your investment of time, money, and effort won’t be wasted or paint you into a corner is very liberating.

  • The Simple Math of DevOps

    The Simple Math of DevOps

    We are living in an information- infused society, much dependent on various technological innovations taking place all around us. In such conditions, as consumers become more connected, they are empowered with more information and choices. They expect quick and consistent information available at all times. Likewise, our business leaders expect their IT teams to build relevant applications and stay ahead of their customers’ needs with an excellent customer experience.

    My IBM colleagues and I have been working very closely with major enterprise clients as they begin their DevOps transformational efforts. We’ve found few key points and messages that resonate very well with decision makers who need to set a new course for their teams in the wake of impact from mobile, big data, analytics, cloud, and the Internet of Things technologies.

    So what is it about DevOps that is so appealing? First and foremost is the notion that DevOps is all about attaining Speed to Value. Speed is king right now because so many of the business leaders we work with are under the gun to deliver value as quickly as possible. Yet, because of the high cost of technical debt they have come to a state where speed has to be sustainable. It’s no longer easiest to go outside their organizations for the high priority key projects as mobile applications were done recently. They need a holistic approach that builds on their existing capabilities.

    Well what does that have to do with math? Here’s the thing, everyone is trying to minimize the time it takes to deliver value and attain feedback.   We can think of the Time to delivery as the following equation.

    TDELIVERY = TPLAN + TDESIGN + TDEVELOP + TBUILD + TDEPLOY + TTEST + TFIX + TRELEASE + TEVALUATE

    In an optimal world or software factory that is fully automated the TDELIVERY is optimized by minimizing the time for each of the tasks required to complete the delivery. Most organizations follow a SDLC that is based on this simple math. They estimate the time to do each part and then the total project plan is the sum of the parts. Further, they tend to define work in relatively large chunks or big-up-front style of work definition which inherently increases the time for each task (TX). As a result, the industry has about a 50% (or worse) failure rate when it comes to completing a project within the estimated time. That causes a lot of hardship because the business can’t count on IT to deliver value to customers in the projected time frames let alone with speed.

    Because application delivery is a set of very human oriented tasks, a key factor that will determine the speed to value is TRUST. It’s pretty obvious when you draw out the value stream mapping of how work gets done at a given customer site. As members of software delivery teams lose trust in the validity of the work as it flows through the lifecycle then a large amount of rework and waste is introduced. In mathematical terms our equation becomes:

     

    Lee Reid Blog equation

    That is, the Tasks we do in a delivery cycle are impacted by the degree of trust we have in the hand-offs from one to another. If we have zero trust then our TDELIVERY will be infinite (divide by zero). 100% trust and our TDELIVERY will be only limited by how fast each task can be performed.

    Check this scenario that represents what we’re hearing from our clients in the workshops:

    “On a given project team the plan has about a 50/50 chance of being right, and the design is about 85% right, and the developers will get about 90% of the implementation right, and the testers have about 90% of the test cases right, and the release team has about 95% reliability of having the right stuff together to release, then your trust factor is something like

    .5 x .85 x .9 x .9 x .95 = 32% Trust”   (and that’s much better than what we usually hear)

    Ouch! That’s a 3x multiplier of the time it’s going to take no matter how fast we do all the delivery tasks.   In other words, we’re always limited by our trust factor. That multiplier comes into play with additional tasks we have to add into the equation to counterbalance the lack of trust. Then our equation becomes something like this:

    TDELIVERY = TPLAN + TRESCOPE + TDESIGN + TARCH REVIEW + TDEVELOP + TTECH DEBT + TBUILD + TREBUILD + TDEPLOY + TREWORK + TTEST + TRETEST + TFIX + TREFIX + TRELEASE + TROLLBACK + TRE-RELEASE + TEVALUATE

    And that’s the issue our customers are facing today. There are a lot of wasteful tasks, checks, and balances that have crept into our delivery process due to lack of trust.

    So, how does DevOps help this problem? It’s simple really; the DevOps approach is to attack the TRUST issue head on while simultaneously reduce the task time (Tx) in order to shorten the overall time to delivery. For example, if we apply a DevOps practice of breaking work up into small chunks, getting it out to users early, and getting the feedback, we can immediately impact the overall TDELIVERY. That’s because we’re positively impacting the numerator and denominator with that practice. That is, by limiting the scope we make it easier to understand, less effort to transform to working code, test, and deploy, sooner to obtain feedback, and more likely to know if we’re on the right track.

    Likewise for other practices!

    The key is finding the balance of what practices to apply to maximize this equation for a given organization and their current bottlenecks & wastes.

    DevOps practices address with three main impact points:

    Speed to Value depends upon TRUST

    TRUST is enabled by:

    1. Clarity: A clear definition of desired outcome(s)
    2. Collaboration: Team-wide communication and visibility
    3. Consistency: Systematic and repeatable steps

    When a team can master these three elements the time to perform each task will shrink and degree of TRUST will increase. DevOps practices impact these trust building elements:

    Clarity:

    • Break work up into smaller chunks and iterate
    • Seek a minimum viable product (MVP)
    • Define clear outcomes, ‘sponsor’ users, and playback (Design Thinking)
    • Reduce dependencies with loosely coupled architectures (containers, micro services, SOA)

    Collaboration:

    • Form cross functional teams (end to end)
    • Use big visual radiators
    • Have a ‘single source of truth’ for all development assets – Requirements, Code, Deployable Assets, Infrastructure as Code, etc
    • Be transparent with metrics
    • Plan and reprioritize frequently
    • Actively seek user feedback

    Consistency

    • Automate, automate, automate
    • Continually improve by replacing manual tasks with automation

     

    Speed to Value will be enabled by eliminating those bottlenecks and waste that have developed due to lack of trust. The key for organizations is to recognize that the current practices are maxed out and not sustainable when trying to move at higher speed. There’s a perfect storm amassing with big data, analytics, cloud, mobile and IOT that will either drive customers to make a DevOps transformation quickly or be a victim of the storm. Try using this simple math to help organizations understand how to move ahead and how our IBM solutions can be applied to attain speed to value.

    This year I have had the great experience to work with our DevOps CTO Sanjeev Sharma, our customer thought leaders such as Carmen DeArdo of Nationwide, and our field teams around the world addressing DevOps transformations. Sanjeev, Carmen and I decided to summarize our experiences in a three part blog. This is the first of that three part series.

    To learn more about IBM’s DevOps approach, check out www.ibm.com/devops. Also, watch the replay of a recent webcast on Variable Speed DevOps, I had a chance to present on, with Sanjeev Sharma and Carmen DeArdo.


    About the Author/ Lee Reid

    Lee ReidLee Reid, Senior IT Specialist, IBM, is a senior solution architect with focus on practical adoption of software engineering tools, software delivery solutions, application life cycle management, and DevOps. He has over 25 years of experience in software engineering, design, programming, testing, development, tools & methodology, team leadership, and strategic planning.