devopsy.kinsta.cloud

Tag: transform enterprise

  • The Simple Math of DevOps

    The Simple Math of DevOps

    We are living in an information- infused society, much dependent on various technological innovations taking place all around us. In such conditions, as consumers become more connected, they are empowered with more information and choices. They expect quick and consistent information available at all times. Likewise, our business leaders expect their IT teams to build relevant applications and stay ahead of their customers’ needs with an excellent customer experience.

    My IBM colleagues and I have been working very closely with major enterprise clients as they begin their DevOps transformational efforts. We’ve found few key points and messages that resonate very well with decision makers who need to set a new course for their teams in the wake of impact from mobile, big data, analytics, cloud, and the Internet of Things technologies.

    So what is it about DevOps that is so appealing? First and foremost is the notion that DevOps is all about attaining Speed to Value. Speed is king right now because so many of the business leaders we work with are under the gun to deliver value as quickly as possible. Yet, because of the high cost of technical debt they have come to a state where speed has to be sustainable. It’s no longer easiest to go outside their organizations for the high priority key projects as mobile applications were done recently. They need a holistic approach that builds on their existing capabilities.

    Well what does that have to do with math? Here’s the thing, everyone is trying to minimize the time it takes to deliver value and attain feedback.   We can think of the Time to delivery as the following equation.

    TDELIVERY = TPLAN + TDESIGN + TDEVELOP + TBUILD + TDEPLOY + TTEST + TFIX + TRELEASE + TEVALUATE

    In an optimal world or software factory that is fully automated the TDELIVERY is optimized by minimizing the time for each of the tasks required to complete the delivery. Most organizations follow a SDLC that is based on this simple math. They estimate the time to do each part and then the total project plan is the sum of the parts. Further, they tend to define work in relatively large chunks or big-up-front style of work definition which inherently increases the time for each task (TX). As a result, the industry has about a 50% (or worse) failure rate when it comes to completing a project within the estimated time. That causes a lot of hardship because the business can’t count on IT to deliver value to customers in the projected time frames let alone with speed.

    Because application delivery is a set of very human oriented tasks, a key factor that will determine the speed to value is TRUST. It’s pretty obvious when you draw out the value stream mapping of how work gets done at a given customer site. As members of software delivery teams lose trust in the validity of the work as it flows through the lifecycle then a large amount of rework and waste is introduced. In mathematical terms our equation becomes:

     

    Lee Reid Blog equation

    That is, the Tasks we do in a delivery cycle are impacted by the degree of trust we have in the hand-offs from one to another. If we have zero trust then our TDELIVERY will be infinite (divide by zero). 100% trust and our TDELIVERY will be only limited by how fast each task can be performed.

    Check this scenario that represents what we’re hearing from our clients in the workshops:

    “On a given project team the plan has about a 50/50 chance of being right, and the design is about 85% right, and the developers will get about 90% of the implementation right, and the testers have about 90% of the test cases right, and the release team has about 95% reliability of having the right stuff together to release, then your trust factor is something like

    .5 x .85 x .9 x .9 x .95 = 32% Trust”   (and that’s much better than what we usually hear)

    Ouch! That’s a 3x multiplier of the time it’s going to take no matter how fast we do all the delivery tasks.   In other words, we’re always limited by our trust factor. That multiplier comes into play with additional tasks we have to add into the equation to counterbalance the lack of trust. Then our equation becomes something like this:

    TDELIVERY = TPLAN + TRESCOPE + TDESIGN + TARCH REVIEW + TDEVELOP + TTECH DEBT + TBUILD + TREBUILD + TDEPLOY + TREWORK + TTEST + TRETEST + TFIX + TREFIX + TRELEASE + TROLLBACK + TRE-RELEASE + TEVALUATE

    And that’s the issue our customers are facing today. There are a lot of wasteful tasks, checks, and balances that have crept into our delivery process due to lack of trust.

    So, how does DevOps help this problem? It’s simple really; the DevOps approach is to attack the TRUST issue head on while simultaneously reduce the task time (Tx) in order to shorten the overall time to delivery. For example, if we apply a DevOps practice of breaking work up into small chunks, getting it out to users early, and getting the feedback, we can immediately impact the overall TDELIVERY. That’s because we’re positively impacting the numerator and denominator with that practice. That is, by limiting the scope we make it easier to understand, less effort to transform to working code, test, and deploy, sooner to obtain feedback, and more likely to know if we’re on the right track.

    Likewise for other practices!

    The key is finding the balance of what practices to apply to maximize this equation for a given organization and their current bottlenecks & wastes.

    DevOps practices address with three main impact points:

    Speed to Value depends upon TRUST

    TRUST is enabled by:

    1. Clarity: A clear definition of desired outcome(s)
    2. Collaboration: Team-wide communication and visibility
    3. Consistency: Systematic and repeatable steps

    When a team can master these three elements the time to perform each task will shrink and degree of TRUST will increase. DevOps practices impact these trust building elements:

    Clarity:

    • Break work up into smaller chunks and iterate
    • Seek a minimum viable product (MVP)
    • Define clear outcomes, ‘sponsor’ users, and playback (Design Thinking)
    • Reduce dependencies with loosely coupled architectures (containers, micro services, SOA)

    Collaboration:

    • Form cross functional teams (end to end)
    • Use big visual radiators
    • Have a ‘single source of truth’ for all development assets – Requirements, Code, Deployable Assets, Infrastructure as Code, etc
    • Be transparent with metrics
    • Plan and reprioritize frequently
    • Actively seek user feedback

    Consistency

    • Automate, automate, automate
    • Continually improve by replacing manual tasks with automation

     

    Speed to Value will be enabled by eliminating those bottlenecks and waste that have developed due to lack of trust. The key for organizations is to recognize that the current practices are maxed out and not sustainable when trying to move at higher speed. There’s a perfect storm amassing with big data, analytics, cloud, mobile and IOT that will either drive customers to make a DevOps transformation quickly or be a victim of the storm. Try using this simple math to help organizations understand how to move ahead and how our IBM solutions can be applied to attain speed to value.

    This year I have had the great experience to work with our DevOps CTO Sanjeev Sharma, our customer thought leaders such as Carmen DeArdo of Nationwide, and our field teams around the world addressing DevOps transformations. Sanjeev, Carmen and I decided to summarize our experiences in a three part blog. This is the first of that three part series.

    To learn more about IBM’s DevOps approach, check out www.ibm.com/devops. Also, watch the replay of a recent webcast on Variable Speed DevOps, I had a chance to present on, with Sanjeev Sharma and Carmen DeArdo.

    About the Author/ Lee Reid

    Lee ReidLee Reid, Senior IT Specialist, IBM, is a senior solution architect with focus on practical adoption of software engineering tools, software delivery solutions, application life cycle management, and DevOps. He has over 25 years of experience in software engineering, design, programming, testing, development, tools & methodology, team leadership, and strategic planning.

  • Follow the Unicorn: Transform your Enterprise Applications

    Follow the Unicorn: Transform your Enterprise Applications

    For those of us that work on Enterprise Systems such as the mainframe, the claims that the Unicorns (1) make, can be pretty astounding. Dozens and sometimes hundreds of releases a day from big companies like Netflix (2) and Etsy (3) are hard to comprehend. But look a little closer and you can see that the key difference is in the way changes are released to customers, not so much the total volume of changes. A traditional enterprise application may have thousands of changes made over the course of 3 to 6 months, but all of those changes are released at once. The Unicorns are typically able to release a very small number of changes at a time because they can develop, build, test, and deploy these changes very efficiently. Small batch releases to customers has significant value. It enables you to get feedback very quickly, and it is significantly easier to verify because the change is not mixed in with other changes.

    So how can you transform your development environment to deliver small batch of changes? I like to tackle the problem like any other computer science problem, no doubt because of my background. Break the large problem into a set of smaller problems and then solve those smaller, less daunting problems:

    Step 0: Start with a Proof of Concept

    Pick a Proof of Concept (PoC) to begin with that is important, but not mission critical. The PoC can be used as a learning ground and hopefully as a template for subsequent PoC’s. Make sure you have a team that culturally is willing to transform. Often the hardest issues to tackle have nothing to do with technology, but are cultural.

    Step 1: Understand your Application

    You need to have a good understanding of what your application consists of. Not only does the source code make up the application, but what source files are used, what the middleware configuration might be, what scripts you may have to build and deploy the application, and what data files (e.g. flat files, databases) that you use are also important. Some of these components may be shared with other applications, so you need to understand those dependencies as well. Once you have a good handle on the application, you can then automate the application build and deployment process so that all members of the team can work efficiently.

    Step 2: Understand your test cases

    If you can build and deploy the application quickly but it still takes days or weeks to test, then testing prevents you from delivering to your clients. An important shift for testing needs to be from testing the entire application to testing the change. You need to focus on running the right tests to ensure the change is correct and causes no regressions. Tests that do not exercise the code change are irrelevant and can be skipped. For many existing applications, the test suites may consist of hundreds if not thousands of test cases. Many of those test cases may be duplicates or testing areas of the code that are not being changed.

    Step 3: Understand your processes

    How fast can you get your updates to your customer? If your bug fixes and enhancements have to sit on the shelf for days, weeks or months before the customer gets them, they will not be satisfied. Streamline not only the development processes but the operations processes in your organization.

    To know more about the topic with the help of a real life case, watch the replay of a recent webcast which IBM in partnership with devopsy.kinsta.cloud hosted with Canada Mortgage Housing Corporation’s Jeff Blackadar on how breaking down IT silos helped speed up mainframe development for them.

    Also, do let me know your thoughts on the blog and on the topic.

    1 the born on the web technology teams

    2 http://www.slideshare.net/jedberg/devops-at-netflix-reinvent

    3 http://www.slideshare.net/beamrider9/continuous-deployment-at-etsy-a-tale-of-two-approaches

     

    About the Author/ Mike Fulton

    MikeFaceMedium-150x150A IBM Distinguished Engineer, CTO Enterprise DevOps at IBM,  Mike is a software development professional specializing in compilers, debuggers, profilers and virtual machines. At IBM, Mike works towards understanding and influencing trends and directions in Enterprise IT delivering new technologies into the DevOps Portfolio. He also overlooks integration of the DevOps Portfolio into IBM and Enterprise Software Solutions. Mike is an IBM Master Inventor with several patents to his credit.

    He is an avid blogger and regularly writes on his blog site – makingdevelopersliveseasier.wordpress.com

    linkedintwitter

  • The Last One That Transformed the Future

    The Last One That Transformed the Future

    The headline made her yelp for joy. A year into the future of D-Chip, on a crisp August morning, Phoebe opened the newspaper and swiped to the technology section on her iPad. What she saw was riveting: D-Chip reborn as a star in wearable computing.

    She immediately thought of the article Marc Andreessen wrote on how software transformed the enterprise. It’s true of our company’s story, she mused. The credit for making the headline a reality was due to the stressful events of the last months. Who said stress couldn’t be a game changer? As she picked absently on a plate of fruit and relished the article for the second time, she recalled the events of the past year.

    Flashback a year ago at D-Chip

    Remember how Rachel pitched her enthusiastic idea to Chandler? She wanted to engage six engineers in six months to build an in-house solution. A solution that would integrate D-Chip and Enogo’s systems by letting them deploy in both OpenStack and AWS and manage wearable chip code from a single deployment platform.

    Chandler agreed in theory with the need for a single management platform but didn’t see the sense of doing it in-house. Why side-track from your business goals and spend months building a solution yourself when there are specialists solving this exact problem for many companies facing the same issues? After much debate, they came to a consensus. They decided to evaluate solutions from the best-of-breed products out there and transform their software development lifecycle to reflect the best agile practices. Thus was born the iTransform project.

    The iTransform project

    The name itself was a reminder that transforming was as much a matter of people internalizing and accepting a new way of doing things as much as it was about aligning and unifying D-Chip and Enogo systems. The first order of priority was to understand the pain-points that assailed the company. After a lot of meetings and detailed analysis, they identified key use cases from internal stakeholders, namely the engineering and IT operation teams:

    • Engineering personas, developers, QA, architects, all said the same things. They didn’t have access to the latest tech because it took weeks or months for approval processes to travel up the chain. Then they waited on builds and environments from IT operation teams. Most of the time, the environments in the private datacenter OpenStack cloud didn’t work properly, and deployments never succeeded the first time. It’s no wonder they took weeks or months to release anything decent.
       
      Self-service and continuous delivery were, therefore, the most important criteria for developers, QA, and architects. Without waiting for IT admins or operations teams, they wanted to spin up build environments in test, staging, even production on their choice of pre-configured infrastructure. They wanted to consume in an on-demand fashion the latest technologies on multiple clouds. Today they were using AWS cloud and OpenStack, but tomorrow they wanted any cloud that matched their workload configuration needs best.

      •  

    • IT operations and system admins brought up their set of challenges. Their primary goal was to configure environments for repeatability; safeguard builds from failures and structure stable workflows. Not an easy job come to think of it. On top of this demands from the engineering teams dictated that they learn and implement new technologies in a very short time. Technologies ranged from configuration management tools, cloud platforms, storage, networking, load balancing services, monitoring services, and more. Besides when engineers wrote configuration templates as Enogo did for AWS, it was hard to figure out who owned the deployment stack. In the case of problems, it resulted in pointless blame games. So they desired a clear level of separation between code, deployment automation, and infrastructure. Technologies and cloud platforms could come and go, but IT operations remained responsible to maintain smooth workflows, stable builds, and error-free deployments.
       
      A service catalog also met the needs of the IT operations teams. They wanted to build and manage specific platforms, runtimes and version configuration changes. Collaboration was a key factor. Many IT operations team members and engineers were experts in some but not all technologies and tools. But together in a collaborative platform, experts could contribute at their skill level and share knowledge with others. Automating configuration changes in bulk saved a ton of time. Therefore, API support for automation was high on their wish-list.

      •  

    • The IT managers were important though indirect stakeholders. They often approved and handled audits, compliance, and budgetary aspects. Policies and cost management were paramount to them. IT managers needed control and oversight to implement compliant deployment policies that gave the right level of access to the right set of infrastructure on pre-approved cloud providers. They needed to set and monitor quotas and track IT spend on cloud deployments against approved budget.

    Transformed enterprise

    After three months of evaluating proof-of-concept solutions with vendors, they zeroed in on a solution that met all of their uses cases and helped set up their current builds. Because of this, the teams were able to ship code ten times faster. It was only a matter of time before D-Chip stood out as a market leader. It was clear to the company executives what was driving this transformation.

    They retired the semiconductor business and switched their business focus entirely to SDK builds of software for wearable computing. The software services for wearable chips was now way more valuable than any piece of hardware. Their customers in wearable computing now consumed D-Chip technology as a service in the cloud. A faster and efficient software development lifecycle enabled them to carve a new market niche and become the de facto leader.

    The transforming story of D-Chip is a reminder for us all. When there are problems in software development, by aligning goals and helping people in different roles work together and participate in the development lifecycle, you increase the pace of innovation dramatically. It can be so powerful that it can transform the company’s core products or services.

    Disruptive change doesn’t happen one step at a time. It happens with mavericks and innovators like Rachel and Chandler leading radical changes in a focused group effort. You want innovation to transform your business? Get a dedicated group of mavericks together and make it happen.

    Just like mankind advanced their understanding through each cycle of civilization from the stone age up to the Renaissance and the Industrial Revolution, the future is heralded by the IT age of our times.