Author: Sanjeev Sharma

  • DevOps and Agility for Multi-Speed IT

    DevOps and Agility for Multi-Speed IT

    Let’s step back for a moment and look at the IT world a bit differently from the traditional technology-centric lens through which most of us IT peeps tend to look at things. Line of business (LOBs) need us, the IT organizations, to deliver capabilities to their customers and users that deliver business value. They also rely on these systems to provide feedback on how customers and users are consuming the business capabilities and deriving value; how they are interacting the business systems; and what capabilities and features they need to get better value. The LOBs are asking the IT organizations to deliver these capabilities with:

    • Velocity
    • Agility
    • Innovation
    • Cost Management

    IT organizations, in most cases, are not designed for velocity, agility or innovation. They are large organizations hindered in their ability to deliver these by legacy systems, myriad practices and tools, overbearing governance and compliance regimes, archaic and siloed organizational structures and cultural inertia. This lack of ability of the IT organizations to deliver agility, velocity and innovation in some cases even results in LOBs creating their own “Shadow IT” organizations to deliver their IT needs.

    Finding Velocity, Agility and Innovation

    The solution of late has been to undergo a DevOps transformation that enables IT organizations to adopt the necessary practices, automation tools and organizational and cultural change to deliver with velocity and agility, and become innovative by being able to deliver rapid experimentation. The origins of DevOps lie in Lean IT, and adopting DevOps is really applying Lean principles to help achieve agility, velocity and innovation.

    DevOps does so by making organizations that are delivering IT systems and software more efficient and lean. Organizations that have adopted DevOps are able to:

    • Get ideas out to production faster
    • Have users/user surrogates use them
    • Get rapid feedback

    Using this feedback the organizations can improve three things:

    1. Applications delivered
    2. The infrastructure delivered
    3. The process by which IT systems and software are delivered

    However, large organizations have been struggling to achieve the results promised by DevOps and achieved by the poster children of DevOps, with whom the movement started. For every Netflix and Etsy out there, there is a large, complex, organization that has been struggling with simply where to start. They may be able to achieve success in some parts of their IT organization—with small, innovative ‘two pizza’ teams working on isolated projects—but not at enterprise scale. They struggle with how to achieve even the results of DevOps adoption they see in their own successful teams, at scale. The organizational and cultural inertia appears to be too difficult to overcome.

    The major reason of this inability to achieve at scale is the persistent struggle in all large organizations between the desire to innovate, and the need for stability. This push-and-pull exists in all organizations—even an innovative startup feels it. The Dev teams are focused on creating innovation thru rapid change and experimentation, and the Ops teams are focused on creating stability and continuity. DevOps addresses this existential struggle by adopting a series of practices that drive cultural change, automation and process adoption that allow these Dev and Ops teams to work toward common goals and balance their needs. Even in large organizations, this can be achieved on small teams that are able to circumvent the legacy organizational structures and practices and figure out how to get all the stakeholders working toward common goals. That is why this works great on small, strategic projects that have well-defined goals and timelines and can “fly below the radar” of corporate governance enforcers. As we move to larger projects with large, potentially distributed teams, spread across different groups and corporate “fiefdoms,” adoption struggles and often fails.

    The Rise of Multi-Speed IT

    One fact these large organizations need to recognize is that they really must have two modes of delivering IT: one focused on innovation (and velocity and agility) and the other focused on optimization (stability and continuity, with agility and velocity). This view is being referred to in the industry as “bi-modal IT” and is actually more of a continuum than a clear-cut two modes model. Hence, the term is evolving now to “multi-speed IT.” However, classifying the continuum into two broad categories based on the business intents of innovation or optimization serves the purpose of determining how to adopt DevOps very well.
    Multi-Speed ITMulti-Speed IT

    One interesting way to look at these two modes is to classify them as:

    1. The ‘Industrialized Core’ that delivers core business capabilities—things that keep the business going, and the primary business intent is optimization
    2. The ‘Innovation Edge’ where experimentation happens and new systems get identified and developed; in other words, the primary business intent is innovation

    Innovation vs OptimizationInnovation vs. Optimization

    The goal of the Innovation Edge is to allow for rapid experimentation, development of minimum viable products (MVP), rapid delivery and fast failure. The goal of the Industrialized Core, on the other hand, is to be lean and efficient and deliver business capabilities in an optimized, predictable, stable and secure manner, with the ability to scale as needed. Note: Eventually systems from the Innovation Edge will need to be “industrialized” once they start serving a core business function. This is a continuum from innovation to optimization.

    The Industrialized Core itself needs to be continuously incrementally industrialized to provide the stability and reliability needed, while getting leaner and more efficient, at scale. The core needs to get to optimization, with agility and velocity, as it delivers the core systems, again at scale. The core, too, needs to adopt DevOps and agile development practices, even for legacy systems, including those on the mainframe. This Industrialized Core is, after all, what keeps the business going and delivers the ongoing business value as the Innovation Edge seeks out new and disruptive business capabilities and avenues to deliver them.

    However, they both need different DevOps approaches and platforms delivering Agility, Velocity and Innovation at different levels.

    These two systems to deliver business capabilities are, in reality, joined at the hip. The systems delivered on the Innovation Edge are dependent on the services provided by the Industrialized Core to deliver the innovations they are developing. The Industrialized Core, on the other hand, consumes most of the budget in typical large organizations, and needs to be optimized to free resources for innovation.

    Consider a typical mobile application (say, mobile banking). To deliver that application, there is a team charged with developing the mobile front end. These kind of applications are developed and delivered in the Innovation Edge. Does all of the data and business logic sit on your mobile phone when you use that application? Of course not. Most likely, the business logic and certainly the data are components of the application that are delivered by pre-existing systems or services in the Industrialized Core. So, the mobile application is hybrid, comprised of components running in both places. To deliver even some experimental features of this mobile app, the company needs to release a hybrid or composite application, delivering components delivered by both the Innovation Edge and the Industrialized Core.

    During the Dev/Test cycle, these two modes of application delivery must be as decoupled as possible, so the development schedules and cycles of one does not drag the other, to prevent the Core from becoming a drag on the Innovation. Creating an API ecosystem is thus essential to decouple the Industrialized Core from the Innovation Edge. The Core provides the business services consumed by the innovative applications being delivered by the Innovation Edge as APIs. A key benefit of developing this API-based architecture is that these APIs potentially can be monetized to third-party partners, generating a revenue source at the same time.

    It is important to note that the architectural and process needs for adopting DevOps in both of these modes are different in many ways. Their needs and how they deliver applications and services differ as they are serving different business goals. In my next post, I will start delving into these architectural and process needs for adopting DevOps for both the Innovation Edge and for the Industrialized Core, which allow the Innovative Edge to allow for experimentation and innovation, and for the Industrialized Core become more optimized.

    Watch this discussion between industry practitioners as to how they have adopted DevOps in a multi-speed IT model: the challenges faced, successes achieved and lessons learned.

    About the Author/ Sanjeev Sharma

    Sanjeev

    Sanjeev Sharma, CTO and Distinguished Engineer – DevOps Technical Sales and Adoption, IBM Cloud Unit, is a 20-year veteran of the software industry with expertise in DevOps, Mobile Development and UX, Lean and Agile Transformation, Application Lifecycle Management and Software Supply Chains. He is a DevOps Thought Leader at IBM and speaks regularly at conferences. He has written several papers and is the author of the DevOps for Dummies book.

  • The Art of DevOps

    The Art of DevOps

    This is the third in a series of posts on DevOps. The first written by my colleague Lee Reid was titled The Simple Math of DevOps. The second The Calculus of DevOps was written by IBM client, and my friend Carmen DeArdo of Nationwide. Lee introduced mathematically improving delivery throughput by leveraging DevOps to improve Trust. And Carmen used Calculus to imply that the business value of IT can be improved by adopting DevOps to increase the frequency of releases. In my post I would like to visit the art of DevOps , or the art of adopting DevOps by picking up right where Carmen left off – by looking at the culture side of adopting DevOps.

    Lee and Carmen have already alluded to some basic premises behind DevOps as an approach – reducing batch sizeand improving collaboration. These result in reducing complexity, increasing throughput, and improving trust. Let’s talk about how to get there.

    As I have worked with customers adopting DevOps, across company sizes, across industries and countries, there are four common threads that make DevOps adoption successful, leveraging all three aspects of DevOps adoption – culture, process and automation. These are:

    1. Reduce batch size: The most effective way to manage risk and quality, while increasing speed is to reduce the batch size in each iteration or ‘sprint’. This is a mind shift to release smaller, more frequent new versions. Where it is not possible to release new versions that frequently to customers, release small batches to a pre-production area. The deliverable in pre-production is tested and made customer-ready, and then released to the customer at formal release dates. As Lee and Carmen have both discussed, reducing batch size is a pre-requisite to improving both trust and business value.
    2. Shift-left Operational engagement: Get operations engaged right from the beginning of a project/initiative. This has been a core principle of DevOps. Do not leave Ops in a silo that one throws code over the wall to, but have them engaged in the development process, right from requirement inception stage. This allows Ops to have visibility into what is coming from Dev; allowing them to deliver the right production-like environments, as and when needed to Dev and Test teams. It also allows Ops to shift operational concerns left, left into the minds of developers, making them more Ops conscious and engaged in what happens after they hand-off to Ops.
    3. Continuous funding: This is a significant change in an organizations funding model to enable continuous availability and engagement of SMEs and business functions, previously not engaged on a continuous basis – like security, legal, marketing, etc. Carmen in his post mentioned several examples of ‘waiting for…’ work, someone else to do something and environments. Another ‘waiting for’ in enterprises is for funding. Funding is typically provided in a waterfall manner, with specific hard dates (Months, quarters, fiscal year) and gates, not suitable for a Continuous Delivery model. Funding too should be continuous.
    4. Create a ‘Product Management’ team: This team includes a Designer, Development lead, operations lead and an Architect at the minimum (4-in-box). They own the ‘product’ through its entire lifecycle, and beyond transient projects. They are responsible for long-term thinking, design thinking, evolutionary architectural thinking, and overall ownership of the product, from concept to end-of-life.
    5. Setting up a DevOps Center of Excellence (CoE): This center of excellence is not an administrative organization or a ‘tools/enablement group’, but a place where DevOps adoptees come to learn from each other and share expertise and lessons learnt. As organizations adopt and scale DevOps adoption, this CoE also becomes a source of DevOps coaches, helping teams and programs adopt DevOps, and own the organizations DevOps framework – their own flavor of DevOps.

    Adopting DevOps is an art. It is not as simple as hiring a consultant who can show how to improve processes; it is not just buying and adopting tools to automate manual tasks; and it is not going through ‘fall-back-into-the-arms-of-the-person-behind-you’ exercises to build trust. It is a combination of all three – process improvement, organizational and cultural change, and automation with tools to replace manual processes. Adopting these in any enterprise, outside of startups, requires overcoming organizational inertia. By changing culture not by edict, but by showing the need and value of change. By building trust, not through meetings, but with improved communication and transparency. By improving business value, not by measuring mandated KPIs, but by improving organizational agility and throughput. Overcoming organizational inertia, by delivering towards a common goal for the business, irrespective of where in the organization one is, and what role one has. By developing a culture where everyone takes responsibility of delivering value to the business.

    Watch the webcast series hosted by devopsy.kinsta.cloud in partnership with IBM or download a copy of DevOps for Dummies book for more information.

    About the Author/ Sanjeev Sharma

    SanjeevSanjeev Sharma, CTO and Distinguished Engineer – DevOps Technical Sales and Adoption, IBM Cloud Unit, is a 20-year veteran of the software industry with expertise in DevOps, Mobile Development and UX, Lean and Agile Transformation, Application Lifecycle Management and Software Supply Chains. He is a DevOps Thought Leader at IBM and speaks regularly at conferences. He has written several papers and is the author of the DevOps for Dummies book.

     

    Contact Sanjeev on linkedintwitter

  • Mapping out your DevOps journey

    Mapping out your DevOps journey

    I am writing this on a flight from Washington, DC to Austin, TX. How long does it take me to get to Austin, you may ask? The flight is 3 hours and 3 minutes long. The time it takes me, however, is much longer as I do not live at the airport (although it surely feels like it). How long a journey takes depends on where you start.

    A question I get asked often is how long will it take my organization to adopt DevOps? First of all, one does not just adopt DevOps. One starts a journey of adopting the capabilities that make up DevOps (more on that soon). Adopting DevOps is not a one-time thing – it is an ongoing thing. That said, even when it comes to adopting a single capability of DevOps, say Continuous Delivery, how long it takes you depends on where you start.

    It’s a journey

    Adopting DevOps is not a “one-and-done” project. It is adopting a mindset, a culture. It is a commitment to a journey of continuous improvement by adopting a set of capabilities and practices that are based on Lean principles. IBM has identified a set of six capabilities to adopt in this DevOps journey:

    1. Continuous Business Planning
    2. Collaborative Development
    3. Continuous Testing
    4. Continuous Release and Deploy
    5. Continuous Monitoring
    6. Continuous Feedback and Optimization

    Adopting DevOps requires process improvement, automation of the processes using tools, and organizational change to enable a DevOps culture.

    Where to Start?

    The question then becomes – where does one start? This requires knowing ‘Point B’ where you want to go – what business goals do you want DevOps to help you achieve? And it requires knowing the ‘Point A’ of your journey – where you are today – and how mature you are when it comes to practicing these capabilities today. Once you know Points A and B, you can chart out an adoption road map. Starting from a level of immaturity where you don’t even have good source code management practices, and wanting to get to ‘no downtime’ deployment, is going to be a long journey.

    The IBM DevOps pipeline Value Stream mapping workshop

    At IBM, we help our customers get started by conducting a ‘DevOps pipeline Value Steam Mapping workshop’. This strategic workshop is a 1-day (usually 6 hour) executive workshop designed to enable IBM to understand your business and IT goals (Point B) and to help you identify gaps in your existing DevOps capabilities (Point A) that IBM can help you address. We identify ‘Point A’ by mapping out your end-to-end Application Delivery pipeline, and identifying areas of inefficiencies in the pipeline.

    The result of the workshop is a prioritized list of DevOps Capabilities and an adoption ‘roadmap’ to start your DevOps adoption journey.

    ‘Get Mapped’ at IBM InterConnect

    During IBM InterConnect, The Premier Cloud and Mobile Conference in Las Vegas, February 22-26, I am conducting condensed versions of these workshops, right there on-site. These are 2-hour sessions during which IBM DevOps SMEs will work with you to map out your application delivery pipeline and identify your key ‘bottlenecks’ or inefficiencies. We can then schedule a follow-up at your site to jointly develop a DevOps adoption roadmap so you can address these bottlenecks using the relevant DevOps capabilities – along with process improvement, tool adoption for automation, and organizational change.

    Space is very limited for dedicated workshops (“DevOps Transformation Workshops”) – contact your IBM rep about nominating your company for one of the sessions. If none are available, sign-up for either the “Innovation Workshop” or “DevOps Technical Workshop,” which are similar, but will have multiple companies participating.

    For more information about IBM InterConnect 2015 DevOps Workshops and how to sign-up please visit: 

  • 5 Ways to Connect with me at IBM InterConnect2015

    5 Ways to Connect with me at IBM InterConnect2015

    interconnect-badge-250x250_im_attendingWell, I am an IBMer and going to IBM InterConnect2015 is a part of my job. And the best part about this event is that I will get the opportunity to meet all my clients and colleagues at one location instead of going to three different conference events which we use to call PULSE, IMPACT and INNOVATE. And for our clients, this is an opportunity to really get the broadest and deepest immersion into what IBM has to offer for cloud, mobile, bigdata, security etc.. and of course DevOps. That being said, I am really looking forward to it. By every measure it is going to be the largest DevOps conference ever with 300+ sessions just focused on DevOps. Although, InterConnect as a whole is not a DevOps conference – it has several areas of foci, across IBM’s massive and diverse portfolio. DevOps just happens to be one of the most significant areas of focus. At the conference, we have IBMers, sharing the IBM DevOps adoption journey with industry leaders, influencers, SME’s, and most importantly several of our customers who will talk about how they have adopted DevOps and achieved business benefits.

    I myself have multiple sessions that I am delivering/participating in at InterConnect. These include:

    1. Understanding DevOps (Session DTS-1778, Monday 12:15PM): This is my flagship session which I have been doing at IBM conferences for three years now. I introduce DevOps – it’s history, what it means, capabilities that make up DevOps, and where to start on your DevOps adoption journey.
    2. Eliminate Dogwork with Leaner and Smarter Methods (Session DEV-6306, Monday 3:50 PM): This is a unique session. I will be delivering this with Walker Royce, IBM Chief Software Economist. It will be in the dev@InterConnect, a developer focused event happening in parallel with the main InterConnect event. You can register just for this event separate from InterConnect. Walker and I will go over how developers can self-empower themselves and reduce the waste in their application development and delivery pipelines. We will explore common sources of waste and techniques on how to identify and mitigate them.

    Panels, Panels, Panels: I am on two panels with a bunch of very illustrious experts.

    1. Mobile to Mainframe Experts: DevOps Best Practices for Systems of Engagement and Systems of Record (Session DEM-4399, Monday 2PM) – This panel is focused on organizations developing multi-platform applications – from Mobile to Mainframe. My co-panelists are: Rosalind Radcliffe (IBM Distinguished Engineer), Carmen DeArdo (Nationwide Insurance), Claus Bjorno (KMD A/S), Slavik Zorin (Synchrony Systems Inc), Jeff Blackadar (CMHC) and Mads Frank Zanderson (Xact Consulting).
    2. Experts in Mobile, Deployment, Web IDE and Agile Answer Your DevOps Questions (Session DEV-1902, Wednesday 8 AM). This is a Q&A format panel with SMEs on all things DevOps. My co-panelists are: Yan Tina Zhuo (IBM), Eric Minick (IBM), Donald Thomas (IBM) and Wayne Monk (Skytap).
    3.  I will also be running a series of DevOps workshops in the ‘Inner Circle’ Event taking place at the MGM.

    So, now you know where you can catch me at the conference, but that’s not all, there is a lot to look forward to with so many different stories at so many different sessions. Check out the entire agenda here. I intend to spend most of my time in and around the Cloud and DevOps tracks, DevOps Transformation workshops, and in the expo center in the DevOps zone. Come look me up. I will also be signing copies of my updated DevOps For Dummies book (2nd Edition) on Monday and Tuesday nights in the expo center.

    Register for IBM InterConnect 2015 today to know what’s happening in the world of DevOps and understand how you can be a part of it. And get your signed copy of DevOps for Dummies book today.

  • Transforming applications with Cloud and DevOps – one hybrid solution for multiple challenges

    Transforming applications with Cloud and DevOps – one hybrid solution for multiple challenges

    Are you on cloud yet or clouded? The answer lies in how you have adopted the transformation. Today, the world is Hybrid. For large enterprises and organizations, it always has been. Almost no enterprise is monolithic when it comes to their environments. They may already have different distributed environments and/or mainframes in their ecosystem. The advent of cloud has only added to this complexity and hybrid nature of the environments. Enterprises which have replaced their datacenter with cloud adoption, in some cases are unable to do so completely or even adopt a singular cloud solution as that remains the domain of born-on-the-web companies that started on the cloud (think Netflix) or started on physical servers and migrated very early to the cloud (think WhatsApp). You cannot think of a big bank or even a telecom giant that is a Cloud Service Provider (CSP) that fits this bill.

    For the typical Enterprise (IBM included), the cloud has replaced and consolidated large sprawls of physical servers. However, physical environments remain, and may never go away and cost of migration plays a very important part in this retention. There may be large data-heavy applications that are hosted on distributed environments that may be just too expensive to re-host on the cloud. Then there are mainframe-hosted applications that cannot be migrated to the Cloud. Or the migration to cloud may be a massive multi-year transformation that you are only starting on and will have to live with workloads spread between the cloud and the physical world for an extended period of time.

    What’s interesting is that, even the cloud itself will be ‘Hybrid’ in nature. There may be combination of on-premises and off-premises clouds adopted, with off-premise’s ‘cheaper’ cloud used for less critical workloads like Dev-Test environments, and on-premise’s cloud for business-critical workloads. There may also be the presence of a Platform-as-a-Service (PaaS) being used for developing innovative Systems of Engagement applications or for experimentation, which co-exists with an Infrastructure-as-a-Service (IaaS) Cloud for Systems of Record applications and other traditional workloads.

    That is the complex world we live in.

    DevOps and Hybrid Cloud

    Cloud is the transformation enterprises are looking for whereas DevOps practices help achieve the transformation with speed and quality. When we talk about DevOps in the Hybrid Cloud environments, there are two main scenarios that can be adopted.

    1. Cloud hosted application delivery pipeline: When adopting DevOps one adopts a set of tools that create what is an Application Delivery pipeline. This includes tools such as an Integrated Development Environment (IDE), a Source Code Management (SCM) System, a Build tool, an asset repository, Test tools, an Automated Application Deployment tool, Environment provisioning tools and application monitoring tools, etc. All these tools can be hosted in the cloud and made available as a service. If you are using a PaaS such as IBM Bluemix as your application delivery platform, it includes these DevOps services.
    2. Cloud Hosted Environments: As applications are developed and delivered to production, they pass through multiple environments. These include development environments, multiple test environments, System Integration Test (SIT), User Acceptance Test (UAT), Performance Test, and potentially other pre-production environments (names and type vary from organization to organization), and of course Production. These environments may be spread over cloud and physical infrastructure, or even multiple cloud environments.

    IBM has hosted offerings for the DevOps delivery pipeline that cover the first scenario, including its Collaborative Lifecycle Management as a Service (CLMaaS) and cloud hosted Rational Software Delivery Platform (SDP) tools available through IBMs business partner CloudOne. As mentioned before IBM Bluemix PaaS includes in it DevOps Services to deliver these capabilities in the PaaS.

    For the purpose of this post, we will examine the second scenario in detail.

    Application Delivery for Hybrid Cloud Environments:

    Let’s examine a customer scenario I worked on recently. This customer, a large bank, was looking to host all their application delivery and production environments to the cloud. For cost management, they had decided to host their highly dynamic Development and Test (Dev-Test) environments on an off-premise’s ‘public’ or ‘virtual-private’ cloud, and host their Production environments on-premise on a private self-managed cloud. At the same time, their core business systems, in this case their core banking systems, all remained hosted on a Mainframe z/OS system. They are exposing their core Banking Applications to other applications that consume their services as REST APIs via an IBM DataPower gateway.

    Enterprise_updated

    The goal of this organization is to be able to deploy complete business capabilities which comprises of services and middleware components deployed to the Production Cloud, and the back-end Mainframe. The primary challenge was to use a Deployment Automation tool that can deploy components and manage application configurations across all these multiple environments.

    The second challenge was the need to deploy applications under development as they progressed through different environments in the delivery pipeline. These environments were hosted in a public cloud. As the production environments are hosted in a private cloud, there was a need to use ‘pattern’ technology that would allow for the provisioning of ‘production-like’ environments through the lower pre-production environments. These ‘patterns’ would allow for the production environments and the pre-production environments that match, albeit with different configurations and scale, to be managed and versioned as code. These patterns would evolve as the applications and the environments they were deployed on evolved through the development iterations.

    Architecture for Application Delivery for Hybrid Cloud

    Our discussion so far now leads us to designing architectures for application delivery for Hybrid Cloud which requires a deployment automation tool that supports multiple technologies comprising the hybrid cloud environment. As discussed before, multiple technologies ranging from off-premise, on-premise clouds to physical infrastructure may be included in these hybrid environments. IBM UrbanCode Deploy with Patterns provides such support.

    IBM UrbanCode Deploy with Patterns supports two cloud pattern technologies:

    1. IBM Virtual System Patterns
    2. OpenStack HEAT/HOT templates

    This allows IBM Urbanenterprise2Code Deploy with Patterns to support both provisioning of cloud environments using these patterns, and the deployment of applications to these environments, whether to an on-premise or off-premise cloud. In addition, it supports application deployment to static environments on physical infrastructure – distributed and mainframe. The Cloud technologies supported by UrbanCode Deploy with Patterns include OpenStack Private Cloud, Softlayer, Amazon Web Services and VMWare vCenter. In addition, IBM UrbanCode Deploy with Patterns also supports deploying buildpacks to IBM Bluemix PaaS.

    It is apparent from the example above that hybrid cloud is changing the face of enterprises and organizations today. With the speed the market is changing today, one cannot depend on waiting for transformation. IBM can help enterprises with their Cloud and DevOps journey leveraging our unique cloud and DevOps expertise. On November 20, you can know more about Hybrid cloud strategies and interact with some of the industry experts like Rosalind Radcliffe (Distinguished Engineer, Chief Architect for CLM and DevOps, Member of the Academy of Technology at IBM), Richard Irving (Certified Client Technical Professional at IBM) and Wayne Monk (Vice President, Business Development and Alliances at Skytap) at the IBM DevOps Symposium in the New York area (Jersey City). We are streaming the first hour as a series of lightning talks given by few distinguished industry leaders – so join us live or on the web. 

  • Enterprise DevOps: organizational change a must

    Enterprise DevOps: organizational change a must

    There has been a lot of discussion in the blogosphere, including here at devopsy.kinsta.cloud, lately about the challenges of adopting DevOps in the Enterprise space. These challenges at the end boil down to the need for organization change. There are organizational structures that appear to hinder DevOps adoption in large enterprises, that are inherent to enterprises and smaller companies do not have. In enterprises different functional teams – Dev, Ops, Security, QA, etc. may be under totally different management chains. It is not uncommon to have to go up thru several layers of management to reach a common senior executive. In other cases, there may be an entire software supply chain in place that is made up of not just internal but also outsourced or external suppliers. These suppliers may or may not ‘play ball’ when it comes to the changes in processes required to adopt DevOps. Especially if they are bound by outsourcing contracts that do not make change possible or profitable for them.

    DevOps as a philosophy has had as its centerpiece the principle that all the application delivery teams (Dev, QA, Ops, Business Analysts, Architects, Security…) need to align better and collaborate towards the common goal of delivering applications, in a efficient and constantly improving manner. This is a people and organizational principle, not just a process centric principle. To me adopting this is more important when adopting DevOps than any process or tool.

    This post discusses key guiding principles on which this DevOps driven alignment of different teams in an enterprise should be based. They are designed to improve collaboration between these teams and to break down the proverbial ‘wall’; to end the water-SCRUM-fall, when it comes to the relationships between the different application delivery stakeholder teams in the enterprise. This is not an easy effort. It depends on the existing culture at the organization, the presence of outsourced suppliers, politics, contracts and above all, the desire to change.

    These guiding principles are:

    1. Shift Left:

    Organizationally the goals of DevOps are to bring all the teams involved in application delivery together closer. Not just at deployment time, as they may do already, but all thru the delivery cycle. It requires Ops to allow Developers to take more responsibility of the operational characteristics of the applications they are building, for example. In turn, it requires Developers to keep Operational considerations in mind while building their applications. It requires analysts and architects to think holistically and work with Dev and Ops teams as they design and enhance the application. This is referred to in the DevOps world as ‘Shift Left’. As in shifting towards the left, Ops responsibilities. Shifting it to earlier in the software delivery lifecycle, towards Dev. It is shifting left the testing of your software to as early in the lifecycle as possible. It is shifting left the point at which collaboration between teams begins. It is shifting left the responsibility to deliver code to production – way left. Can someone in finance be held responsible for code not being delivered on time because they delayed approving a budget line item?

    2. Collaborate across all teams:

    Different functional teams typically use different tools to manage their projects and activities, for change management and (outside of email) different collaboration tools. In order to better collaborate across the organizational boundaries, all teams should start using the same Change Management and Work Item Management tools or in the worst case, use tools that are integrated. This allows seamless visibility across tools and traceability between respective Change Requests. Real-time collaboration using a common tool is obviously the best scenario.

    3. Build ‘Application Aware’ Environments:

    As we move towards Software Defined Environments, we have the ability to build, version and manage complex environments, all as code. All of the benefits of this are moot if the environments are not a perfect fit for the applications and more importantly the changes to the applications being delivered. The goal is hence to build Environments that are ‘Application Aware’ or are fine-tuned for the applications they are designed to run. No more cookie cutter Virtual images for all kinds of applications. More importantly, one needs to ensure that the environments are architected in a manner to allow for the evolution of the applications, both as they are developed; as they are projected to change or as they may evolve in the future. This obviously, would require close collaboration between Dev and Ops. Development Architects and Product Managers need to work with the IT Architects and Operations Managers to architect environments and their projected evolution to align with that of the application. Not only that, but also allow enough resilience in the environments to allow for unexpected change. For example, massive change caused by a super successful App. Think Instagram and how the founders had to keep changing their server environment almost daily as millions joined their service.

    4. Environment Sprints:

    Agile Development Principles prescribe that at the end of every Sprint, developers have a build. An executable build that runs, even if does not do much in terms of functionality. The next evolution of this concept is to extend it to environments. This would mean that at the end of each Sprint, the Dev team would have an executable AND the Ops team would have an environment, a production-like environment ready that the executable can be deployed on. This would allow the build to be tested. That too in a production-like environment. This would also provide immediate feedback to the Ops teams on the behavior of the application in their environment and use that feedback to improve the environment. This requires the Ops and Dev teams to align better. They will both have to use a single Sprint plan for releases – of their Builds and Environments respectively and will provide feedback to both at the end of every Sprint, using which Dev can enhance their Apps to improve functionality and performance and Ops can enhance the environment for the same. Leveraging full stack deployment – deploying at each instance the full stack that includes the environment, middleware and the application being deployed – could very well be called the ‘holy grail’ of DevOps. (Brief commercial break – IBM just announced the new offering UrbanCode Deploy with Patterns that provides full stack deployment leveraging OpenStack HOT patterns).

    The Human factor

    These principles are designed to foster Application Delivery team alignment from a teaming perspective, from a people perspective. These however do not replace the need for good old team building. Whether it is thru face to face get togethers or in todays distributed worlds, thru regular virtual meetings and online collaboration in real time. The best teams are the ones where the people know each other, trust each other, look out for each other and when there are challenges, know who to pick up the phone (or launch Skype) and talk to.

  • DevOps and PaaS: ‘Give me a platform. Let’s rock, let’s rock, today’

    DevOps and PaaS: ‘Give me a platform. Let’s rock, let’s rock, today’

    Give me a platform. Let’s rock, let’s rock, today.

    – Dewey Finn, School of Rock

    The term Platform as a Service (PaaS) has become overloaded to define any ‘platform’ that provides ‘services’ to its users. The term PaaS is formally defined by NIST as:

    “The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages, libraries, services, and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly configuration settings for the application-hosting environment.”

    – The NIST Definition of Cloud Computing, NIST, US Department of Commerce, Sept. 2011

    Like most terms in the IT industry (or to really generalize, the human communication medium known as language), PaaS is overloaded, overused and misunderstood. A quick web search or even a visit to the Wikipedia page on PaaS proves my point.

    I prefer to look at such overloaded terms in context. In this post I am going to look at PaaS thru the lens of DevOps (no surprise to a reader of this website). I will explore what it means to an organization looking to adopt DevOps and leverage the services offered by utilizing a PaaS platform. I will explore and contrast a ‘build your own’ in-house platform as a Service or a subscription based commercial PaaS offered by a vendor. I will use PaaS offerings from IBM to illustrate my points. (Disclosure – IBM pays my mortgage).

    Lets start.

    Defining PaaS:

    The most critical section of NISTs definition of PaaS, that differentiates PaaS from an ‘Infrastructure as a Service’ (IaaS) consumption model for cloud is:

    “… The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly configuration settings for the application-hosting environment.”

    This picture better illustrates the differences.PaaS-IaaS

    As you can see in this picture, what distinguishes the two cloud adoption models – IaaS and PaaS – is how much of the stack is managed by the user (or client) vs how much is managed by the Cloud Platform provider. What is important to highlight is two things:

    1. Every capability in the stack is available as a managed service – this would be a shared, multi-tenant, service available to users, with the underlying implementation abstracted from the user
    2. The Client/user only has to be concerned about managing their own applications, data and user access, leveraging the services available on the platform, which manages the rest

    Platforms as a Service may be either public (IBM BlueMix, CloudFoundry from Pivotal, Google App Engine, etc) or private (hosted CloudFoundry; self built and hosted platform). Organizations may also build their own hosted and managed leveraging multiple technologies, self hosted on a private cloud. For a private instance of course, the organization hosting it in their own data center would be responsible for managing all the services hosted on the platform. Building a PaaS on an IBM PureApplication System, would be a good example.

    DevOps and PaaS:

    When one looks at DevOps on PaaS one has to look at the services that need to be hosted on the platform in order to implement a DevOps services platform. If one looks at a DevOps pipeline and the core components that make up a delivery pipeline up, each of the components need to be available as a service, in order to provide a complete ‘DevOps on PaaS’ solution. Namely:

    DevOps Services

    • Development tools as a service (and/or an ability to connect the services on the platform to a desktop IDE)
    • Build as a service
    • Deploy as a service
    • Test as a Service
    • Monitoring as a Service

    These would be what one refers to ‘DevOps Services’.

    In IBM’s new PaaS offering – Codename: BlueMix (in open beta), these DevOps are services are an inherent part of the platform. There services on Bluemix are:

    • Git hosting as a Service
    • Web based IDE
    • Agile planning & tracking, team collaboration as a Service (Jazz.net)
    • Mobile Quality Assurance (MQA) as a Service
    • Continuous Integration as a Service (Jenkins as a Service)
    • Deployment Automation as a Service (powered by UrbanCode Deploy)
    • Performance Monitoring as a Service

    These capabilities working in concert, provide a Continuous Delivery Pipeline on IBM Codename: BlueMix.

    DevOps as a Service (DaaS?)

    So, to conclude. The value proposition for adopting a PaaS platform is self-evident. If you are an organization that delivers software, and is looking to adopt DevOps, a PaaS offering that includes DevOps services allows you to adopt DevOps at a very low cost of entry. You do not need to craft a Delivery Pipeline and implement the whole Continuous Delivery toolchain. Integrations, hosting, servicing – not your problem. Pay-as-you-go, and allow for scale. It is however, not a viable option for all organizations, as it is a drastic shift in corporate culture and potentially policies. Especially if you look at a public PaaS. If you are a born-on-the-web organization, or an enterprise with born-on-the-web applications, you should take a closer look. Check out Codename: BlueMix at www.bluemix.net.