devopsy.kinsta.cloud

Tag: continuous deployment

15 DevOps Expert Opinions on Continuous Integration and Delivery

We at Shippable recently approached some DevOps experts and asked them what they really think about continuous integration (CI) and continuous delivery (CD). The responses were just amazing, proving that CI/CD’s future is just bright and long in the software development process.

Perhaps this insight might be of help to organizations, regardless of whether they already practicing CI/CD or have yet to adopt it in their organization.

Sit back, relax and read these opinions from DevOps Experts:

1. Chris Nwamba

Chris Nwamba is a self-described code beast who works with scotch.io. “Continuous integration is simply ‘building big and shipping small,’” he said. “The developer or engineer is not expected to produce a complete result; the project is divided into milestones and he or she can produce the expected output for each milestone which is reviewed and pushed. Continuous delivery, on the other hand, involves delivering and deploying a tested and usable feature to the end user as soon as it is available from the engineer. No constraints such as milestones or daily reports are placed on the engineer; rather, the product is delivered when it is ready and tested within a reasonable amount of time.”

2. David Bernstein

David Bernstein is the author of “Beyond Legacy Code: Nine Practices to Extend the Life (and Value) of Your Software.” He says, “I’ve written a lot about CI and CD. It’s one of the nine core practices in my new book. The first principle of the Agile Manifesto is, ‘Our highest priority is to satisfy the customer through the early and continuous delivery of valuable software.’ The practice of continuous delivery is the only way I know to achieve this principle and is at the very core of Agile software development.”

3. Tomohiro Iizuka

Tomohiro Iizuka is a full-stack web application developer. He says, “Continuous integration and delivery are still at the frontier of IT technology, like NoSQL. It will change the business style of freelance engineers. Businesses using cloud servers will especially change. However, there are some issues to solve in order to execute it. An IT system consists of some subsystems that affect how teams work together on development and management after the first release. If an engineer can provide a subsystem that’s comprehensive to end users, freelance engineers can have their own product and they can concentrate on maintaining their own product continuously. That will make use of cloud platforms more popular, and they can have opportunities to gain profit continuously in addition to development fees. Open-source products like Eclipse, Firefox, WordPress are some examples. However, in general, that is not done. I think that the platform for making independent software components is necessary for freelance engineers to make new businesses with continuous integration and delivery.”

4. Anubhava Srivastava

DevOps expert Anubhava Srivastava is a lead engineer-architect at AOL Inc. According to him, some of the advantages of CI and CD are: “DevOps and CI/CD give more power to developers. Developers have greater control over the application stack and require access to production servers. The exact same repeatable deployment process is used across different environments like dev, QA, staging and production. Everything is automated right after code is checked in. There’s fast and quick resolution time for emergencies. Due to code coverage reports being monitored and shared widely, developers are forced to write test cases for every line and every condition in the code. Due to stress on TDD, developers are producing higher-quality applications with significantly fewer bugs. It has definitely helped the organization, as they don’t need the same headcount for QA and Ops teams anymore.”

5. Erik Dietrich

Erik Dietrich is the founder and principal of DaedTech, LLC. He says, “In short, both of those [CI and CD] are practices that I endorse and help people move towards. On the whole, I’m a huge fan of automating the automatable to avoid mistakes and save time.”

6. Tom Zorde

Tom Zorde is an enterprise architect at Monadelphous. He says, “There is an inevitable need for CI, no doubt. But developers need to have autonomy to self-elect to skip a check-in if they don’t feel their code contribution is ready. This will reduce the wasted effort of dealing with failed integrations and keep developers focus in the zone.”

7. Ian Robertson

Ian Robertson is a product owner at River Island. He says, “I work in retail (fast fashion). While we do build core ranges to meet the seasons, it is our quick response to rapidly changing trends that generates the excitement that drives our brand values. An emerging style cue seen on the catwalks one Friday can be in our stores and online by the following weekend. Those same customers expecting the latest fashion will be interacting on our app and website, so we need to be as Agile in development and deployment as we are in apparel design and production to keep up with customer expectations. It’s not just the front end; our supply chain, logistics and merchandising systems need to be able to respond rapidly to real-world changes. Adopting CI and CD allows us to reduce the time to market, reduce risk, deliver better customer experiences and drive value for the business.”

8. The Folks at Browser Up

Browser Up is a performance and load testing tool. Folks there say, “Using CI and CD is a force multiplier that amplifies the speed at which software can be developed and delivered. To be effective, a CI and CD pipeline must include a robust test suite that include continual performance and functional testing.”

9. Georg von Sperling

Georg von Sperling is a practitioner, evangelist and general geek on software development methodologies, technology, patterns, systems, delivery practices and DevOps. He says, “CI and continuous deployment are very technical in nature and benefit the technical teams. They are crucial to address. They do not, however, define continuous delivery. CI could be defined as an early detection system at the code and build level. Early detection at those levels have been proven beneficial as these are types of bugs that are exponentially more costly to remediate at later stages. Continuous deployment is an ability to promote code technically down the pipe into production at velocity, allowing consumption faster. There are facets of continuous deployment that must be considered: rollback, A/B testing and metrics gathering. Many fail here very quickly. Continuous delivery (of which I am an avid advocate for) is a business subject; although, in software powered organizations and/or digital business models, it has a fairly large technical component. It is very difficult, if not impossible, to translate CI and continuous deployment into direct business benefits. Continuous delivery, however, is business metrics-driven and defensible at the CxO level to stakeholders and shareholders.”

10. Endi Sukaj

Endi Sukaj is a full-stack developer who specializes in back-end development. He’s currently a community team member at Slant.co. He says, “CI is a crucial part in the development and deployment process of projects of a certain size especially if two or more people are working on it. The most valuable resource is time, and a good CI service and implementation helps developers in saving as much time as possible by avoiding ‘integration hell’ and by running all unit tests in the same environment as in production.”

11. Jordi Mozcode

Jordi Mozcode is a software developer focused on .NET Core, MVC, TDD, SOLID, clean, software quality, best practices, secure coding and pentesting. He says, “In my professional experience, the adoption of continuous integration and continuous delivery improves three basic facts: quality, flexibility, and speed. Quality, because CI guarantees a much lower rate of development bugs and CD provides a robust and secure system throughout the development cycle. Flexibility, because at every step of the development cycle you have the control of continue to next step or going backward easily. Speed, because in an idyllic case with no development bugs the release can be deployed in production almost instantly.”

12. Dave Kerr

Dave Kerr is a software developer for everything from device drivers to full enterprise applications. He says, “If the org is building tech services or products, CI and CD are critical. Deployment doesn’t need to be continuous but the team needs to be confident that what they are building is shippable.”

13. Matt Kurleto

Matt Kurelto is a serial entrepreneur and CEO at Neoteric. He says, “I think it’s a necessity if you are thinking seriously about cloud services. I think that microservices architecture can only be well-leveraged with automated CI, CD and QA flow, and for me, this setup is a must if you want to take advantage of lean business and agile project management.”

14. Sean Handley

Sean Handley is the lead cloud applications engineer at DataCentred. He says, “CI and CD are things we practice on our tech team. They allow us to ship production changes many times per day in confidence. My view is that any software team not using CI and CD needs to seriously reconsider their methods!”

15. Manisha Sahasrabudhe

DevOps Expert Manisha Sahasrabudhe is the co-founder at CMO at Shippable. She says, “All companies are software companies today, and faster innovation is the key to winning against your competitors. Continuous integration and delivery are crucial in helping you accelerate your release cycles and innovate faster than anyone else. This will become especially important as microservices become ubiquitous and your application is composed of hundreds of services that need to be deployed, scaled and monitored independently. CI and CD automation is the only way to bring some method to the madness.”

We hope these DevOps Experts opinions are enough for you to figure out how important it is to adopt to CI/CD practices.

— TeamShippable

January 4, 2017
How Often Should You Release with Continuous Delivery?
If you do continuous delivery, you could release software updates as frequently as you want. But should you? Here’s what to consider when deciding how often to release.

Although the terms continuous delivery and continuous deployment are often used interchangeably, they don’t mean the same thing. Under continuous delivery, you prepare all software updates as potential candidates for release. But continuous delivery on its own does not mean that you release all of those updates into production as soon as they are complete. You would have to adopt continuous deployment to do that.

So, if you’re doing continuous delivery, you have the option of releasing as frequently as you like—every day, every month or even every hour. Determining how often you actually release requires weighing several important factors, including:
- How important are your updates? If changes to your app mostly involve mundane things such as interface tweaks, continuous deployment probably will confuse users more than please them. They will have to adapt constantly to new features. But if you can roll out big, user-friendly changes on a continual basis, then deploying continuously can help keep your users happy.
- Is there a marketing advantage to be gained by waiting longer to release? Sometimes, advertising new features but waiting a while to release them can help build excitement. Case in point: Apple Inc.
- Do other apps depend on your app? If you’re working within an ecosystem where your app integrates with other apps or platforms, continuous deployment makes the lives of partner developers more difficult because it requires them to adjust constantly to changes in your app. To help them ensure their work will always be compatible with yours, provide space between your releases.
- Is your development pace consistent? If not, continuous deployment can be problematic because users will see rapid updates at some times, but stagnation at other times. Continuous deployment only works well for users if they can count on a relatively steady stream of updates.
- Do your updates create a learning curve? If users will have to learn new tricks to use an updated app, continuous deployment may create more confusion than happiness.
- Can multiple versions of your app coexist? If all users need to be running the same version of your app in order to be current, continuous deployment can help assure that they are. But if you can support multiple versions of the same app at once, less frequent releases should work for you.
If you can’t decide how frequently to release, you might consider releasing on a fixed time schedule—once every two months, every six months, every year or whatever works for you. This is what Canonical has done with Ubuntu Linux for a long time: It releases a new version of the operating system on the second Thursday of October and the fourth Thursday of April each year (with a few exceptions). This system assures that developers and users always know the schedule for a new release.

Whatever approach you take, the most important thing is to realize that continuous delivery does not require you to release as often as you can. It gives you the flexibility to release as often as you want. Deciding how often that should be is up to you.

— Chris Tozzi
December 16, 2016
Continuous Integration vs. Delivery vs. Deployment: What’s the Difference?

“Continuous” is one of the most popular words in the DevOps lexicon. What does it really mean, and what is the difference between continuous delivery, continuous deployment and continuous integration? Keep reading for some perspective.

Like many technological concepts, continuous integration, continuous delivery and continuous deployment are terms that DevOps teams tend to use without defining them precisely. For that reason, it’s worth taking a closer look at what each of these terms actually means.

Continuous Integration

Continuous integration can really mean one of two things, depending on the context.

In the narrower sense, continuous integration refers to the work done by a continuous integration server, such as Jenkins or Bamboo. A continuous integration server automatically tests code written by individual developers to make sure it can be merged into the main code base. By automating this process, continuous integration servers help developers to write and test small chunks of code on an ongoing basis, which minimizes the risk that a code change could cause serious problems and force developers to revert their application to an earlier state.

In a broader sense, continuous integration means the constant integration of changes to an application at all stages of the delivery chain. This includes but is not limited to the automated integration testing and code merging performed by an integration server. In this broader definition, continuous integration can include the integration of new design concepts to the delivery pipeline, for example. It also can involve other types of software testing beyond just automated integration tests.

Continuous Delivery

In most definitions, continuous delivery refers to the process of delivering software updates to users on a nearly constant basis. Continuous delivery is made possible by continuous integration and other optimizations at earlier stages of the development process. But the definition of continuous delivery gets a little cloudy when you start comparing it to continuous deployment.

Continuous Deployment

What’s the difference between continuous delivery and continuous deployment? For some DevOps teams, nothing. People tend to use these terms interchangeably. In fact, they are usually simply abbreviated as “CD,” leaving it up to the listener to decide what the speaker is actually talking about when he hears the term during a conversation about DevOps.

Some developers draw a distinction between continuous delivery and continuous deployment, however. For example, Mirco Hering writes that continuous delivery requires that the DevOps team manually release updates to users. In contrast, the continuous deployment pipeline is fully automated; users get updates as soon as they are written and tested, with no manual intervention by developers.

This distinction is a good way to think about the difference between the closely related concepts of continuous delivery and continuous deployment. It’s a distinction that will become increasingly important as technologies such as Docker make it easier to automate application deployment by allowing DevOps teams to drop new container images into production.

What Does Continuous Mean, Anyway?

It’s worth mentioning, too, that the word “continuous” is usually something of an exaggeration when used by DevOps folks. It implies that software changes roll down the pipeline in a totally constant, never-stopping fashion.

That is an exaggeration. Integration, delivery and deployment are almost never completely continuous. In practice, a “continuously” integrated application is likely to be rebuilt and delivered something like every 24 hours, not every single time a code change reaches the end of the pipe.

That rhythm is still a lot faster, and closer to continuous, than the “waterfall” delivery pipelines that were common before the introduction of DevOps. But it’s still important to understand that, even with all of the automation tools available today, software development is rarely completely continuous.

— Chris Tozzi

November 18, 2016
7 Strategies for Continuous Delivery and Deployment Success

One of the key advantages of cloud-native applications is the ability to quickly push updates into production for specific microservices—giving developers the ability to respond rapidly to changing business requirements without having to upgrade the entire application. Successfully doing this requires far more than just leveraging a continuous delivery platform. Everything from the culture of the development group to the type of testing required to the underlying platform it will run on is critical.

Based on applying DevOps principles across several projects to date, I have found that development teams can benefit from following these seven key strategies before launching continuous delivery and deployment initiatives. By doing so, you’ll increase your opportunity for success.

Culture

Just having the right tools and technology isn’t enough in today’s enterprises. Successfully adopting an agile, continuous delivery and deployment process will require a culture shift within most development shops. Developers are no longer responsible only for their code; they must take full ownership of the development, testing, automation tools and, most importantly, building operational capabilities into the pipeline so the move to production is easier. Getting developers and the operations team on board with this may take some time and training, but the effort is critical, and your projects won’t succeed without this.

Automation Platform

Carefully consider the tool you’ll use for your continuous delivery pipeline because the current tools on the market have different optimal use cases. For example, if you need to do A/B testing you’ll want a tool that has capabilities for phasing the rollout based on the test results. If your application requires multiple updates in a short time span, you’ll need a tool that has advanced release management capabilities including good pipeline visualization and support for the management of cloud environments.

App Architecture

Design your applications so they are cloud-native. This is typically associated with microservices and containers and will enable you to independently and frequently update each service independently. The typical stages of a deployment pipeline are build, unit testing, integration testing (including functional testing), API testing, performance and staging, then, finally, on to production. However, with microservice architectures and advanced deployment techniques, more and more customers are bypassing certain stages and testing in production, albeit very contained production environments.

Each stage also can include the creation of dynamic environments in the cloud, enabling the stage itself to be provisioned, executed and decommissioned as a part of the pipeline. If you choose to do this, look for a tool that has a number of built-in cloud plugins, so creating and destroying environments in AWS, Google Cloud and other clouds can be done easily.

The application and infrastructure stack for each microservice also should be specified. This could include the application Jar file, a Docker file, or server image such as an AMI; any related services such as AWS Lambda functions; and all configurations and policies. These specifications all should be managed in an infrastructure-as-code manifest, which ensures versioning, consistency, change auditing and testing. It also ensures that the build resources, such as baked images, JARs and AWS Lambda functions, are consistently passed to each stage of the pipeline, from testing to production, eliminating the risk of imposing inconsistent environments and configurations. An AWS CloudFormation template is an example of an infrastructure-as-code manifest.

Security

Security is a necessity in the world of increasing threats that we live in; however, the common approach of waiting to test security right before the push to production is the wrong approach. Security should be built in to the pipeline itself and automated as part of the DevOps CI/CD process. It’s essential to validate the building blocks without slowing down the process. For example, leading companies are automating code scanning, template checking and other security tests in within the phases of the pipeline itself.

Insight

Visualizing and tracking pipelines, environment variables, test results, versions, issues and other artifacts allows developers and the operations team to share a common view throughout the project life cycle. This improves collaboration and traceability.

Delivery Strategies

Automating tests is critical to pipeline design and delivery. Without comprehensive, high-quality, fully automated tests of the deployment pipeline, production failures will occur. Code coverage no longer is a good representation of code quality. In the cloud-native world, test cases and test suites need to be mapped to functionality and business outcomes desired to ensure that the test team gives the green light and then everything goes red in production. In situations where time limitations prevent a comprehensive approach, first automate the critical capabilities and flows, then assess the cost-benefit of automating additional tests.

Be sure that the data and environment for testing and automating purposes mirrors the production data and environment as closely as possible. Most pipelines have a “pre-production” stage before pushing software into production. You may consider building multiple stages across multiple AWS regions before pushing the software to production.

If your goal is to update software in production very frequently and without any downtime, make sure your pipeline tool can easily manage multi-region deployments, such as blue/green and canary deployments for server groups and clusters across AWS and other public infrastructure-as-a-service (IaaS) clouds.

Monitoring and Feedback

Finally, it is essential to get continuous feedback on the user experience, response times, performance of key business services, and the technical metrics from all stages of the life cycle—development, QA and production. Only through this continuous feedback can you ensure that no degradation occurs as you push out releases with increasing frequency. This too needs to be built into each phase of the pipeline to understand performance and improve things earlier in the process, where it costs much less.

There’s no doubt that a successful continuous development and deployment process can deliver significant benefits to users, who constantly require new and enhanced capabilities and can’t wait the weeks or months that a traditional development cycle requires By carefully considering these seven strategies, your projects will benefit from reduced application errors, less downtime and much happier users.

About the Author / David Cramer

David Cramer is vice president of Product Management for the Cloud/DCA business unit at BMC. Prior to BMC, David was head of product management for CA Technologies. During his tenure at CA, David was responsible for application delivery, cloud management, virtualization and Infrastructure automation solutions. Before joining CA, David held executive positions at AlterPoint, Motive, NetSolve and Nortel Networks. David’s focus is on continuous delivery, cloud management and web-scale IT. David received his MBA from Southern Methodist University and a BS in Finance from Georgia State University. Connect with him on LinkedIn and Twitter.

October 28, 2016
An IT Practitioner’s Guide To DevOps
Two technology innovations of the past decade have fundamentally changed how IT works and given rise to the “DevOps” phenomenon. Before we dive into what DevOps is and how IT organizations should adopt this culture, let’s review what gave rise to it in the first place.

Pull from Agile Development

The last decade has seen many organizations abandon the traditional “waterfall” software development process in favor of “agile” development. Agile is all about small, specific user-centric stories, described by a product owner and delivered in short sprints of two to four weeks by small, focused development teams.

Agile development led to the emergence of more frequent code drops from software engineers, which needed to be deployed more quickly. No longer was it okay to take days or weeks to deploy a new software version; developers working in two-week sprint cycles want deployment to work within minutes with a high degree of predictability.

Enabled by the Cloud

Before the cloud came along, access to infrastructure required human interaction and manual operations. Processes typically could not be automated effectively. This meant that operations were slow and error-prone and created a huge point of friction in the software development life cycle.

Around the same time that developers were adopting agile development, the cloud became real, and infrastructure could be orchestrated effectively using APIs to provision everything from machines and networks to storage, routing policies and firewalls.

DevOps: Closed Loop Automation Between Development and IT

Agile development and cloud computing created the perfect storm in which DevOps was born: this new way of doing IT is all about ensuring closed loops, with 100 percent automation.

Figure 1: DevOps is about ensuring a closed-loop, fully automated workflow between software development and deployment.

In the simplest terms, this meant that each of the steps outlined in Figure 1 need to be fully automated, and integrated with preceding and following steps. Doing this ensures software can be developed faster, deployed quickly, and iterated upon more rapidly.

Benefits

Done right, DevOps leads to a virtuous cycle of greater productivity across development and IT, greater agility in bringing new experiences to market and greater impact on the business.

Adopting DevOps: You Can Phase It In

Organizations looking to adopt DevOps should do so in phases and define metrics for each phase. Here’s a typical phased approach that you can use as a baseline and customize to your organization’s unique needs.

Phase 1: Adopt Agile Development and Cloud Computing

Agile development and cloud computing are profound technology shifts and create the underlying culture in which DevOps can thrive. So, the first step is to ensure organizational buy-in for both of these trends:
1. Agile development can be adopted in stages. The most important is to agree that software will be built and shipped in short, two-to-four-week sprint cycles, and with a focus on user stories.
2. Cloud computing can be adopted no matter what your IT infrastructure looks like today. Use hybrid cloud platforms such as OpenStack to provide programmatic access to a VMware or Linux environment, or use a public cloud platform such as Amazon Web Services.
Phase 2: Commit to a CI/CD Workflow

Continuous integration (CI) systems such as Jenkins or TeamCity provide a way to pipeline and integrate the various steps in a DevOps workflow. The choice of system doesn’t matter as much as ensuring that all steps of the workflow will be integrated (eventually) into this workflow.

Phase 3: Repeatable, Automated Software Deployment

Many organizations tend to have some manual steps in the packaging and deployment of their software, especially for getting it into production. This often reflects underlying assumptions about manual oversight and intervention, and it breaks the closed-loop philosophy of DevOps.

To succeed with DevOps, you need to fully automate the software packaging and deployment process. In addition, ensure that production builds are deployed in the exact same manner as developer builds, using identical environments.

Cloud technologies such as orchestration (OpenStack HEAT or AWS CloudFormations) and configuration management tools such as Ansible, Puppet or Chef can be used to implement this phase.

Phase 4: Automate Software Testing (Continuous Delivery)

Once software deployment has been automated, the next phase is to automate the validation process for new software releases. If you’re like most organizations, testing involves several rounds of back and forth with QA teams and product managers, and it often takes weeks to resolve the last few bugs in each release. To reach the next level of DevOps automation, this process has to be completely lights-out and automated.

This requires a fully automated software test suite, across the back-end and front-end, so that product managers have confidence in the quality of the software being released. Once implemented, you finally have a closed loop between the critical elements of the DevOps life cycle. While release to production may still require someone to approve it or “push a button,” this is a very respectable state in the DevOps journey.

Phase 5: Continuous Deployment

To get to the most advanced DevOps phase, you need to automate the “go live” process for software releases. This in turn will allow you to take even smaller, more frequent code drops and push them into production.

The most advanced software teams in the world now operate this way, and the organizational implications can be transformative: Developers come into work feeling that they can implement a feature and have it go live immediately, and product managers can make commitments to the business with the confidence they can deliver, because they have this incredibly efficient software delivery machine backing them. Most importantly for you, IT is no longer seen as a cost center; it’s seen as the fundamental enabler for business productivity and time-to-value.

Closing Thoughts

Getting to Phase 5 represents the state-of-the-art in the industry today. However, another significant breakthrough promises to raise the bar even higher.

In each of the prior phases, the DevOps workflow is integrating software development with IT operations; but there is nothing influencing software architecture to take operations into account.

In other words, we’re still bolting IT operations automation onto software development.

This is about to change. We’re on the cusp of a new era in software: instead of bolting operations onto the software development process as an afterthought, a new framework promises to bring development and operations together by design. With declarative, infrastructure-agnostic constructs to describe how applications are composed, interact and managed, Kubernetes promises to increase the operability of modern software systems by an order of magnitude.

Everyone interested in the DevOps journey should be learning about Kubernetes and evaluating how it will improve your software development and operations workflows.

About the Author / Sirish Raghuram

Sirish Raghuram is co-founder & CEO of Platform9. As CEO, Sirish quarterbacks the team at Platform9 in realizing the company’s vision, which, on a given day, can range from strategy and road map to shipping code and running tests. Before co-founding Platform9, Sirish was an early engineer at VMware, where he held technical and management leadership roles and helped ship multiple vSphere products. Sirish’s work was reflected in several patents, VMworld talks and internal product innovations within VMware Engineering. Sirish has a B.E. in Computer Science from the University of Pune, India. Connect with him on LinkedIn.
October 25, 2016
What Can Digital Transformation Do For You?
Contrary to what the massive articles around digital transformation suggest, your business lines’ primary concern isn’t the amazing technology innovations that pop up every day but how well they’re going to deal with your industry’s disruption. Something is sure, your IT organization’s digital transformation cannot be narrowed to technology changes; that’s a simplistic vision which ignores the disruptive impacts on businesses, particularly on the IT function.

Howard King, writing in “What is Digital Transformation?” published in The Guardian, makes it clear, “Transformation is a whole scale change to the foundational components of a business: from its operating model to its infrastructure. What it sells, to whom and how it goes to market.” He’s right, the business-oriented vision of IT demands a transformation of your IT organization including its infrastructure, applications delivery approach and operating model, and DevOps increasingly is cited as the foundation of the new IT.

This article shares my experience and lessons learned implementing DevOps as the foundation of digital IT organizations.

Top Priority: Dealing with Your Industry Disruption

The belief that implementing a continuous delivery infrastructure is all your business need to survive digital disruptions is a marketing hype DevOps practitioners have grown accustomed to. It’s a myth that prevents you from making informed digital transformation decisions. Emphasizing the technology at the expense of your industry’s disruption forces you to make technology change prematurely; your priority actually should be to figure out how to deal with the disruptions hitting your industry.

Ask yourself this: What’s the interest in investing in scattered continuous delivery infrastructure implementations, when your favorite vendors and consultants aren’t able to clearly address these concerns:
- How can you restructure your business value stream to make it lean and agile so that the right services are timely delivered to the right market segments?
- How can you integrate DevOps, IT as a service, cloud, Internet of Things and big data into a coherent and consistent business capability so that the right services are timely delivered to the right market segments?
- What’s the architecture of your future information system, including your virtual data center and application delivery infrastructure?
- What will your role be as the CIO? How will the relationships with your business lines and vendors be managed?
Make no mistake about it, these are your business line concerns of the moment. Unless you address them, Chef, Puppet, CodeDeploy or any other software application alone won’t help to keep up the pace imposed by Uber, Apple, Google and the like. You’ll need to bring together your business and IT staff in a lean and agile capability, which demands a re-engineering of your IT structure, processes, practices and governance.

DevOps is the Foundation of Your Digital IT Organization

Let’s remove the ambiguity: Infrastructure as code isn’t DevOps. The increasing confusion of DevOps with continuous delivery infrastructure is slowly but surely killing it—it’s even disqualifying it as a competitive advantage. That vision focused on tools has nothing to do with the logic underneath DevOps.

As I’ve been making clients happy with it, DevOps is primarily an application delivery model that seeks to make businesses responsive to market opportunities through accelerating added-value applications delivery and with the goal of continuously generating value. It’s based on three principles helping businesses survive digital disruptions, then grow and prosper:
- Using continuous delivering infrastructure to speed application delivery doesn’t make sense unless tangible business benefits are generated.
- Applications are the means by which value is brought to customers and business benefits are brought in.
- Agile principles across your software development life cycle makes your value stream lean and agile which in turn results in continuous value.
DevOps Delivery Model – (Credit: Philippe Abdoulaye – ITaaS Now LLC)

The delivery model uses three levers to implement the above principles and establish DevOps as the foundation of your digital IT, they include:
- The rules of the game, which are agreed-upon principles, practices, roles and responsibilities and tools institutionalized to ensure agility and accelerated applications delivery;
- The operating model, which is the lean and agile work environment resulting from the application of the rules of the game; and
- The continuous delivery infrastructure, which is a platform that leverages infrastructure-as-code solutions to accelerate applications development and deployment.
The Digital IT Organization: Approach, Principles, Tools

Success in IT organization transformations often come more from a stream of small wins than from a one-time flood. That’s what I call incremental transformation, illustrated in the following digital five-stage transformation journey:

The transformation journey – (Credit: Philippe Abdoulaye – ITaaS Now LLC)

The following recommendations definitely will make you instrumental to your IT transformation and help your business meet the digital economy’s competitive challenges.

Strategize

A CIO who’s expected to help business lines survive their industry’s disruption, grow and prosper first must clarify their business objectives; estimate potential gains, losses, and risks; and create a consensus on the DevOps delivery model to implement in terms of high-level architecture and implementation timing. That’s the purpose of the Strategize stage.

Never skip this stage—it’s where the executive buy-in is obtained and the vision of the future IT capability is developed. Make sure the following is accomplished:
- Creating an executive task force co-led by the CIO and a business representative;
- Scheduling one to three workshops addressing your transformation business case, transformation road map, high-level blueprint of the future IT capability and the DevOps adoption plan;
- Designating a project manager; and
- Communication by task force members regarding the whys and wherefores of the transformation initiative.
Plan

Digital transformation involves many functions within your business in the form of a cross-functional team: the concerned business and IT staff. Your responsibility as the CIO is to bring the voice of the business across the transformation journey, from business objectives clarification, to design and implementation, to pilot experiment, to deployment.

Again, do not overlook this stage—it’s about translating the effort into team structure, implementation schedules and operational and strategic reviews. It’s where the risks likely to hamper projected objectives are anticipated. Make sure workshops involving the business and IT are set up to address the following:
- Team structure and roles and responsibilities;
- Transformation road map steps translated into operational timelines associated along with expected outcomes; and
- The project’s operational and strategic reviews, including a risk management plan.
Design and Implementation

The third stage is when the three dimensions of the digital IT organization are designed and implemented. They include the DevOps rules of the game, the operating model and the continuous delivery infrastructure.

Never improvise this stage—use a proven approach to address the dimensions. Favor a collaborative approach that includes workshops or brainstorming sessions with the concerned business and IT staff.

In the rules of the game brainstorming sessions, make sure the following is addressed:
- Organization and operational dysfunctions preventing cross-functional collaboration, rapid problem-solving and decision-making;
- Recommendations to the dysfunctions in terms of agile processes, practices, roles and responsibilities and tools;
- Recommendations mapping to the appropriate DevOps principles, practices, roles and responsibilities and governance structure; and
- The IT organization’s DevOps charter, thought to accelerate the rules of the game and the operating model adoption.
In the continuous delivery infrastructure workshops make sure the following is addressed:
- A delivery pipeline, including steps and for each of them what follows:
  - Required staff, skills and background
  - Implementation, testing and deployment agile processes and practices
  - Implementation, testing and deployment tools, such as Chef, Jenkins or CodePipeline
Experiment

For this stage, set up a pilot experiment involving, in a small-scale real-business setting, a panel of business and IT staff to evaluate the likelihood of the project benefits.

Neither skip nor overlook that step, it’s where improvements are identified. Throughout the pilot project, make sure the following is in place:
- Executive task force monitoring the pilot project through bi-weekly reviews;
- Concerned business lines and IT business lines are involved;
- Training campaigns are organized around the DevOps charter, the operating model and the continuous delivery infrastructure;
- Strategic management tools, such as balanced scorecards to monitor progress, risks and achievements; and
- Measurement of the impacts of the operating model and the continuous delivery infrastructure against the projected benefits.
Deploy

This final stage is a Go-No-Go milestone in which your task force, based on the pilot’s outcome, decides whether to deploy your digital IT capability at the scale of the company. Throughout the deployment:
- Capitalize on the benefits generated by the pilot experiment to launch an information campaign emphasizing the benefits for the company and for the staff; and
- Train business and IT staff on the DevOps charter, the operating model and the continuous delivery infrastructure.
Wrapping up

Today’s DevOps practices focused on continuous delivery infrastructure aren’t likely to help businesses grow and prosper in the digital economy. Thought leader Joe McKendrick in, “The road to digital bliss is paved with service thinking and DevOps,” confirms that fact.

The approach discussed in this article are detailed in my recent book, written to help make IT leaders instrumental to their business digital transformation:

If you’re considering a DevOps-focused IT transformation, what will be your first steps? I’d love to hear your thoughts, let’s meet @TheFutureOfYourIT.
June 13, 2016
Echidna Takes DevOps to E-Commerce Site Creation

The Echidna e-commerce firm installs and evolves e-commerce platforms, providing ongoing development so its clients can leverage e-commerce experiences to the satisfaction of their customers. “We apply carefully selected user experience technology, marketing and analytics abilities to our clients’ strategic goals to do this,” taking what it calls an “anti-Big Bang” approach to continuous deployment, says Mike Pierce, CTO of Echidna.

“Traditional e-commerce deployments were often Big Bang deployments that people developed on the basis that they had to pressure the business leaders into feeling like they had only one chance to pack as much scope as possible into a large software development project. That approach voids any practical innovation,” says Pierce. With that mindset, it can take years to finish e-commerce sites, and e-commerce companies can go out of business by the time they actually finish such a site, since it would be far out of date by then.

Echidna sees continuous deployment as one piece in a much larger agility play designed to ensure that businesses compete in digital economies, now and in the future. Continuous deployment is the ultimate goal of DevOps, an approach that enables the business to innovate fast enough to maintain the business’ sharp marketplace edge through acute, turn-on-a-dime competitive dexterity. “In order to do this, continuous deployment helps companies learn to develop high-quality software and deploy it on a rapid, reliable and repeatable schedule,” says Pierce.

Continuous Deployment Timelines and Challenges

Depending on the client, its needs and the features in the pipeline, Echidna deploys software to each e-commerce platform from monthly to daily, and at intervals in between those extremes. Echidna seeks to increase the degree of automation of each deployment over time. The challenge is that with so many different clients and platforms, Echidna must achieve its automation victories for each customer individually, solving similar and unique issues. “We are finding our greatest success using PaaS (platform-as-a-service) e-commerce platforms like Four51 because it eliminates a lot of the administrative overhead that is typical with deployments to on-premise platforms,” says Pierce.

The development model Echidna follows to deploy e-commerce sites affects each software component, even though components deploy at different times. “Payment system modifications require significantly different testing—more testing—and we must deploy them much less often than other components. Marketing components don’t need as much regression testing as other software modules and so we can deploy them much more quickly,” he says.

The most difficult software to move into continuous deployment in an e-commerce platform are the components that integrate deeply into enterprise systems. That is why Echidna advises business customers to use DevOps methodologies to deploy their enterprise production software at accelerated rates. When they do this, Echidna potentially can speed its deployment of the e-commerce platform components that are enmeshed with those production systems.

The Four51 Path, Through Jitterbit

Echidna connects Four51’s OrderCloud to the Jitterbit iPaaS integration server platform, a point-and-click integration tool. “This enables us to streamline further integrations and data exchanges with many of the disparate systems in an ecosystem,” says Pierce. Echidna sees increasing continuous deployment orchestration opportunities as more and more third-party application vendors become compatible with these kinds of integration tools.

Four51 enables an ecosystem of connected digital services and applications where e-commerce uses numerous disparate systems that exchange data and facilitate work streams in the enterprise in question, Pierce says. “As these systems eliminate disparities and we align them on the central integration and deployment standards that we use, there will be increasing opportunities to introduce clients to lean development practices like continuous delivery and continuous deployment.”

Conclusion

Continuous deployment is miles apart from what information technology is used to, especially where manual change management and change control are concerned. But once Echidna exposes its customers to continuous deployment, their managers, C-levels and boards soon begin to witness the accelerated software innovation continuous deployment provides and how this makes information technology—and the business—more nimble, both technologically and competitively.

May 25, 2016
Continuous Delivery and Deployment: A Tale of Two Models

EnerNOC (Energy Network Operations Center) is an energy intelligence software provider with more than 1,300 employees in its Boston headquarters and international locations. The public firm helps global customers to efficiently consume energy. “Our mission is to change how the world uses energy, helping customers to improve profitability and visibility for better decision-making and sustainability by focusing on how they buy energy and when and how they use it,” says James Nichols, principal engineer and cloud architect at EnerNOC.

A regular user of Splunk (the company uses other DevOps tools and platforms as well, including AWS and Oracle), EnerNOC has fully automated monitoring in its perpetual drive to move past continuous delivery and on to continuous deployment.

Embedded in EnerNOC’s story of why it uses continuous development and continuous deployment models is a chapter on its move to fully automated monitoring.

EnerNOC DevOps Models

EnerNOC’s DevOps model is in constant transition toward full automation. Currently, it has fully automated deployment, fully automated monitoring and semi-automated quality assurance (QA), which pull the company’s continuous development approaches.

A change to “builds on commit” of smaller applications within the legacy software monolith at EnerNOC came hand in hand with further automation, targeting what the company calls “trustworthy continuous deployment,” Nichols says. That was the crux of EnerNOC extending itself into deployment automation.

Wherever the model is not fully automated and continuous in deployment, it remains continuous delivery. “Most of the variance between the approaches is due to aspects of the delivery pipeline that still have manual processes; these parts are usually comprised of the remaining manual testing procedures,” says Nichols.

Change Control Necessitates Two Models

EnerNOC develops and maintains mission-critical software that must pass through a change management approval process. This process includes communications within and across the company and customers, as well as user acceptance testing. This is where the manual testing comes in. Only at the end of this approval pipeline can EnerNOC pass the software baton into the hands of production.

Due to EnerNOC’s tightly yoked systems and the manual testing, change control remains, along with a mix of continuous development and semi-automation with fully automated continuous deployment. “DevOps is a cultural movement as well as a technical one, and so the existing model could evolve with the organization’s quality and maturity evolutions,” says Nichols. If the manual testing and the relationships between systems give way to more fully automated testing and more amenable relationships, QA could more closely embrace full automation.

Monitoring Challenges/Achievements

EnerNOC’s application diversity includes hybrid software with a foothold in the Amazon cloud, in EnerNOC’s data centers and on-premises. “We use three tiers on top of Oracle in the data center. In AWS, we use a combination of big data, cloud-friendly and serverless technologies,” says Nichols. Business transaction data traverses many machines that belong to service groups’ stakeholders. “The distributed, decentralized architecture makes troubleshooting, performance management, security operations and production operations challenging,” notes Nichols. Better monitoring was the answer.

EnerNOC wanted to offer nontechnical users easy access to systems-level monitoring data with transparency into the many systems through a single pane-of-glass dashboard with readily available system notices and troubleshooting abilities. “We would use Splunk to do this and then feedback everything we learned from using Splunk into driving our DevOps and engineering processes into the future,” he says.

The EnerNOC DevOps performance team built the dashboard that accompanied the Splunk service, ensuring it would register the performance metrics that are of value to the organization. The performance, product management and development teams maintain an awareness of the metrics and receive the related alerts. “The development team incorporates the ongoing metrics early in the development life cycle to improve application performance,” says Nichols.

EnerNOC first applied Splunk to web logs for operational capacity planning, then used Splunk to give the Ops team transparency into the function of its new real-time data product that was working in production. As new products came on board, the organization added them to Splunk to provide Ops teams with dashboards, at the urgent request of Ops team members.

Current Status: All Systems Good

EnerNOC applies Splunk to test and production systems for views into consistency and acceleration. “Splunk is our primary alerting infrastructure for low-level system metrics like CPU utilization as well as for complex business transactions that span multiple systems,” says Nichols. Splunk has eyes on application health, cluster state across machines, expected systems tasks and suspect or known-bad system behavior, as well as the presence or lack of any significant event.

May 20, 2016
Faster, Smarter DevOps

Whether you call it DevOps or not, if you are concerned about releasing more code faster and at a higher quality, the resulting software delivery chain and process will look and smell like DevOps. But for existing development teams, no matter what the velocity objective is, getting from here to there is not something that can be done without a plan.

Moving your release cadence from months to weeks is not just about learning agile practices and getting some automation tools. It involves people, tooling and a transition plan. I will discuss some of the benefits and approaches to getting there.

Waterfall to agile, agile to continuous integration, continuous integration to continuous deployment. Whatever your processes are, the theme is the same: Find a way to get code to users faster without sacrificing quality. But speed and quality are sometimes at opposition to each other. Going faster means things can break faster, and when we only think about DevOps as releases, it’s easy to fall into this trap.

Established development shops cannot just jump from one flow to another. Unless you start out net new, the goal is to introduce new processes without delaying releases for three months or more to do transition in lump. This often is done using a pincer approach that addresses both bottom-up tactics and top-down oversight and culture at the same time.

However, because adopting DevOps tools is so easy, the trend is to focus on tactics only and adopt from the bottom up without consideration of the entire pipeline. The outcome is release automation tools that dictate your delivery chain for you, and not the other way around.

Structured Automation

DevOps requires automation. But what is often not considered is automation that sustains and fits into the entire delivery chain. Governance, artifacts organization and inventory, metrics and security must be considered. If an organization establishes a vetting process for all new automation and how it fits into the pipeline’s orchestration, then new automation will support what exists today and in the future.

For example, many organizations driving down the DevOps path have encountered challenges when trying to incorporate practices from security or governance teams. Historically, these teams have resided outside of the Dev and Ops echo chamber and their processes were asynchronously aligned to the work being done. The challenge for many organizations is to determine the best ways to bring the people, processes and technology supporting these initiatives into the fold without slowing things down. The best organizations are finding new ways to automate policies from security and governance teams by shifting away from artisanal, asynchronous approaches to synchronous processes earlier in the life cycle.

Let’s take a look at an example of application security. A number of technology vendors in the application security arena are touting “automation” as key value point for their solutions to better fit them into a DevOps tool chain. In some instances, automation means that machines are now fully responsible for monitoring, analyzing and fixing security vulnerabilities for those applications at wire-speed. In other instances, automation implies facilitation of human-centric workflows that might represent hours or days of asynchronous analysis not fit for continuous operations. In both cases, the technologies may accomplish similar ends, but their approaches could be dramatically different.

Also, one solution might be built to support asynchronous investigations by a security professional, while the other might provide synchronous support to a developer at the design and build stages of the software development life cycle. Establishing a vetting process can help determine if the automation levels required by a team or process can truly be delivered before investments are made. It is also worth noting that layers of obscurity frequently exist within words such as “automation,” “integration,” “configuration,” and “continuous.”

There are other areas that also tend to get overlooked in building a DevOps environment. The complete story can be found here.

May 12, 2016
Trust, You Must: 8 Tips for Adopting CI/CD

We have done continuous deployment (CD) for four years, and enable many of our customers to do continuous deployment with quality assurance (QA). We’ve helped tons of companies adopt and optimize their CD setups. We’ve found that continuous deployment is all about trust — augmented with processes and infrastructure.

To that point, here are some common tips for adopting continuous integration (CI)/CD that I find myself sharing frequently:

Use a CI Tool

This sounds obvious, but you’d be surprised at how many companies try and roll their own. Don’t. There’s no upside for your company—and infinite downside. We love Circle, but there are plenty of other great options. Most are at least somewhat prescriptive when it comes to workflow, which is important when you’re getting started.

Do Unit Testing

It’s always worth reminding everyone that continuous deployment is simply continuous integration applied to deployments. To do continuous integration, you need good unit test coverage. If you haven’t got a solid base of unit testing and continuous integration already, you’re not ready for continuous deployment.

Have Solid Production Monitoring

Bugs and regressions will get shipped; it’s inevitable. Put a system in place, by looking at the data in production, to tell you when a regression or bug was shipped. Tie this into automated rollbacks, so that if a significant performance or key performance indicator (KPI) regression (i.e., signups per minute) appears, your CD system rolls back to the most recent stable version while your team investigates.

Trust Your Team

We talked about the infrastructure requirements, but if you don’t have trust, then continuous deployment isn’t going to work for you. Trusting your team is about giving your developers the freedom to make mistakes, deploy as they see fit and check each others’ code. It is possible to combine continuous deployment with a culture of hierarchical permissions, but we have yet to see that truly work.

Remove Red Tape from the Code Review Process

Your team should review code changes, typically through a pull request, which goes hand in hand with trusting your team. Developers should be trusted to choose the most qualified and insightful person to review their code. One of the most common mistakes among teams trying to adopt continuous deployment is that they only let one or two “senior” developers review the rest of the team’s code, “bottlenecking” all releases. Don’t make this mistake.

Involve Developers Closely in Production Operations

The most common mistake we notice among companies that don’t make a successful transition to continuous deployment is that their development organizations are siloed. Dev and Ops should be joined at the hip. Have your developers deeply involved in the setup and definition of your CD infrastructure, and half the battle is already won.

Shift Left: Test Early and Often

Your team needs constant feedback. Think of your QA objective as getting the right feedback in the right hands at the right time. When it comes to deployments, this means knowing what’s working and what’s broken as the breaking change is made. The sooner you uncover the bug after it’s created, the easier it is to fix. The best companies doing continuous deployment have comprehensive unit test and integration test coverage that runs before any code hits the staging server.

Treat Staging as Your Holy Place

The final pre-production environment is usually known as ‘staging.’ A common mistake is to have developers ship new code direct to staging, while figuring out if the new code is ready to go live. This is a terrible idea. As soon as more than one developer is trying to ship their changes, your Slack channel will blow up with cussing and recriminations. Isolate the pre-production environments to each project, and enforce the “staging as ready to go live” philosophy. This way, integration testing can happen at a self-defined pace, without anyone else’s release being blocked.

About the Author / Fred Stevens-Smith

Fred Stevens-Smith is Co-founder and CEO of Rainforest QA, a quality assurance-as-a-service startup backed by Storm Ventures, Rincon, Andreessen Horowitz, Morado and Crosslink.

May 5, 2016