Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • When Technology Doesn’t Matter

    When Technology Doesn’t Matter

    Over 10 years ago, Nick Carr wrote an article in the Harvard Business Review with the title, “IT Doesn’t Matter.” As you might expect, an article with a title like that generated a significant amount of controversy. CEOs from leading technology companies and other writers at the time panned Carr’s premise in a way that, in retrospect, seems a bit overly defensive.

    His point, after all, was not that IT was not unimportant; rather, IT (as vaguely defined in the article) was so ubiquitous that it was no longer a competitive advantage. Rather, it was what companies did with their IT technologies that provided competitive advantage. It’s also worth noting that his (seemingly) intentionally vague definition appeared to focus on the generic notion of IT as PCs, servers and things that had become commonplace at the time.

    This has been a common pattern throughout the history of technology: Early adopters enjoy a competitive advantage, but only as long as the technology they’ve adopted remains scarcely used. Any advantage goes away when the technology becomes ubiquitous. The technology remains important—even critical—to a business’ operations, but it’s hardly an advantage when everyone else has the similar technology.

    Carr’s article uses the example of electricity: In the late 19th century, when electricity was an emerging and scarce technology, factories that were able to leverage electricity for their manufacturing efforts held a significant advantage. Today, no factory can operate without electricity, yet it’s hardly considered a strategic asset.

    Today in the 21st century, we’re seeing a similar trend repeated, but on much compressed time scale. Providing free Wi-Fi was once a lure for customers of restaurants, coffee shops and the like, but today it’s essentially table stakes. Likewise, leveraging clouds was once a competitive advantage, but with more than 90 percent of companies using some form of public or private clouds, one would be hard-pressed to argue that cloud usage alone provides any significant advantage. However, how you use these clouds can, in fact, become a competitive advantage. CA offers a variety of tools to help you manage, secure and accelerate your use of clouds.

    It’s an important lesson to learn: In the application economy, early adopters enjoy an important—yet relatively short-lived— competitive advantage. Knowing when to jump on an emerging technology can be the difference between surviving and thriving.

    Today, there are several technologies that are on the cusp and moving quickly to adopt them can provide a significant competitive advantage. The ability to deliver software continuously—while not exactly a single technology—is on this cusp. Continuous delivery is a core component of a successful DevOps implementation, and means essentially what it sounds like: the ability to deliver software from planning to production in a near constant and continuous fashion. Achieving true continuous delivery means you must test continuously, develop continuously and release continuously.

    Continuous delivery is not easy to achieve, but for those who do the advantage is significant. Companies implementing continuous delivery can get to market 20x faster with 98 percent fewer production errors. That’s a powerful benefit.

    To learn more about CA’s portfolio of continuous delivery solutions—and hear directly from several customers—you can check out the replay of Jeff Scheaffer’s keynote presentation from CAWorld 2016.

    — Brendan Hayes

  • Webinar:  Using APM to Turbocharge Continuous Integration

    Webinar: Using APM to Turbocharge Continuous Integration

    Continuous Integration and Continuous Delivery have dramatically accelerated the rate of innovation within both large and small organizations – allowing teams to build and deploy software faster than ever before.

    Yet, as release automation accelerates the volume, variety and velocity of application changes, an entirely new set of challenges have appeared. Let’s state the obvious here – DevOps, empowered by continuous integration and delivery, doesn’t merely end at deployment. In fact, the fun is just getting started.

    Part of the mission for today’s IT monitoring teams relates to tracking build numbers, status, success, failure and the rest. Yet, if 80% of application performance problems can be tied to some environmental change, clearly, there’s more to the story.

    In this webinar we’ll discuss these issues and more, along with some of the relevant capabilities supported via the integration of CA Technologies Application Performance Management and the CloudBees Jenkins automation platform. Join us to hear first-hand how tying together APM and release automation can help identify and remediate changes that impact applications performance, turbocharging the software pipeline.

     

    Recording

    — Jules Louis

  • Interview w/ Aruna Ravichandran, CA

    Interview w/ Aruna Ravichandran, CA

    Always a great interview, Aruna Ravichandran of CA is one of my own personal heroes. She is woman who has accomplished so much in her career. Having written literally a million lines of code in products before continuing her education with advanced degrees and moving to the business side of the house, Aruna is someone every women and every young person can look at as a role model to aspire to.

    Besides talking about CA here we have a chance to hear from Aruna about her own professional journey and how she is helping her own daughter and other young women to pursue careers in engineering and technology. Her new book is coming out soon and she is donating the profits from it to Girls Who Code.

    This is a long clip but well worth listening to all of it. Great interview.

    — Jules Louis

  • Interview with Jeff Scheaffer, SVP and GM Continuous Delivery and DevOps Business Unit at CA Technologies

    Interview with Jeff Scheaffer, SVP and GM Continuous Delivery and DevOps Business Unit at CA Technologies

    At Jenkins World 2016, devopsy.kinsta.cloud caught up with Jeff Scheaffer, SVP and GM Continuous Delivery and DevOps Business Unit at CA Technologies. Jeff spoke to us about the importance of CD and DevOps at CA. Jeff also I thought gave us the best explanation and reasoning for DevOps Express. With Jeff’s leadership, we should see more on DevOps Express soon.

     

    — Jules Louis

  • Welcome to the CD Neighborhood

    Welcome to the CD Neighborhood

    Today we opened a new neighborhood or section here on devopsy.kinsta.cloud. It is called the CD neighborhood and it will feature content on CD and related content. The neighborhood is sponsored by CA Technologies, who will also contribute content to the new neighborhood.

    CA authored content will be clearly marked. CA employees have contributed by lined articles to devopsy.kinsta.cloud for a long time and they have some great DevOps thought leaders. Recently they have also acquired Blazemeter and just announced the intent to acquire Automic as well. This should mean even more great content from them.

    But CA is not the only content provider for the new CD neighborhood. The devopsy.kinsta.cloud staff writers will be contributing to the neighborhood on a regular basis. In addition, contributed articles that are on theme will also be posted here on a near daily basis. In fact I encourage you to submit your thoughts on CD to me at editor@devops.com, if you have something to say on the subject.

    In addition to written articles, the CD neighborhood will also host both audio and video content that is on point. Many will come from our DevOpsTV YouTube channel and DevOps Chats hosted on SoundCloud.

    With CD being such an important part of DevOps we think that the CD neighborhood is going to be a favorite destination for many of you. We will do our best to keep it fresh, informative and useful!

    — Alan Shimel

  • Continuous Integration vs. Delivery vs. Deployment: What’s the Difference?

    Continuous Integration vs. Delivery vs. Deployment: What’s the Difference?

    “Continuous” is one of the most popular words in the DevOps lexicon. What does it really mean, and what is the difference between continuous delivery, continuous deployment and continuous integration? Keep reading for some perspective.

    Like many technological concepts, continuous integration, continuous delivery and continuous deployment are terms that DevOps teams tend to use without defining them precisely. For that reason, it’s worth taking a closer look at what each of these terms actually means.

    Continuous Integration

    Continuous integration can really mean one of two things, depending on the context.

    In the narrower sense, continuous integration refers to the work done by a continuous integration server, such as Jenkins or Bamboo. A continuous integration server automatically tests code written by individual developers to make sure it can be merged into the main code base. By automating this process, continuous integration servers help developers to write and test small chunks of code on an ongoing basis, which minimizes the risk that a code change could cause serious problems and force developers to revert their application to an earlier state.

    In a broader sense, continuous integration means the constant integration of changes to an application at all stages of the delivery chain. This includes but is not limited to the automated integration testing and code merging performed by an integration server. In this broader definition, continuous integration can include the integration of new design concepts to the delivery pipeline, for example. It also can involve other types of software testing beyond just automated integration tests.

    Continuous Delivery

    In most definitions, continuous delivery refers to the process of delivering software updates to users on a nearly constant basis. Continuous delivery is made possible by continuous integration and other optimizations at earlier stages of the development process. But the definition of continuous delivery gets a little cloudy when you start comparing it to continuous deployment.

    Continuous Deployment

    What’s the difference between continuous delivery and continuous deployment? For some DevOps teams, nothing. People tend to use these terms interchangeably. In fact, they are usually simply abbreviated as “CD,” leaving it up to the listener to decide what the speaker is actually talking about when he hears the term during a conversation about DevOps.

    Some developers draw a distinction between continuous delivery and continuous deployment, however. For example, Mirco Hering writes that continuous delivery requires that the DevOps team manually release updates to users. In contrast, the continuous deployment pipeline is fully automated; users get updates as soon as they are written and tested, with no manual intervention by developers.

    This distinction is a good way to think about the difference between the closely related concepts of continuous delivery and continuous deployment. It’s a distinction that will become increasingly important as technologies such as Docker make it easier to automate application deployment by allowing DevOps teams to drop new container images into production.

    What Does Continuous Mean, Anyway?

    It’s worth mentioning, too, that the word “continuous” is usually something of an exaggeration when used by DevOps folks. It implies that software changes roll down the pipeline in a totally constant, never-stopping fashion.

    That is an exaggeration. Integration, delivery and deployment are almost never completely continuous. In practice, a “continuously” integrated application is likely to be rebuilt and delivered something like every 24 hours, not every single time a code change reaches the end of the pipe.

    That rhythm is still a lot faster, and closer to continuous, than the “waterfall” delivery pipelines that were common before the introduction of DevOps. But it’s still important to understand that, even with all of the automation tools available today, software development is rarely completely continuous.

    — Chris Tozzi

  • 7 Best Practices for Continuous Delivery Success

    7 Best Practices for Continuous Delivery Success

    Continuous delivery refers to the development of software in short and ongoing build-test-release cycles along the deployment pipeline. Repeated testing of processes and scripts before deployment to production means most errors are discovered early on and have minimal impact. Plus, finding and fixing such errors is much easier with fewer changes per release—software is released faster and more frequently with fewer deployment problems due to a heavy focus on visibility, instant feedback and incremental changes.

    To accomplish this, companies must have a straightforward and repeatable deployment process. Most reliable, of course, is an automated deployment pipeline that can feed into any environment.

    We have identified seven continuous delivery best practices that are supported by a set of complementary task-specific tools, in what’s known as the “DevOps toolchain,” to automate the process from end to end and minimize human error.

    Version-Control Everything

    The native code deployment process generally includes a built-in safety net, preventing out-of-process, locally generated artifacts from entering the production environment. Every change a developer makes must be documented in the source control repository or it is not included in the build process. If a developer copies an artifact generated locally to a test environment, the next deployment will override this out-of-process change.

    This insistence on having a single source of truth—and a reliable one—creates a stable foundation for all development processes. However, the processes are only as strong as their weakest link. That’s why the primary golden rule for effective continuous delivery is to version-control everything. What works well for code also will preserve the integrity of configuration, scripts, databases, website html and even documentation.

    Build Binaries Once

    Not every organization’s build process is the same or uses the same tools, but every build process in a single organization must be completely consistent. Whether the build is a single file deployed to an automated test environment or a complex build with several different possible deployment versions, each build version should happen exactly the same way and result in unique binary artifacts.

    This one-time-only compiling eliminates the risk of untracked differences due to various deployment environments, third-party libraries or different compilation contexts or configurations that will result in an unstable or unpredictable release. Save the compilation phase output (the binaries) to a binary repository, from which the deployment process can retrieve the relevant artifacts.

    Deploy the Same Way Every Time

    An inconsistent deployment process can become a source of configuration drift across environments, especially with the rapid releases common in a continuous delivery system. As a result, valuable time and effort are wasted in identifying and addressing difficulties arising from the application working in one environment and not another.

    For a reliable process in all environments, then, the same set of steps must be repeated from start to finish. This ensures the same results in lower environments, with more frequent deployments (such as integration, QA, etc.), as in higher environments (pre-production and production), with fewer deployments.

    Smoke Test

    Smoke testing deployments is a very rapid way to make sure that the most crucial functions of a program work and will pass basic diagnostics. This non-exhaustive software testing (of all elements, such as services, database, messaging bus, external services, etc.) does not provide the same fine-grain comprehensiveness as full test suites; however, it can be run frequently and quickly, often in a matter minutes (rather than hours or days). This allows for a much quicker turnaround on what can become very time-consuming and basic issues.

    The faster the feedback loop works (especially with automated test tools such as Telerik, QTP, TestComplete), the higher the final product’s quality will be and the quicker it will reach its release-ready state.

    Deploy into a Production-like Environment

    Each new deployment (using, for example, IBM uDeploy, CA Release Automation, XebiaLabs, Automic) should be made into an environment that mimics as closely as possible the actual final production environment. This includes infrastructure, operating system, databases, patches, network topology, firewalls and configuration. By validating software changes in this type of detailed pre-production environment, mismatches and last-minute surprises can be effectively eliminated and applications can be released safely to production at any time.

    Instant Propagation

    Continuous delivery by definition must be an ongoing process of building, testing and releasing. Every check-in to a source control repository should trigger compiling (if needed) and packaging by a build server. This, in turn, should automatically initiate certain predefined testing until the software can either be marked as releasable or returned to development.

    Continuous integration tools (e.g., Jenkins / CloudBees, Bamboo, TeamCity) ensure that cascade of actions along the delivery pipeline is consistent, automatic and instant. Thus, development teams merge developed code and get feedback from automated test tools many times a day, for a more efficient deployment and update process.

    Include the Database

    No list of continuous delivery best practices would be complete without advising that the database be managed using the same basic protocols that ensure secure and reliable source code, task, configuration, build and deployment management. However, many times this is neglected because the database is unlike other software components or compiled code that are easily copied from development to testing to production. Yet, the database is actually a repository of the most valued and irreplaceable asset—our data—and preserving it accurately is imperative to continuous delivery.

    Although the database poses several unique challenges, specialized database tools such as enforced database source control (for all environments), a database build automation tool and database release and verification processes can ensure a stable resource in your DevOps chain.

    Considering these seven continuous delivery best practices will allow you to create a deployment pipeline with increased productivity, faster time to market, reduced risk and increased quality.

    The key to ensuring a quick, robust and accurately repeatable continuous delivery is, as we have seen, automation, deployment consistency, comprehensive testing and database version control. With these fail-safes in place, pushing the “release to production” button can be done much more often and with much more confidence.

    About the Author / Yaniv Yehuda

    yaniv-e1404303491389Yaniv Yehuda is the co-founder and CTO of DBmaestro, an enterprise software development company focusing on database development and deployment technologies. Yaniv is a DevOps expert who spent the last couple of years raising awareness about the challenges surrounding database development and deployment, and how to support database continuous delivery. Connect with him on LinkedIn and Twitter.

     

  • Why PR, Legal and HR Matter for Continuous Delivery

    Why PR, Legal and HR Matter for Continuous Delivery

    By most definitions, DevOps involves making developers and IT Ops teams work better together. But integrating other parts of an organization, such as legal and public relations, into DevOps is also important. Here’s why.

    DevOps has become popular because it helps to speed the delivery of software. The goal is to reach continuous delivery, which means code is changed, tested, staged and delivered to users on a (nearly) continuous basis.

    If you think only from a technical perspective, then getting to continuous delivery of software involves only technical people. Developers, IT Ops admins and software testers are all you need.

    DevOps in the Big Picture

    Yet, to deliver software continuously on a reliable basis, you need to think beyond just technical processes and personnel. Effective DevOps workflows have to involve all of the other parts of the organization that play a role in creating software, delivering it to users and maintaining it once it is in production.

    For example, PR teams need to be plugged into the software delivery chain so they know when new features are coming down the pipeline and can prepare to promote them. Legal professionals should play a role as well, since they’ll know when code changes require tweaks to licensing, compliance procedures and so on.

    The list can go on. In most organizations, human resources has a role to play in DevOps, since personnel changes can affect the continuous delivery chain. Supply chain managers should be involved, too, since they have a hand in provisioning the infrastructure on which DevOps teams rely.

    To be sure, these various non-technical teams don’t need to be as plugged into the DevOps workflow as deeply developers and admins do. Code can be written and tested without their perpetual involvement. Still, at a minimum, all parts of the organization should at least be aware of the state of the delivery chain. Otherwise, visibility and communication break down.

    Taking this big-picture approach to DevOps is important because ultimately, despite the relatively narrow purview implied by the term DevOps itself, the value of DevOps lies in adding efficiency and promoting better communication across the entire organization in ways that optimize software delivery.

    So, if you want to make the most of DevOps, start thinking beyond just Dev and Ops.

    — Chris Tozzi

  • 7 Strategies for Continuous Delivery and Deployment Success

    7 Strategies for Continuous Delivery and Deployment Success

    One of the key advantages of cloud-native applications is the ability to quickly push updates into production for specific microservices—giving developers the ability to respond rapidly to changing business requirements without having to upgrade the entire application. Successfully doing this requires far more than just leveraging a continuous delivery platform. Everything from the culture of the development group to the type of testing required to the underlying platform it will run on is critical.

    Based on applying DevOps principles across several projects to date, I have found that development teams can benefit from following these seven key strategies before launching continuous delivery and deployment initiatives. By doing so, you’ll increase your opportunity for success.

    Culture

    Just having the right tools and technology isn’t enough in today’s enterprises. Successfully adopting an agile, continuous delivery and deployment process will require a culture shift within most development shops. Developers are no longer responsible only for their code; they must take full ownership of the development, testing, automation tools and, most importantly, building operational capabilities into the pipeline so the move to production is easier. Getting developers and the operations team on board with this may take some time and training, but the effort is critical, and your projects won’t succeed without this.

    Automation Platform

    Carefully consider the tool you’ll use for your continuous delivery pipeline because the current tools on the market have different optimal use cases. For example, if you need to do A/B testing you’ll want a tool that has capabilities for phasing the rollout based on the test results. If your application requires multiple updates in a short time span, you’ll need a tool that has advanced release management capabilities including good pipeline visualization and support for the management of cloud environments.

    App Architecture

    Design your applications so they are cloud-native. This is typically associated with microservices and containers and will enable you to independently and frequently update each service independently. The typical stages of a deployment pipeline are build, unit testing, integration testing (including functional testing), API testing, performance and staging, then, finally, on to production. However, with microservice architectures and advanced deployment techniques, more and more customers are bypassing certain stages and testing in production, albeit very contained production environments.

    Each stage also can include the creation of dynamic environments in the cloud, enabling the stage itself to be provisioned, executed and decommissioned as a part of the pipeline. If you choose to do this, look for a tool that has a number of built-in cloud plugins, so creating and destroying environments in AWS, Google Cloud and other clouds can be done easily.

    The application and infrastructure stack for each microservice also should be specified. This could include the application Jar file, a Docker file, or server image such as an AMI; any related services such as AWS Lambda functions; and all configurations and policies. These specifications all should be managed in an infrastructure-as-code manifest, which ensures versioning, consistency, change auditing and testing. It also ensures that the build resources, such as baked images, JARs and AWS Lambda functions, are consistently passed to each stage of the pipeline, from testing to production, eliminating the risk of imposing inconsistent environments and configurations. An AWS CloudFormation template is an example of an infrastructure-as-code manifest.

    Security

    Security is a necessity in the world of increasing threats that we live in; however, the common approach of waiting to test security right before the push to production is the wrong approach. Security should be built in to the pipeline itself and automated as part of the DevOps CI/CD process. It’s essential to validate the building blocks without slowing down the process. For example, leading companies are automating code scanning, template checking and other security tests in within the phases of the pipeline itself.

    Insight

    Visualizing and tracking pipelines, environment variables, test results, versions, issues and other artifacts allows developers and the operations team to share a common view throughout the project life cycle. This improves collaboration and traceability.

    Delivery Strategies

    Automating tests is critical to pipeline design and delivery. Without comprehensive, high-quality, fully automated tests of the deployment pipeline, production failures will occur. Code coverage no longer is a good representation of code quality. In the cloud-native world, test cases and test suites need to be mapped to functionality and business outcomes desired to ensure that the test team gives the green light and then everything goes red in production. In situations where time limitations prevent a comprehensive approach, first automate the critical capabilities and flows, then assess the cost-benefit of automating additional tests.

    Be sure that the data and environment for testing and automating purposes mirrors the production data and environment as closely as possible. Most pipelines have a “pre-production” stage before pushing software into production. You may consider building multiple stages across multiple AWS regions before pushing the software to production.

    If your goal is to update software in production very frequently and without any downtime, make sure your pipeline tool can easily manage multi-region deployments, such as blue/green and canary deployments for server groups and clusters across AWS and other public infrastructure-as-a-service (IaaS) clouds.

    Monitoring and Feedback

    Finally, it is essential to get continuous feedback on the user experience, response times, performance of key business services, and the technical metrics from all stages of the life cycle—development, QA and production. Only through this continuous feedback can you ensure that no degradation occurs as you push out releases with increasing frequency. This too needs to be built into each phase of the pipeline to understand performance and improve things earlier in the process, where it costs much less.

    There’s no doubt that a successful continuous development and deployment process can deliver significant benefits to users, who constantly require new and enhanced capabilities and can’t wait the weeks or months that a traditional development cycle requires By carefully considering these seven strategies, your projects will benefit from reduced application errors, less downtime and much happier users.

    About the Author / David Cramer

    david-cramerDavid Cramer is vice president of Product Management for the Cloud/DCA business unit at BMC. Prior to BMC, David was head of product management for CA Technologies. During his tenure at CA, David was responsible for application delivery, cloud management, virtualization and Infrastructure automation solutions. Before joining CA, David held executive positions at AlterPoint, Motive, NetSolve and Nortel Networks. David’s focus is on continuous delivery, cloud management and web-scale IT. David received his MBA from Southern Methodist University and a BS in Finance from Georgia State University. Connect with him on LinkedIn and Twitter.

  • Continuous Integration vs. Continuous Delivery: There’s an Important Difference

    Continuous Integration vs. Continuous Delivery: There’s an Important Difference

    If you read DevOps blogs today, it can be easy to assume that Continuous Integration (CI) and Continuous Delivery (CD) are synonymous terms. People often use them that way. But that’s a mistake.

    As an example of conflation of the terms CI and CD, take this page, which purports to be a “history of CI/CD tools.” The author does provide a truly excellent history of CI platforms.

    But CI tools are the only subject that the page discusses. It’s not a history of CI and CD. It’s just a history of CI platforms. To work the history of CD into the picture would require discussion of how software testing and delivery practices have changed over time, not just an overview of the history of CI software over the past several years.

    I don’t mean to knock the page. It’s a great read, and the mistake the author makes in treating CI and CD as synonymous is hardly rare. He or she is just repeating a common confusion. You see the same lack of distinction between CI and CD on Docker’s discussion of CI/CD, for instance.

    The Difference between CI and CD

    It’s unclear how this conflation of the two terms originated. But it’s easy enough to clear up.

    Simply put, CI is the process of integrating code into a mainline code base. In almost all scenarios today, CI is done using platforms designed specifically for the purpose. Implementing CI is, therefore, as simple as using the right tools.

    CD is more complicated. CD is about the processes that have to happen after code is integrated for app changes to be delivered to users. Those processes involving testing, staging and deploying code. There is no single type of tool that does all these things. These processes take different forms, depending on the culture of the team and the type of app it is creating.

    In actuality, then, CI and CD share little in common. Besides starting with the same word (“continuous”) and being different stages in modern software delivery pipelines, they don’t have much to do with one another.

    Why the Difference Matters

    Why is it important to steer clear of this confusion? The main reason is because, by conflating CI with CD, organizations can end up doing only the former but thinking they have implemented the latter.

    Setting up a CI server such as Jenkins or Bamboo to integrate code changes continuously does not mean you are doing CD. It just means you are using a CI server.

    To get to CD, you also have to organize your software testing, staging and deployment processes in a way that automates them as much as possible and makes them continuous. Doing so requires more than simply turning on a certain type of tool.

    How you actually do CD is fodder for a separate, much longer article (or you could just read the book about it). But recognizing that CI is not CD, and avoiding confusing formulations including CI/CD, is a start.

    — Chris Tozzi