devopsy.kinsta.cloud

Tag: software integration

  • 5 Areas to Consider When Building an Open Source Stack

    5 Areas to Consider When Building an Open Source Stack

    Open source software has gone mainstream, propelled by the increasing maturity of many communities and tools working together with organizations seeking ways to deploy software faster and cheaper. According to Gartner Research, over 95% of them are deploying open source software, and while it brings many benefits, it also introduces a whole new set of challenges, beginning with choosing the right open source stack. There are multiple risks including disappointing software performance, tools that cannot connect well with others, surprise additional costs, lack of control and security problems.

    Choosing the right open source stack is not easy. Many organizations lack the internal expertise to know what to look for or don’t have the time to research open source software sufficiently. Software engineers can’t be expected to become open source market experts overnight.

    Some years ago I worked with a team that certified open source solutions to be used in production. We created a list of 42 necessary qualifications to consider, which is (at least part of) the answer: Start by knowing what questions to ask when selecting open source software.

    Open Source Considerations

    Here are five areas to consider when choosing open source software:

    Time: While the ability to download open source and get it running fast might be tempting, it is better to invest in due diligence or think about working with an external specialist who can carry out the research. Take time in building the right solution for your use case(s). It is important to note that open source is often purpose-driven and chosen to perform a specific task; you may need to combine several different projects together to arrive at a complete solution.

    The Backers: While volunteers play an important role in successful open source, the fact is that the majority of the most well-known tools have sponsorship. Having a committed backer with a strong track record is a good sign for both longevity and software quality. The example I often cite is the Cloud Native Computing Foundation, which is behind Kubernetes, Linux and Prometheus.

    The Community: The commitment and responsiveness of the community are essential. Essential questions to ask include:

    • How fast does the community respond to queries or bugs?
    • How well do its members document and share collective knowledge?
    • How active, large and stable is the community? Is it likely to be around long enough to support the required use case for which the software is being selected, or is the community and the software likely to pushed sideways by something else on the horizon?

    The Different Types: Even some very knowledgeable CTOs do not properly understand the different types of open source, especially the difference between copyleft licensing and permissive licensing. Each has very different implications for an organization. The first requires contributors to share their changes with the relevant software community, while the second doesn’t make that obligation. Businesses are nervous about having to share parts of their proprietary source code and possibly their IP, though this only applies if the business modifies the code.

    Unforeseen costs are implicit in all of the following categories:

    • Time: could be wasted instead of saved if the solution doesn’t end up matching the use case.
    • Solution updates: A project without “the backers” runs the risk of dying and not receiving any more updates, forcing a business to spend energy choosing a new solution. The same is true for the community.
    • Legal ramifications: Businesses have faced expensive legal action if they modify and distribute copyleft source code without sharing it back.
    • Availability and support: Will the organization be supported by a third party, the community and by its own team? Consider the potential costs of all of the options.

    Software Quality and Integrations: It’s important to look at coding practices (do they align with the organization’s own standards?), usability and support for distributed resources. Above all, scrutinize integrations, because one of the biggest risks when building an open source stack is ending up with a bunch of tools that won’t work together or are not all aligned on the use case.

    Examine integrations throughout the stack: platform, database, middleware, application runtime and monitoring tools. Look for solutions that have lots of backing and that are known to interoperate well with lots of other solutions, such as Kubernetes.  Concentrate on well-adopted language distributions such as OpenJDK, which can run tons of other enterprise-class open source on top.

    It can be difficult, but choosing the right components for an open source stack will help make future curation easier and benefits clearer to see. Spending the time on research and asking the right questions from the beginning could save a lot of headaches in the future.

  • Optimizing an Effective CI/CD Pipeline

    Optimizing an Effective CI/CD Pipeline

    Continuous integration and continuous delivery (CI/CD) are often cited as pillars of successful DevOps. To establish and optimize the CI/CD development model and reap the benefits, companies need to build an effective pipeline to automate their build, integration and testing processes. This requires architecting CI/CD with five key ingredients which, when combined effectively, operate as a successful cornerstone of DevOps and provide myriad bottom-line benefits.

    Building the Pipeline

    Let’s first take a look at the five essential ingredients needed for a healthy CI/CD architecture, following up with a discussion of their benefits to the business:

    • Micro-agility in the form of testing and committing code frequently: The development life cycle begins with a request to development, which then takes the request and designs a solution, codes it, tests it and commits it a code repository. From here, the code is tested by QA and then deployed to production. While each step is important, we begin here, as local testing occurs most frequently and bugs found at this stage of the process help avoid the large expense of bugs found in production.To achieve micro-agility, developers must have a clear understanding of where they are expected to commit their code after changes are made and they should be expected to test and update their code frequently—many times a day. This means development testing should be easy to run locally and is a subset of QA. The result is that code pushed by development is by definition QA-ready.
    • Automated QA: In my work I see many organizations whose QA team is one of the largest bottlenecks in the process. It’s not for a lack of skill, but too few staff who spend a good portion of their time waiting on the QA test environments they need to do their work. To avoid this situation, the QA process should be fully automated, from QA ready code coming in the door to one-click promotion on the other side. In between the two, QA should have a constant feedback loop with development where communication about—and tracking of—bugs is clear and easy to follow, allowing development to easily repair issues and recommit the code for further testing. Moreover, automated QA in a CI/CD environment should focus on microcycles consisting of small test batches rather than weeklong macro tests. Automation and communication are the overriding imperatives here, exhibited by reports to developers that are communicated immediately and code commits that are fail-rejected automatically.
    • Production-like environments: To increase efficiency and effectiveness, development and testing environments should mirror production as closely as possible. They should follow the PRQ rule, as I like to call it—production-like, repeatable and quick. Specifically, it should be fast and easy to set up new environments, to the point where developers can, with the push of a button, deploy new environments for their development and testing needs. Doing so allows developers to fail fast, breaking and re-creating environments without fear as they know they can easily recreate the same environment every time. I often recommend using tools such as Jenkins to help with this process; if developers and IT operations use the same tool, it makes it that much easier to streamline the CI/CD process and achieve the promise of DevOps.
    • One-click promotion: As I alluded to earlier, a well-architected CI/CD pipeline provides one-click promotion to production. This ensures changes are seamlessly moved to the overall production environment with the least amount of friction possible between development and operations. To that end, this reduction in friction should occur across all the steps of the process, with efficient code delivery pipelines that are designed to balance security and agility from the outset. Cloud infrastructure coupled with container technology and automation can easily optimize the process across teams and throughout the DevOps journey.
    • Fully automated deployments: The last factor of an efficient CI/CD pipeline is full automation of deployment. Any configuration changes made by development and accepted by QA should automatically be applied. Continuous integration in this way helps avoid system downtime and human errors are decreased greatly. Deployments also should be continuously monitored so that IT is the first to learn of potential problems. This helps nip issues in the bud, allowing for faster rollback and return of code to development to fix anything that may not be functioning properly. Catching issues before customers do helps ensure customer satisfaction and strength of reputation.

    Benefits of the Pipeline

    In addition to these benefits, when combined effectively, these ingredients minimize the time and cost of local test, QA and production processes; reduce the time it takes to identify problems and their causes; and accelerate delivery of production-ready code. We can see this throughout the CI/CD pipeline:

    • Developers have greater control over the process and can respond more quickly to market demands by quickly bringing up new environments, testing against them and easily starting over, if need be. Moreover, developers can quickly adjust to new environments—an approach that has proven to decrease failures and resolution time when they do occur.
    • Operations benefits as reduced friction in the CI/CD pipeline created by automation decreases repetitive, manual work and with it the opportunity for “fat finger” mistakes and the greater opportunity to introduce inefficiency into the process.
    • And, less manual, repetitive work provides everyone in the process with more time for strategic work, which provides direct bottom-line value to the organization.

    CI/CD is a pillar of successful DevOps and according to research by CA, organizations that apply DevOps discover an 18 percent reduction in time to market of new applications and a 21 percent increase in delivering new software that they would not otherwise be able to complete. Teams that master an efficient CI/CD pipeline deliver faster, higher-quality applications to market bringing with it a 19 percent increase in revenue. Mixing these ingredients together results in an efficient, streamlined CI/CD pipeline that serves as a fundamental pillar of DevOps success that, when mixed together, delivers direct business benefits.

    About the Author/Aater Suleman

    Aater SulemanAater Suleman, CEO & co-founder at Flux7, is an industry veteran in performance optimization on servers and distributed systems. He earned his PhD at the University of Texas at Austin, where he also currently teaches computer systems design and architecture. As a faculty member at UT-Austin, Mr. Suleman focuses on optimizing DevOps architectures involving the AWS cloud. Along with the Flux7 consultant team, he works at every step of the dev flow, from creating local developer environments using Vagrant and Docker, to continuous integration and continuous delivery using Jenkins or RunDeck, to configuration management using Chef/Puppet/Ansible, to creating efficient systems for deploying new code from Git using Capistrano/Fabric/bash. His background is as an architect, and he has experience with both hardware and software. This combination brings a unique perspective and broad view to development and operations.