Tag: DevOps pipelines

  • Continuous Delivery Challenge Looms Large in 2022

    Continuous Delivery Challenge Looms Large in 2022

    As 2022 kicks off, it’s becoming apparent that the major DevOps challenge organizations will face is mastering best practices that enable continuous delivery.

    A global survey of more than 19,000 software development professionals conducted by the Continuous Delivery (CD) Foundation found 44% of respondents use either continuous integration or deployment. However, less than one in five (18%) use both practices to automate the entire software delivery life cycle. Continuous integration is currently the most popular DevOps-related activity (32%) followed by continuous deployment at 29%, the survey found.

    The survey also found that only 14% of respondents said, on average, that they can go from code checked-in to successfully running that code in production in one day. Just 10% said they routinely deploy code to production each day. Nearly a third of respondents said they still manually approve code deployments to production.

    Tracy Miranda, executive director for the CD Foundation, said it’s clear there is a need for a better understanding of continuous delivery as a distinct DevOps discipline that stands apart from continuous integration processes. While the two DevOps practices are complementary, she noted that continuous delivery requires a level of engineering expertise that many developers either lack or are simply not interested in gaining.

    In fact, the survey found more than a quarter of respondents (26%) are not involved in DevOps-related activities at all. At the same time, however, the survey revealed that 80% of respondents that work for companies with two or more employees are involved in DevOps to some degree. Nearly 60% of enterprise developers that participated in the survey employ a continuous integration/continuous delivery (CI/CD) platform, as opposed to 48% of developers working in small businesses and 42% of contractors. However, it’s clear from the survey results that much of that activity is focused on continuous integration rather than continuous delivery.

    The survey revealed, for example, that less than a quarter of respondents (23%) are involved in infrastructure provisioning and management at all. That compares to nearly a third (32%) that are involved in application and infrastructure monitoring. Only 20% are involved in creating automated tests, Collectively, the survey results make it clear there is still a long way to go in terms of shifting responsibility for DevOps practices further left toward developers.

    One reason continuous delivery is so challenging is that automating deployments across platforms has never been especially easy. Each platform tends to be customized to the point where it is difficult to automate application deployments. In effect, each platform is its own unique type of “snowflake.”

    However, with more attention focused on the integrity of software supply chains in the wake of a series of high-profile breaches, Miranda noted that automating software delivery will play a major role in achieving automation goals by reducing the need to rely on manual processes that are more easily compromised.

    At the same time, more organizations are starting to embrace Kubernetes clusters—this can simplify automation of software delivery across a set of platforms that all share a common application programming interface (API), she noted.

    The growing complexity of application development projects, coupled with increased reliance on software to drive digital business transformation initiatives, will also accelerate the adoption of continuous delivery best practices, added Miranda.

    One way or another, application deployment in 2022 is shaping up to be a lot more automated than it was in previous years. The only thing left to be determined now is the degree to which that goal can be achieved.

  • Open Source License Compliance Tips for DevOps Pipelines

    Open Source License Compliance Tips for DevOps Pipelines

    Open source components are used in most modern organizations, either included directly into software as code or as tools integrated into workflows. In DevOps environments, often a combination of the two occurs. These components can help speed development processes, provide functionality that isn’t otherwise available and decrease operating costs of teams.

    To successfully implement open source components, in software or pipelines, teams need to carefully evaluate what projects are included. Although most open source is free and supported by developer communities, there is no oversight guaranteeing that components are well-made and licensing can vary widely. This means that teams are responsible for ensuring that code is safe, stable and implemented legally.

    DevOps and Open Source Software Development

    When you are considering including open source components in your projects and pipelines, it is important to evaluate those components first. A well-developed evaluation process can help teams verify open source components and protect from:

    • Security vulnerabilities—Components are made by a community of developers, many of whom do not have extensive security knowledge. Additionally, because projects are worked on by many developers, code may not be uniform, creating bugs that can turn into security vulnerabilities.
    • Operational risks—Open source projects are maintained voluntarily. This means that project support may wane or unexpectedly drop. Additionally, updates and patches may not be released as reliably as with proprietary software. Since continued support is not guaranteed, you may be left with significantly more responsibility for maintaining and updating components that you originally planned for.
    • Legal risk/license IP compliance—Not all components are freely available for use by every organization or project. If you use these components unlawfully you can later be held accountable for breaking license agreements, even if you were unaware of doing so.

    Why Open Source Compliance Is Important

    Open source license compliance and legal risk are especially important and complex for commercial development teams. There are more than 200 open source license variations, each with different standards for when projects can be used, by whom and in what capacity. Compliance strategies ensure that you are aware of the licensing requirements of any component you use.

    In particular, the things you need to be aware of when it comes to open source compliance is whether projects can be included in commercial products or used by teams of your size and type. If projects cannot be used in commercial projects or require projects to be made open source upon inclusion, you are risking your own intellectual property.

    Including open source projects against license can open you to legal action from project maintainers, contributors or third-party organizations. For example, some projects are part of a dual-licensing strategy with companies looking to protect their own financial interests. Or, cases may be brought as a means to raise awareness about open source licensing by concerned community members.

    Another possibility is being actively targeted for financial gain. Some projects are contributed to by open source trolls, software developers who have made a business of suing for license non-compliance. These developers take advantage of licenses such as GNU General Public License v2.0 (GPL-2.0), which terminates usage and distribution rights when breached, enabling developers to sue to stop distribution or to demand a portion of profits.

    Open Source Compliance Tips for DevOps Pipelines

    There are several practices you can implement to ensure that your pipelines and products are in compliance when it comes to open source. These practices are especially important to integrate throughout your pipeline to help identify and correct any issues that may exist before products are pushed to production. Below are some practices to get started with.

    Create an Open Source Review Board (OSRB)

    An OSRB is a group responsible for creating and enforcing open source policies and procedures within your organization. These boards often include members from development, operations, legal and management teams.

    OSRBs can serve as advocates for open source use with executives and stakeholders. They can also serve as a source of information for teams looking to incorporate or contribute to open source projects. These boards can help you ensure that you remain in compliance while deriving the greatest benefit from open source.

    Incorporate Compliance Processes Throughout Development

    Compliance policies should be created and incorporated into all steps of the software development life cycle (SDLC). These policies should outline how components are vetted, by whom, how and when components can be included, and how tracking is done.

    Ideally, compliance processes and policies should be built into your pipeline processes. For example, by returning builds to developers if they include blacklisted licenses. By incorporating compliance, you reduce the chance that it is overlooked and minimize interruptions to productivity. Incorporation also helps you ensure that licensing issues are found sooner rather than later, minimizing the impact if components need to be replaced or removed.

    Create ‘Playbooks’ for Commonly Used Licenses

    While there are many licenses that open source projects can fall under, there are several that are significantly more common than others. For example, MIT or Apache—for these licenses, you can simplify compliance procedures by creating playbooks.

    Playbooks outline the details of licenses, including license grants, obligations, restrictions and patent impacts. When a developer wants to include a component, they can use these playbooks to quickly determine whether that component is eligible for use. While this won’t eliminate all legal questions when it comes to licensing, it can significantly reduce how often you need to confer with legal counsel about inclusions.

    Use Software Composition Analysis (SCA) Tools

    SCA tools can help you identify open source components in your own products and in any tools you adopt. These tools enable you to scan source code to determine what components are inside and create an inventory that you can then use to verify compliance. As a bonus, these tools can also help you identify components with known vulnerabilities, which you can then mitigate.

    SCA is especially helpful when you are newly implementing open source compliance policies. It can provide information on the types of components you are already using and help you ensure that those existing components meet your newly applied standards. Some tools even enable you to set up alerts for license conflicts or irregular version use, making verification easier.

    Conclusion

    Open source compliance is a critical component of most DevOps implementations. To ensure compliance, teams need to carefully choose and then properly keep track of open source components. You can do that by creating an open source review board (OSRB), which can help enforce open source compliance policies. You should also incorporate compliance throughout your pipeline, create playbooks for commonly used licenses and leverage SCA tools.

  • How DevOps Orchestration and Feedback Loops Can Enhance Your DevOps Pipeline

    How DevOps Orchestration and Feedback Loops Can Enhance Your DevOps Pipeline

    DevOps orchestration and feedback loops can significantly impact and improve your organization’s delivery pipeline. In this post, we’ll define and illuminate the importance of DevOps orchestration and feedback loops, and how together they can make a difference in your pipeline.

    DevOps Orchestration

    Although people widely use the terms “automation” and “orchestration” interchangeably, their scopes vary with significant process differences between them. DevOps automation automates a single function or a task in a DevOps pipeline that is routine and monotonous. Orchestration describes a string of such automated tasks consolidated into a streamlined workflow. In environments where automation would not work because of barriers among the groups, orchestration can be used to help organizations solve issues despite such barriers. Hence, DevOps orchestration simply brings together already existing automation networks into a cohesive, consolidated platform.

    The Need for DevOps Orchestration

    DevOps teams deal with constant navigation between development and operations. It would be highly beneficial if a single platform could pull together the tools teams typically use as various automation pieces from multiple DevOps toolsets. This is where DevOps orchestration services come into the picture, as orchestration enables teams to achieve that.

    The primary goal of DevOps orchestration is to ensure seamless and quick delivery of new builds into production. It also aims to achieve streamlined and error-free software releases. Different organizations use various coding standards, guidelines and policies. DevOps orchestration plays a major role in efficiently managing and ensuring adherence to such policies. DevOps orchestration also is pivotal in properly defining and maintaining third-party software and their multiple versions across different environments.

    DevOps Monitoring

    Effective DevOps monitoring is vital. It is the key to understanding and evaluating various performance metrics in the production environment. When done right, DevOps monitoring can provide visibility into different critical aspects such as deviations in performance, application functionality issues and service availability problems. In any DevOps environment, the ability to identify and pinpoint such issues proactively can be highly beneficial, and DevOps monitoring enables executives to do so.

    Feedback Loop

    Feedback loops are the essence of any process improvement and DevOps is no exception. Typically, such loops initially function with an organization’s policies and progress all the way through implementation, policy adjustments (if any) and usually ending with reimplementation. They are not limited to specific process cycles in a DevOps environment—for example, it’s applicable to CI/CD practices, infrastructure monitoring and application development

    Types of Feedback Loops

    Broadly, there are two categories in feedback loops: reinforcing feedback loop and balancing feedback loop. People refer to the former as “a loop of accelerating change.” In a reinforcing feedback loop, increased value in the first entity results in an increase in the second entity, which in turn further increases the value of the first entity. Conversely, in a balancing feedback loop, value increase in one entity results in a decreased value in the second entity, which consequently decreases the value of the first entity.

    Significance of Feedback Loops

    One of the major capabilities that differentiates and carves a unique niche for DevOps from the rest of its predecessor methodologies is its feasibility to incorporate feedback from customers as well as other systems in the pipeline, including the monitoring system. The feedback obtained helps to bridge the gap between the software’s function and customer expectations. It also provides valuable insights about what you can do to improve the build of the software and ultimately enhance its functionality and reliability.

    Feedback loops are particularly crucial when it comes to continuous testing. To carry out continuous testing effectively, building automated tests is only a job half done; the rest depends on the visibility of the test results and how you can utilize them to enhance the existing process. To do that, you need comprehensive feedback of your software’s performance across various stages of the software development life cycle (SDLC), starting right from development and continuing all the way to post-production. Implementing the right feedback loops will help you gather such detailed feedback.

    Thus, feedback loops serve as the difference between merely speculating how an end-user might consume a product and methodically improving the existing workflow to better suit the end-user requirements through feedback.

    — Kavin Elango