devopsy.kinsta.cloud

Tag: CI/CD

Choosing the Right Software Deployment Tools
Imagine this scenario: After investing countless hours in developing your software, it is finally ready for deployment. Now, you must select the appropriate software deployment tools to guarantee a seamless release.

This article will guide you through the concept of software deployment and discuss various tools that cater to different needs—from dedicated deployment tools to comprehensive CI/CD pipeline solutions and GitOps approaches. By understanding these categories and their benefits, you’ll be well-prepared to make an informed decision on which tool is the best fit for your organization’s needs.

What is Software Deployment?

Software deployment plays a crucial role in the software development life cycle. It involves packaging, distributing and installing an application on target systems or environments, aiming to make your software accessible and usable for end-users seamlessly.

In today’s fast-paced world, companies must be able to deliver applications rapidly and reliably. A well-planned software deployment strategy ensures that new features reach users promptly while minimizing downtime and errors during updates.

What to Look For in a Software Deployment Tool

When selecting the ideal software deployment tool, consider the following factors:
- Automation: The tool should automate complex software deployments, making the deployment process faster and more efficient.
- Continuous deployment: Continuous deployment tools allow you to deploy software repeatedly, ensuring consistent deployments. Choose a tool that automates release management and integrates automated builds throughout the development process.
- Scalability: The tool should be capable of handling large-scale deployments and distributing software across multiple servers. Opt for a tool that enables applications to deploy to multiple servers simultaneously.
- Integration: The tool should integrate with version control and other third-party tools, making the deployment process seamless. Look for a tool that integrates with modern DevOps processes and connects automated builds to configuration management.
Full CI/CD Tools

Bamboo

Bamboo by Atlassian is a robust tool that integrates seamlessly with other Atlassian products, such as Jira and Bitbucket.

It offers first-class support for multiple languages and platforms while providing built-in deployment projects for smooth releases.

GitLab CI/CD

GitLab CI/CD, an integral part of GitLab, simplifies pipeline management using YAML configuration files stored within your project repository.

This approach enables easy collaboration among team members while ensuring consistent build environments across all stages of development.

Codefresh

Codefresh is a cloud-native solution designed specifically for Kubernetes deployments, It provides a modern interface to manage pipelines visually, along with advanced real-time debugging capabilities.

Dedicated Deployment Tools

Jenkins

Jenkins was originally conceived as a continuous integration (CI) tool, but many organizations use it mainly for deployment. It is an open source automation server that provides support for building and deploying software projects, as well as to schedule and run tasks at regular intervals. Learn more about Jenkins in this in-depth blog post.

Harness

Harness is an intelligent software delivery platform that simplifies deployments through automation and machine learning. Their continuous delivery-as-a-service (CDaaS) approach helps you deploy faster while minimizing risks and errors.

AWS CodeDeploy

If you’re using Amazon Web Services (AWS), AWS CodeDeploy might be a perfect fit for your needs. This completely managed service automates code deployments to any instance or serverless Lambda function in AWS, guaranteeing consistent results every time.

Spinnaker

An open source project backed by Netflix and Google, Spinnaker offers powerful multi-cloud continuous delivery capabilities for deploying applications at scale across various cloud providers like AWS, GCP, Azure, Kubernetes clusters, etc.

GitOps Tools

Argo CD

Argo CD is a declarative, Kubernetes-native continuous delivery tool that uses Git repositories as the source of truth for application definitions. It simplifies rollbacks, provides audit trails and enforces consistency across environments.

Flux CD

Flux CD, another Kubernetes-focused option, automates deployments by monitoring your code repository for changes and applying them to your cluster accordingly. Bonus: It supports multi-tenancy and integrates with Helm charts.

Weave GitOps Core

Lastly, Weave GitOps Core offers an open source foundation for managing Kubernetes applications using the power of GitOps workflows. This versatile tool can be extended to fit various use cases while maintaining simplicity in its core functionality.

Conclusion

There are many software deployment tools available that can help automate the software delivery process, reduce errors and improve collaboration between development and operations teams. From open source tools like Jenkins to cloud-native solutions like AWS CodeDeploy, each tool has its own strengths and features that make it suitable for different use cases.

When selecting a software deployment tool, it is important to consider factors such as compatibility, automation, scalability, security, monitoring, deployment environment compatibility, integration with existing tools and support and documentation. By carefully evaluating these factors and choosing the right tool for your specific needs, you can ensure that your software deployment process is efficient, reliable and successful.

Image Source: Freepik
June 21, 2023
Cycode Leverages eBPF to Secure CI/CD Pipelines

Cycode today added a Cimon extension to its application security platform that uses extended Berkeley Packet Filtering (eBPF) to thwart cyberattacks against continuous integration/continuous delivery (CI/CD) pipelines.

Alex Ilgayev, head of security research for Cycode, said Cimon uses eBPF to inspect network connections, running processes and file modifications within a pipeline to learn standard behaviors. Armed with those insights, Cimon will then surface any anomaly in real-time, he added.

DevOps teams can then craft and embed security and compliance rules within their pipelines based on the severity of the anomalies discovered, said Ilgayev.

eBPF makes it possible to run sandbox programs at the kernel level without adversely impacting performance. In effect, eBPF bridges the boundary between kernel and user space by enabling developers to combine and apply logic across multiple subsystems that, historically, were completely independent of one another. That approach enables, for example, a security tool to scale to the point where it can identify threats at much higher levels of throughput to improve overall scale at a time when the volume of cybersecurity attacks continues to increase.

Cycode is taking advantage of eBPF to enable DevOps teams to not only defend against known attacks involving malicious package installation, typosquatting, repojacking, dependency confusion, dependency hijacking and other types of dependency attacks but also zero-day threats, noted Ilgayev.

Cycode provides a platform for mapping metadata and events using graph technology that makes it simpler for application development and cybersecurity to consolidate alerts generated by their DevSevOps tools. Most recently, Cycode added a Cycode Application Security Orchestration and Correlation (ASOC) module that will automatically discover all the tools that make up a DevSecOps workflow without requiring any integration effort.

The approach enables the Cycode platform to discover, for example, dependency issues that would otherwise be missed by software composition analysis (SCA) or static application security testing tools (SAST).

While a lot of progress has been made in terms of adopting DevSecOps best practices, many organizations are still struggling with securing their software supply chains. Most of the members of a DevOps team have limited cybersecurity expertise, so they need the help of a cybersecurity professional to determine what specific actions are required to remediate a vulnerability. Cybersecurity professionals, conversely, don’t have a lot of application development expertise.

As the number of regulations specifically focused on application security steadily increase, every organization that builds software will soon need to embrace DevSecOps to better secure software supply chains. Unfortunately, developers have too often viewed cybersecurity as an obstacle to deployment while cybersecurity teams have historically considered developers to be a primary cause of the breaches they are expected to clean up. The first step toward bridging that divide is, of course, to realize that application developers and cybersecurity teams need to be invested in each other’s success.

June 12, 2023
The API Life Cycle Mixer’s Manual: What Goes Into a Single Source of Truth?

In bartending, the mixer’s manual is a trusted source of information. These books can contain hundreds of recipes for classic cocktails, some of which might be 100 years old, while others might be for drinks that only recently arrived on the scene. Mixer’s manuals also can be great sources of inspiration for new, yet-to-be-tested recipes for exciting additions to a bar’s or restaurant’s offerings. This balancing act of preserving the old while encouraging exploration into the new can provide a valuable lesson for API development in the software world. An API’s “single source of truth” is, in many ways, the software organization’s own mixer’s manual. This is especially true when the right approach is taken to ensure that software remains functional and performant and serves up experiences that delight customers and keep them coming back for more.

The Rise of APIs and Mixology

While we might think of mixology and the mixologists who practice this craft as relatively new terms, the first recorded use of the word “mixologist” dates back to 1852 and “mixology” followed shortly after.

Just how similar are the work, mindsets and passions of mixologists and API developers and designers? Check out this definition of a mixologist and count the similarities for yourself:

“The term ‘mixologist’ refers to someone who studies the history of mixed drinks, has a rich appreciation of the ingredients and techniques used and regularly creates new and innovative mixed drinks … their job title implies that they do a significant portion of their job behind the scenes, creating new craft cocktails and putting their signature twist on existing favorites.”

The best API developers, designers and architects of the world should recognize a lot of the above in their own work. They have deep respect for their craft and are always honing it further. They seek to understand the tastes and palates of their consumers. They factor in other stakeholders’ inputs, they test, iterate and test their creations again before “serving” those APIs to production.

Like mixologists and bartenders (if you see them as two different roles), every professional along the API life cycle should understand the needs of consumers, customers and the business–and then design and develop their APIs accordingly. An essential part of this process is continuous validation and adaptation to market requirements.

Maintaining Confidence in Ever-Changing Recipes

Changes to APIs, even minor ones, can have significant downstream effects. And the same can be said for tweaks made to an age-old cocktail recipe, where adding or removing a single dash of this or that can completely change the flavor of a drink … as well as people’s interest in it.

While there’s no “standard” rate of change to APIs to plan for, you can expect changes to be needed with each new software release. This could mean changes are being made (and need to be tested against and validated) every few weeks, every month or even every day, depending on your release cadence. There also can be as-needed, ad-hoc changes.

What’s crucial for teams to ensure is that these changes do not disrupt an API’s functionality or performance. To make this possible, teams are increasingly leveraging API contract testing to provide a safety net of sorts. This comes in the form of testing that validates changes against a “contract” that details an API’s original, agreed upon and, therefore, required functionality.

The ability to foresee the impacts of changes to APIs can depend on the maturity of an engineering team and its ability to collaborate across departments, time zones and varying levels of technical expertise. Tools and guidelines that promote standardization and collaboration can prevent mishaps from overlooking or being blind to negative downstream effects.

Documentation is Key

Whether we’re talking about being able to trust a single source of truth for the most up-to-date version of a published API or the latest edition of a mixer’s manual for tried-and-true drink recipes, clear and concise documentation is key. But it’s also not enough for this documentation to simply exist.

It seems like common sense—although you’ve probably worked somewhere where it wasn’t—that your API documentation also needs to be easily accessible, understandable and usable. Ensuring all of these things leads to smoother onboarding and a positive developer experience and, ultimately, to wider adoption of a functional, performant API that meets or even exceeds expectations.

API documentation is often the first point of contact a developer has with an API. It serves as a guide, walking developers through the API’s features, endpoints, request/response models and error messages. It also provides insight into the underlying business logic and unique characteristics of an API.

Good API documentation follows the principle of a single source of truth, meaning that it should be comprehensive, up-to-date, and consistent. It should always mirror the actual behavior of the API, reducing the guesswork for developers and minimizing the potential for misunderstandings and errors.

Just as a mixologist relies on a detailed and accurate cocktail recipe to deliver a pleasing experience to a patron, API developers depend on good documentation to create and deliver useful, value-add APIs and powerful applications.

Continuous Improvement = Continuous Education

Like a mixologist who meticulously adjusts the ingredients of a cocktail or who creates new cocktails to meet the evolving tastes of customers, developers must constantly adapt their APIs to meet changing market demands, business needs and industry standards.

This is achieved through a potent blend of market research, collaboration, adherence to standards and governance and meticulous testing. However, as software development, testing and deployment technologies and processes continue to evolve, the tech industry must continue educating itself and refining its practices.

With a mindset focused on constant learning and continuous improvement and a dependable single source of truth as their guide, development teams can confidently serve up exceptional API experiences in every release. In the end, the aim is to create APIs that are reliable and easy to consume and that meet the unique needs of the moment–much like a mixologist’s perfect beverage.

June 9, 2023
Mastering DevOps Automation for Modern Software Delivery

DevOps automation services are a crucial aspect of modern software development that helps organizations to improve their software delivery processes and keep up with the pace of innovation. With DevOps automation, developers can automate repetitive and time-consuming tasks, enabling them to focus on writing code and delivering features faster and with higher quality. By automating processes such as continuous integration, continuous delivery and infrastructure-as-code, organizations can reduce costs, improve collaboration between teams and respond quickly to changing business needs and customer requirements.

In this article, we will discuss the basics of getting started with DevOps automation services, including popular tools and best practices that organizations can use to implement a successful automation strategy.

Agility and Efficiency for Smart Digitalization

Emerging technologies like multi-cloud, IaC, AI/ML and IoT have only increased the importance of DevOps automation, as they bring new challenges demanding agility, flexibility and efficiency in software delivery processes. Multi-cloud and hybrid cloud environments, for example, can benefit from automating the deployment and management of software across multiple cloud environments. Similarly, AI/ML and IoT require organizations to collect and process large amounts of data in real-time, which can be challenging without automation. DevOps automation can help organizations automate data pipelines and analytics, enabling them to process data faster and more accurately.

Here are some benefits of DevOps automation:

● Speed and Efficiency: DevOps automation enables organizations to deliver software faster and with higher quality. By automating repetitive and time-consuming tasks, developers can focus on writing code and delivering features.
● Agility and Flexibility: DevOps automation provides organizations with the agility and flexibility to adapt to changing business needs and customer requirements. Automated processes make it easier to release software updates and respond quickly to market demands.
● Cost-Effectiveness: DevOps automation reduces the costs associated with software development by reducing manual labor and minimizing errors. It also enables organizations to use cloud resources more efficiently, which can result in cost savings.
● Improved Collaboration: Bringing the different stakeholders along with devs and operations, DevOps fosters a collaborative approach toward enhanced CI/CD. By automating processes, teams can work together more effectively and share knowledge and feedback in real-time.
● Scalability: DevOps automation enables organizations to scale their software delivery processes quickly and efficiently. Automated processes make it easier to deploy software across multiple environments and platforms, which is essential for organizations that need to scale rapidly.

Getting Started With Efficient DevOps Automation

Implementing DevOps automation requires a well-planned strategy, best practices and the right set of tools. In this section, we will discuss how to successfully implement DevOps automation in your organization.

Understanding DevOps
Before you begin with automation, it is essential to understand the principles and practices of DevOps. DevOps is a culture that emphasizes collaboration and communication between software development and IT operations teams to build, test and release software more quickly and reliably. There are some common tools that DevOps teams use for their work, collaboration and communication:
● GitHub: The Git repository hosting service used for version control and source code management functionality, especially for distributed projects.
● JIRA: A project management tool that helps teams plan, track and manage software development projects.
● Slack: A communication platform that enables real-time messaging, file sharing and collaboration.

Choose the Right Tools
Automation is at the heart of DevOps, and selecting the right tools is crucial. Choose tools that fit your requirements, budget and infrastructure. Some popular DevOps automation tools include:

● Jenkins: An open-source automation server that enables continuous integration and continuous delivery (CI/CD) pipelines.
● GitLab: A web-based Git repository manager that provides CI/CD capabilities, project management, and collaboration tools.
● Puppet: An open-source configuration management tool that automates the deployment and management of software across multiple environments.
● Chef: A configuration management tool that automates the deployment and management of software across multiple environments.
● Ansible: Open source tools for automating software provisioning, deployment and configuration management

Identifying Processes for Automation

First, list all the critical processes within your organization that can be enhanced with automation. Start with the repetitive and time-consuming tasks that can be automated, such as code testing, deployment and monitoring. There are a number of popular tools that can help with such automation:

● Selenium: An open source testing framework that enables automated testing of web applications.
● Docker: A containerization platform that enables the creation, deployment and management of containerized applications.
● Kubernetes: The open source container orchestration tool is essential in the DevOps pipeline for orchestration and automation of the deployment, scaling and management of containerized applications.

Creating a Plan

Once you have identified targets for automation and selected tools, create a plan for automation that includes a roadmap and milestones. Start with small projects and gradually expand the automation to cover all aspects of the software development life cycle. Some popular tools for strategizing these life cycles are:

● Trello: A project management tool that enables teams to organize and prioritize tasks using Kanban-style boards.
● Asana: A project management tool that enables teams to track and manage tasks using to-do lists, calendars and timelines.

Training The Team

Automation requires new skills, so it is essential to train your team to use the tools and processes effectively. Provide training and resources to help your team learn new skills and techniques. Some popular education and training resources include:

● Udemy: An online learning platform that provides courses on DevOps automation tools and techniques.
● Coursera: An online learning platform that provides courses on DevOps and related topics.
● Pluralsight: An online learning platform that provides courses on DevOps automation tools and techniques.

Implementing Continuous Improvement

A continuous feedback loop is essential for DevOps’ iterative process and continuous improvement. Integrate mechanisms for evaluating your automation processes, identify areas for improvement and make necessary changes. Here are two common tools used to do this:

● New Relic: An application performance monitoring tool that provides real-time visibility into the performance of software applications.
● Splunk: An operational intelligence tool that enables organizations to analyze and visualize machine data in real-time.

June 6, 2023
The Future of Continuous Testing in CI/CD

If you ever wonder why continuous testing is so important, think about the following: In the past, software testing was typically done after the code was written and sent to the QA department for independent testing. When bugs were discovered, the code was returned to the developers for correction. While this testing method worked, it was time-consuming. Nowadays, companies want to produce high-quality products quickly, so they need a faster and more efficient testing process. This is when continuous testing comes in.

In this article, we will explore continuous testing from its current state and its future in CI/CD to the emerging technologies and trends that will shape its development. We will also cover the importance of continuous testing for businesses and some of its challenges.

What is Continuous Testing?

Continuous testing is a software testing approach in which tests are automated and run throughout the software development life cycle. It is a part of the continuous delivery (CD) process, in which software changes are delivered to customers continuously and iteratively.

Continuous testing aims to identify defects and issues as early as possible, reducing the cost and time of fixing them later. Continuous testing provides immediate feedback to the development team, helping them to detect and address potential problems quickly and efficiently.

According to the Reports and Data source survey, 21% of QA professionals have incorporated continuous testing, source code management and continuous integration into their DevOps process to speed up code release.

Advantages of Continuous Testing

Here are several advantages of implementing continuous testing as a part of the software development process:
● Continuous testing enables teams to identify bugs earlier in the software development process;
● Continuous testing provides immediate feedback to developers on the quality of their code, enabling them to address any issues immediately;
● You can minimize business risks by assessing and identifying potential issues before they become real problems;
● Automating your test cases can reduce the time spent on testing;
● Accelerating the process of software delivery to production helps release the product faster;
● You can run performance tests in similitude, which can increase the speed of test execution.

Disadvantages of Continuous Testing

While continuous testing has become an integral part of the software development process, it’s not without its challenges. As software development teams work to incorporate testing into their CI/CD pipelines, they may encounter obstacles that can hinder their progress.
● Continuous testing requires a different skillset than traditional testing, including expertise in DevOps, automation and CI/CD pipelines;
● Different tools and technologies are available for continuous testing, and choosing the right ones can be tough. Integrating these tools with other parts of the development process can also be challenging;
● To fully realize the benefits of continuous testing, you need to update your testing strategy by moving away from relying solely on traditional testing methods and adopting a well-defined approach to test data management;
● Failing to integrate code regularly can result in issues such as duplicated coding efforts and non-compatible code, leading to defects.

Emerging Technologies in Continuous Testing

With the increasing demand for faster and more reliable software releases, emerging technologies such as AI/ML, automation and DevOps appeared in continuous testing. These technologies help teams to automate and streamline their testing processes, identify defects more quickly and improve overall software quality. Let’s examine some of them:

Artificial intelligence/machine learning (AI/ML). AI and ML refer to the field of computer science that deals with developing intelligent machines that can simulate human intelligence and learn from data.

One of the main benefits of AI and ML in continuous testing is the ability to automate test case generation, which reduces the need for manual testing and frees up time for other tasks. This enables testers to test applications more comprehensively and frequently, ensuring defects or issues are identified and addressed quickly. AI and ML increase testing accuracy by identifying patterns and anomalies in the testing data.

AI and ML techniques can efficiently and rapidly analyze extensive amounts of data, allowing them to recognize patterns and irregularities in testing data that human testers might overlook.

Automation: Automation in continuous testing refers to using automated tools and processes to execute software tests repeatedly and consistently. It involves automating various stages of the software testing life cycle, such as test planning, test case design, test execution and test reporting.

Automation can efficiently perform regression testing, ensuring that changes do not introduce new defects or issues in the software. Tests can be executed 24/7 without human intervention, reducing the time and cost required for testing.

DevOps: DevOps in continuous testing is the application of DevOps principles to continuous testing, focusing on integrating testing into the software development life cycle and enabling faster and more reliable delivery of software.

In the context of continuous testing, DevOps emphasizes testing integration into the development process, enabling developers to detect and fix defects early in the cycle. DevOps also promotes automation, collaboration and communication between development and operations teams, allowing faster and more efficient testing and deployment.

Blockchain: Blockchain technology has been gaining increasing attention in the software development and testing industry due to its ability to provide secure and tamper-proof data storage.

In continuous testing, blockchain stores testing results and data in a decentralized and transparent way, making it virtually impossible for anyone to modify or manipulate the data. This enhances the credibility of the testing process and outcomes and provides an added layer of security to ensure the integrity of the data.

Blockchain technology enables efficient and seamless data sharing between stakeholders involved in the testing process, enabling faster and more accurate decision-making.

Internet-of-Things (IoT): In continuous testing, IoT simulates many IoT devices and platforms, enabling testers to test their applications across multiple devices and platforms. This ensures that applications are compatible with various IoT devices and platforms and perform reliably and consistently in various environments.

For instance, when creating an application that needs to run on multiple devices, it is possible to use an emulator to test the app on your laptop without setting up a development environment for each type of device.

IoT-enabled continuous testing provides real-time feedback on the performance and functionality of applications, enabling organizations to identify and address issues quickly and effectively.

Trends in Continuous Testing

Continuous testing is rapidly evolving to meet the needs of modern software development practices, with new trends emerging to address the challenges development teams face. Three key trends currently gaining traction in continuous testing are cloud-based testing, shift-left testing and security testing. These trends are driven by the need to increase efficiency and speed in software development while ensuring the highest quality and security levels. Let’s take a closer look at these trends.

Cloud-Based Testing: Continuous testing is deployed through cloud-based computing, which provides multiple benefits like ease of deployment, mobile accessibility and quick setup time. Businesses are now adopting cloud-based services due to their availability, flexibility and cost-effectiveness. Cloud-based testing doesn’t require coding skills or setup time, which makes it a popular choice for businesses.

AI-Driven Testing: AI-driven continuous testing is often integrated into cloud-based testing platforms, providing end-to-end automation for web, mobile and API testing. AI and cloud-based testing allows for access to testing environments anywhere, anytime, and provides for regular automated testing using platforms like Selenium.

Shift-Left Testing: Shift-left testing is software testing that involves testing earlier in the development cycle rather than waiting until later stages, such as system or acceptance testing. The goal of shift-left testing is to identify and address defects as early as possible.

Delaying testing until the end of development often results in more complicated bug fixes since the software is already fully built. This can cause an increase in cost and time to market. Identifying and fixing bugs earlier in the software development life cycle is generally more cost-effective.

As IBM’s Systems Sciences Institute claimed, repairing a bug discovered during implementation costs roughly six times more than one found during design. If a mistake is detected after product release, the cost of fixing it is four to five times greater than one found during design and up to 100 times greater than one found during the maintenance phase. This suggests that a bug’s cost increases exponentially as the software progresses through the SDLC.

Low-Code/No-Code: Low- or no-code in continuous testing refers to using visual interfaces and pre-built elements or modules to create and execute automated tests instead of writing complex code. Low-code/no-code testing tools often provide drag-and-drop interfaces and pre-built components that can be combined to create tests, reducing the time and effort required to develop and maintain test scripts.

This type of testing provides several advantages, such as allowing non-technical business experts to run testing, speeding up test production and execution and enhancing product quality by enabling more thorough and efficient testing.

Generally, according to statistics, by 2025, around 70% of new business applications are predicted to use low-code or no-code technologies.

Security Testing: Due to the rising incidence of cybersecurity threats and data breaches, security testing has become a crucial aspect of software quality assurance. In the year 2023, the tech world anticipates that there will be a stronger focus on security testing, with the emergence of more advanced tools and methodologies aimed at detecting and resolving security weaknesses in software applications.

June 6, 2023
CDF Marries Emporous Repository to Ortelius Management Platform

The Continuous Delivery Foundation (CDF) has added the open source Emporous repository for software artifacts to an Ortelius platform for managing the software artifacts that make up a microservice.

Ortelius was originally created by DeployHub and subsequently contributed to the CDF, which also oversees the development of the open source Jenkins continuous integration/continuous delivery (CI/CD) platform.

Emporous is based on a Universal Object Reference (UOR) conceptual model that enables dynamic schema registration and decentralized content-addressable reference of a wide spectrum of software artifacts. Originally developed by Red Hat, Emporous will now be managed as a sub-project within the larger Ortelius incubation-level project by the CDF.

DeployHub CTO Steve Taylor said this integration would make it simpler to store, organize and search metadata related to software artifacts, along with the artifacts themselves, within the context of the Ortelius platform for managing microservices.

Ortelius provides insights into the relationships and dependencies between different software components and applications. It does this by collecting and aggregating pipeline data from multiple sources within a software supply chain. Emporous will be incorporated within Ortelius to enhance metadata search capabilities and provide a single repository to store any type of artifact, from containers to jar files. It will now also be able to store metadata and also the artifacts themselves within the same Emporous catalog.

Those capabilities will significantly improve security and compliance reporting across a software supply chain, added Taylor. DevOps and cybersecurity teams will be able to automatically apply and enforce supply chain policies and procedures, he said.

DevOps teams need a way to aggregate all the data from the various tools they use and the data must be easily searchable. Armed with those insights, it should become simpler to surface insights into, for example, which components of an application contain known vulnerabilities.

In the longer term, IT organizations will be able to apply machine learning algorithms and other forms of artificial intelligence (AI) to automatically surface insights that will enable DevOps teams to automate workflows, said Taylor.

Microservices-based applications are considerably harder to build and manage than monolithic applications, but they are also easier to maintain because individual microservices can be ripped and replaced.

Most microservices today are built using containers, but there are also microservices that are written in Java, for example. In some cases, developers view monolithic applications as being a large, less flexible microservice that they invoke via an application programming interface (API).

There’s no doubt that the rise of microservices made application environments a lot more complex. The tools needed to simplify the management of the next generation of applications are finally starting to become more widely accessible. In theory, the number of organizations that can successfully build and deploy these types of applications should steadily increase in the months and years ahead.

In the meantime, there’s clearly still plenty of work that needs to be done to improve the overall state of software delivery. A recent CDF survey found only 22% of respondents were using continuous integration and continuous delivery (CD) to automate the building, testing and deployment of code in a production environment. As CD becomes more distinct from CI, however, the number of organizations that can achieve that goal when deploying microservices-based applications should steadily increase.

May 26, 2023
GitLab Adds More AI and Cybersecurity Capabilities to CI/CD Platform

GitLab this week delivered an update to its continuous integration/continuous delivery (CI/CD) platform that adds additional generative artificial intelligence (AI) and cybersecurity capabilities.

The GitLab 16 release included cybersecurity capabilities such as centralized policy management, expanded compliance reports and controls, compliance dashboards and default Level 3 attestation for the supply-chain levels for software artifacts (SLSA) framework.

At the same time, GitLab this week also made available an emergency patch for two modules that have a path traversal flaw, tracked as CVE-2023-2825, that could allow unauthenticated users to read arbitrary files on the server under certain circumstances.

In terms of generative AI, GitLab 16 added a value stream forecasting capability that automatically generates workflow summaries for senior managers. GitLab is also adding a code refactoring capability and the ability to automatically resolve vulnerabilities to an update to GitLab 16. Previously, GitLab made available a ‘suggested reviewers’ features, an ‘explain this code’ and an ‘explain this vulnerability’ capability that are all enabled by large language models (LLMs) developed by Google. GitLab has also partnered with Oracle to provide additional AI capabilities enabled by machine learning algorithms. In 2021, GitLab acquired UnReview, a provider of a tool that uses AI to identify which expert code reviewers to assign to a project based on the quality of their previous efforts and current workloads.

David DeSanto, chief product officer at GitLab, said with 10 AI capabilities GitLab is providing more of these types of capabilities than any other DevOps platform provider. In addition, those capabilities are distributed across the entire software development life cycle rather than being focused solely on making developers more productive, he noted.

That’s critical, because if AI capabilities are only focused on developer productivity, an imbalance is created; DevOps workflows will not be able to absorb the increased amount of code that will soon be moving simultaneously through pipelines, DeSanto added.

AI will also further fuel adoption of platform engineering as a means for centralizing DevOps management. It’s becoming more apparent that infusing these types of capabilities into DevOps platforms built and maintained by an internal IT team is going to be a major challenge, he added.

There is no doubt that generative AI capabilities will soon become widely integrated across DevOps workflows. That will enable developers to build more secure code faster while at the same time making it simpler for software engineers to maintain large codebases. Just about every job function imaginable will be impacted to varying degrees. In the case of DevOps teams, the ultimate impact should involve less drudgery as many of the manual tasks that conspire to make managing DevOps workflows tedious are eliminated. In fact, AI may ultimately make DevOps, as a methodology for managing IT, more accessible to a much wider range of organizations.

As AI advances, there should still be plenty of demand for human expertise. For better or worse, the nature of the expertise required, however, will undoubtedly be at a much higher level than it has previously been as more low-level tasks are automated. The challenge is determining how and where to refocus the efforts of the DevOps teams to add higher value.

May 26, 2023
How Test-Driven Methodologies Reduce CI/CD Lead Time
Modern development environments, in which rapid continuous delivery is facilitated by automated continuous integration/continuous development (CI/CD) pipelines, require thorough and automated testing in development prior to integration. This article explains how test-driven development (TDD) and acceptance test-driven development (ATDD) methodologies reduce CI/CD lead times.

Lack of test automation in development impacts CI/CD lead time for subsequent integration, delivery and deployment stages of a CI/CD pipeline in the following ways:
- Increased defect rate: Without automated testing during the development stage, more defects and issues are likely to be introduced into the codebase, increasing the time spent on debugging and fixing issues in later stages.
- Slower feedback loop: Without test automation, the feedback loop between code changes and test results is slower, making it more challenging to identify and resolve issues quickly. This delay can impact the overall lead time in the CI/CD pipeline.
- Reduced confidence in releases: When testing is not performed adequately or isn’t automated, there is less confidence in the quality of the software being released. This may lead to more manual testing and validation, further increasing lead times.
- Inefficient use of resources: Without test automation, developers and testers need to spend more time on manual testing, reducing the time available for developing new features or fixing existing issues.
- Increased merge conflicts: When test automation is not performed early in the development stage, the risk of merge conflicts increases as different developers work on the same codebase. Resolving merge conflicts can be time-consuming and may cause delays in the integration, delivery, and deployment stages.
- Decreased stability and reliability: Inadequate test automation during the development stage can lead to less stable and reliable software. This may require additional time and effort to address issues and ensure that the software is production-ready.
- Inconsistent test environments: Manual testing may not always be conducted in a consistent environment, leading to potential discrepancies between development, staging, and production environments. This can cause unexpected issues and delays in the pipeline.
- Hindered collaboration: Lack of test automation may reduce visibility into the quality of the codebase, making it more difficult for team members from development, QA, security and operations to collaborate effectively and identify issues early.
Although CI/CD pipelines help catch issues and ensure code quality through automated builds and tests, having a strong foundation of test automation during the development stage helps minimize the potential for issues in the pipeline and ensures a smoother CI/CD process.

There are several reasons why many development teams do not automate tests prior to integration, despite the benefits of doing so.
- Lack of awareness or understanding: Some development teams may not be aware of the benefits of test automation or may not fully understand how to implement it effectively in their development process.
- Insufficient skills or experience: Test automation often requires specialized skills and knowledge, and some development teams may lack the necessary expertise to implement it effectively.
- Time constraints: Developing and maintaining automated tests can be time-consuming. Teams working under tight deadlines might prioritize feature development over creating automated tests, believing that it will save time in the short term.
- Limited resources: Test automation tools and infrastructure can require significant investments in terms of cost and resources. Some organizations may not have the necessary budget or resources to invest in test automation, especially for smaller projects or teams.
- Resistance to change: Some teams may be resistant to adopting new practices, particularly if they have been successful with their existing processes. Changing established workflows and learning new tools can be challenging and may encounter resistance from team members.
- Incomplete or rapidly changing requirements: In situations where requirements are unclear or frequently changing, development teams may struggle to create meaningful automated tests, as they may require continuous updates to stay relevant.
- Overemphasis on manual testing: Some teams may place a greater emphasis on manual testing due to a belief that it provides more accurate or thorough results. This perception can lead to a reluctance to invest time and effort into developing automated tests.
- Fear of increased maintenance: Automated tests, like any other code, need maintenance and updates as the application evolves. Some teams may be concerned about the additional effort required to maintain a test suite, particularly if they have limited resources.
The amount of test automation that should be done in development prior to the integration stage of a CI/CD pipeline depends on several factors, including the complexity of the application, team resources and the specific project requirements. However, here are some general guidelines to help you determine the right balance:
- Unit tests: Developers should aim to automate most, if not all, unit tests during the development stage. They are typically fast to execute and provide quick feedback, allowing developers to catch issues early and minimize the number of defects that make it to the integration stage.
- Component and integration tests: While unit tests focus on individual components, component and integration tests verify the interaction between different components or services. These tests should be automated as much as possible to ensure that any issues arising from component interactions are identified and resolved quickly.
- Static code analysis: Automating static code analysis checks during development helps identify code quality issues, security vulnerabilities, and potential performance bottlenecks early in the process.
- Code coverage: Strive for a high level of code coverage with your automated tests. However, it’s important to remember that code coverage is just one indicator of test quality and should be complemented by other testing strategies.
Striking the right balance between automated and manual testing is important. Automate tests that are repetitive, time-consuming or prone to human error. Manual testing should be reserved for exploratory testing, usability testing and other areas where human intuition and judgment are necessary.

Test-driven development (TDD) and acceptance test-driven development (ATDD) are development methodologies that emphasize creating automated tests before writing the actual code, helping teams ensure code quality and catch issues early prior to integration.

Test-driven development (TDD) is a software development methodology that emphasizes writing tests before writing the actual code. The primary goal of TDD is to create clean, maintainable and bug-free code by catching issues early in the development process. TDD follows a short iterative cycle consisting of three main steps: Write a test, make the test pass and refactor the code. The developer starts by writing a test for a specific feature or function, which initially fails. Next, the developer writes the minimal amount of code necessary to make the test pass. Finally, the developer refactors the code to improve its structure and readability while ensuring that all tests still pass.

Acceptance test-driven development (ATDD) is an extension of TDD that focuses on defining and validating the high-level requirements and acceptance criteria for a feature or functionality before development begins. ATDD involves collaboration among developers, testers, and business stakeholders to ensure that the application meets the needs of its users. The process begins with writing acceptance tests, which are derived from the user’s perspective and based on the acceptance criteria. These tests are written in a human-readable language to facilitate clear communication among all team members.

Once the acceptance tests are defined, the development team follows the TDD process to implement the feature, using the acceptance tests as a guide. As the team develops the code, they also create unit tests and other lower-level tests to validate the implementation. The acceptance tests are executed as part of the development process and serve as a measure of progress. When all acceptance tests pass, it indicates that the feature or functionality meets the defined acceptance criteria, and the development task is considered complete. By using ATDD, teams can improve collaboration, ensure that the application meets user expectations and reduce the risk of miscommunication or misunderstanding requirements.

Hurdles to overcome when transitioning to TDD and ATDD include:
- Mindset shift: Adopting TDD and ATDD requires a change in mindset, where testing is considered an integral part of the development process rather than an afterthought.
- Skill development: TDD and ATDD require specialized skills in writing effective tests and implementing test automation. Development teams may need to invest time and resources in acquiring these skills.
- Time and resource allocation: Transitioning to TDD and ATDD might initially lead to a perceived increase in development time, as writing tests before code can seem slower. However, the long-term benefits of improved code quality and reduced debugging time should be considered.
- Collaboration and communication: ATDD relies on effective collaboration and communication among developers, testers, security, operations and business stakeholders. Establishing a culture of open communication and collaboration is crucial.
- Tooling and infrastructure: Implementing TDD and ATDD may require investments in new tools or changes to the existing development infrastructure to support test automation and continuous integration.
In the context of TDD and ATDD, the roles of quality assurance (QA), security and operations teams are crucial for ensuring a well-rounded and robust software development process. Their involvement helps to address various aspects of the application, leading to a more reliable and secure product.
- Quality Assurance (QA): QA team members collaborate closely with developers in both TDD and ATDD methodologies. They can help define test cases, create test plans and provide input on acceptance criteria. In ATDD, they play a particularly significant role in developing and validating acceptance tests, ensuring that the application meets the defined requirements and user expectations. QA team members may also be involved in setting up and maintaining test automation infrastructure, analyzing test results and identifying areas for improvement.
- Security: Security professionals contribute to TDD and ATDD by identifying potential security vulnerabilities and ensuring that the application is developed with security best practices in mind. They can help define security-related test cases and help to embed security into the application, reducing the risk of vulnerabilities and improving the overall security posture.
- Operations: Operations teams are responsible for managing the infrastructure and deployment processes that support the application. In TDD and ATDD, they can collaborate with developers to ensure that the application is designed and developed with operational concerns in mind, such as scalability, performance and maintainability. Their involvement also helps to promote a DevOps culture where development and operations teams work together to ensure the application’s success in production environments.
A practical roadmap for teams to transition to TDD and ATDD while overcoming the hurdles mentioned earlier could look like this:

Build awareness and buy-in:
Provide training and skill development:
Start small and iterate:
Foster a culture of collaboration:
Set up the necessary tooling and infrastructure:
Monitor progress with metrics:
Continuously improve and adapt:
Summary

TDD and ATDD methodologies contribute to reducing CI/CD lead times by automating tests during development. This improves code quality, minimizes defects and promotes collaboration among team members from the outset of the development process. By writing tests before the actual code and focusing on meeting acceptance criteria, teams can catch and resolve issues early, leading to fewer delays and reduced time spent on debugging and fixing errors during the integration, delivery and deployment stages. These practices also help ensure that the application aligns with user expectations and operational requirements, resulting in a smoother and more efficient CI/CD pipeline.

Anyone who is passionate about modern quality engineering processes for building high-quality software to participate is encouraged to join new ShiftSync Community from Tricentis to engage in discussions about this topic and other development methodologies.
May 19, 2023
CDF Report Reveals Lack of DevOps Maturity

A global survey of nearly 26,000 DevOps professionals published today by the Continuous Delivery Foundation (CDF) at the cdCon + GitOpsCon event found 84% of respondents are involved in some type of DevOps activity. However, the report also found just under half 49% are using a continuous integration (CI) or continuous delivery (CD) platform and only 22% are using CI and CD approaches to automate all building, testing and deployment of code in a production environment.

The report, conducted in collaboration with SlashData, found the number of developers using CI or CD platforms only increased two percentage points year-over-year, with the use of self-hosted CI/CD tools dropping from 32% in the first quarter of 2022 to 23% in the same period this year.

The survey also noted that only 7% of respondents this year described themselves as a DevOps engineer or specialist.

Overall, medium-sized businesses have the highest involvement in DevOps practices (94%), followed by small businesses and large enterprises at just over 90%. More than three-quarters of developers (79%) are now using DevOps practices.

The most common DevOps activity is monitoring software or infrastructure performance (47%), followed by application security testing and using CI to build and test code, tied at 37% each, and approving code deployments into production environments at 36%, the survey found.

The average number of DevOps-related technologies used by developers (4.5) has, meanwhile, remained stable over the last year. The most widely used tools address issue tracking (39%), source code management (38%), test automation management (34%), collaboration and knowledge-sharing tools (32%) and application performance management/observability and agile project management tools tied at 31%.

The survey also noted that the percentage of top performers—defined as those with lead times of less than one day—now stands at 15%. Survey results for top performers over the years continue to fluctuate between 13% and 17% of the survey total. On the plus side, 40% of developers have lead time changes of less than one week, the second-highest ranking for this survey.

More than a third of all DevOps practitioners (34%), however, take longer than a week to restore service. Only 12% of DevOps practitioners take less than an hour to restore service.

In general, the survey finds that organizations that use 10 or more DevOps tools are twice as likely to be high performers.

Fatih Degirmenci, executive director for the CDF, told conference attendees the consortium, which now oversees nine projects, including the open source Jenkins CI/CD platform, continues to make progress as additional projects are added.

However, while it’s clear that many organizations have adopted multiple DevOps tools and platforms, far too many of them still have a long DevOps journey ahead. Each organization will need to decide whether DevOps workflows need to be adopted to fit their needs, even as the pressure to build and deliver software faster continues to mount. In fact, at a time when more organizations than ever are dependent on software to drive mission-critical digital business transformation initiatives, time may be running out to adopt DevOps workflows at a deeper level if organizations hope to stay competitive.

May 8, 2023
GitLab Survey Reveals DevSecOps Gains

A global survey of 5,010 IT leaders, CISOs and developers published by GitLab this week found well over half (56%) of respondents are working for organizations that are using DevOps or DevSecOps practices. However, 72% of respondents said they are using a DevSecOps platform (36%) or are considering adopting one in the next year (36%).

Overall, less than a third of survey respondents (30%) said they were “completely” responsible for application security, with 53% reporting they are part of a larger application security team. Well over a third (38%) of the 1,453 security professionals surveyed said they are part of a cross-functional team focused on security.

License compliance checks and security capabilities for cloud-native or serverless tied for the top of the list of current priorities (19%), but shifting security left (29%) was the top focus for the coming year. Nearly three-quarters (74%) of security professionals said their organizations either shifted left or plan to in the next three years. Nearly as many (71%) said a quarter or more of all security vulnerabilities were discovered by developers.

Top frustrations identified specifically by security professionals included testing happening too late in the development cycle (43%) and difficulty prioritizing vulnerability remediation (41%).

Those frustrations might increase in the coming year, with 85% of security professionals reporting they either have the same or lower budget in 2023.

On the plus side, nearly two-thirds of the 1,954 developers surveyed said they are either using artificial intelligence (AI) and machine learning (ML) algorithms in testing today or will be in the next three years. A full 62% said they used AI/ML to check code, while 53% use bots for testing and 36% use AI/ML for code review.

Nevertheless, more than two-thirds of security professionals (67%) said they are concerned about the impact of AI/ML capabilities on their job, with 28% of them admitting they are “very” or “extremely” concerned. Of those respondents who expressed concern, 25% said they are worried about the potential for AI/ML to introduce errors that will make their job more difficult.

David DeSanto, chief product officer at GitLab, said GitLab is also working to address those concerns by embedding a range of AI/ML capabilities into its platform to optimize DevSecOps workflows and, most recently, enhanced its GitLab Remote Development module to make it easier to centralize the maintenance and security of development environments.

GitLab this week also announced it is partnering with Oracle to make it easier to use the GitLab continuous integration/continuous delivery (CI/CD) platform to run AI and ML workloads on the Oracle cloud platforms. AI/ML is going to play a critical role in enabling organizations to build and deploy secure code without slowing down the pace of application development, said DeSanto. In fact, many organizations will soon be consolidating toolchains to enable them to achieve that goal, he added.

The GitLab survey found 57% of security respondents said they use six or more tools, but many of those tools are legacy software composition analysis (SCA) tools that will be replaced as more responsibility for application security shifts left toward developers, noted DeSanto. Two-thirds of survey respondents (66%) reported they want to consolidate their toolchains this year.

Collectively, the GitLab survey makes it clear that DevOps and DevSecOps practices and platforms continue to evolve rapidly. The challenge, as always, is keeping up with the pace of that change.

April 21, 2023