devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

Cognition Labs Previews Devin AI Software Engineer

Cognition Labs, a startup with $21 million in funding, this week previewed what it describes as the world’s first software engineer, dubbed Devin, that is based on artificial intelligence (AI).

Devin is capable of responding to text commands and can be assigned tasks such as assessing the benchmark performance of an application. It will then build a plan and configure the required tools using its own command line interface (CLI), code editor and browser through which it can access, read and comprehend, for example, documentation using a reasoning engine and long-term planning capability based on advances in reinforcement learning.

Those capabilities make it possible for Devin to, for example, build a website, autonomously identify and fix bugs in codebases, deploy applications and even train other AI models.

Cognition Labs said a benchmark evaluation of Devin, using an SWE-bench tool that asks agents to resolve tasks, found Devin correctly resolved 13.86% of the issues end-to-end, far exceeding the previous state-of-the-art of 1.96%. Even when given the exact files to edit, the best previous models could only resolve 4.80% of issues.

Mark Hinkle, CEO of Peripety Labs, a consulting firm, said it’s too early to tell when Devin might be ready for use in enterprise IT environments, but the demonstrations showed how rapid advances in AI are about to transform how software is built and constructed. The user experience is substantially different from that of GitHub Copilot or CodeWhisperer from Amazon Web Services (AWS), which are more focused on helping developers write code rather than executing assigned tasks, he noted.

It’s not clear how much infrastructure is required to run Devin, which, given the ongoing shortage of graphical processor units (GPUs), may prove to be a limiting adoption factor, at least in the short term, added Hinkle.

Regardless of Devin’s current capabilities, which software engineers can only access by invitation right now, it’s clear advances to the reasoning capabilities of AI models will enable DevOps teams to assign tasks to an AI model much like they would any other member of their team. The impact that capability will have on demand for software engineers remains to be seen, but there is still a need to understand what to ask Devin to build and review how the software Devin builds has been constructed. The one thing that is certain is the pace at which software can be built and deployed is about to accelerate.

Undoubtedly, Microsoft, AWS, and other providers of software engineering tools are researching similar capabilities, so it may now be only a matter of time before advances in reasoning and long-term planning are broadly applied to software development.

In the meantime, DevOps teams might want to reassess their strategic plans for the coming years as it becomes easier to build software. Projects that might have required large teams of software engineers to complete might soon be accomplished using much smaller teams. In turn, the number of software development projects that could be launched by organizations large and small will increase as more of the toil historically required continues to be eliminated.

March 16, 2024
CircleCI Adds Release Orchestration Capability to CI/CD Platform

CircleCI today added a release orchestration capability to its namesake continuous integration/ continuous delivery (CI/CD) platform to give developers more control over application deployments.

CircleCI CTO Rob Zuber said a CircleCI releases capability provides development teams with the ability to release a controlled subset of their code in a live production environment in a way that provides immediate feedback on how that code is performing. That level of visibility can then be used to determine whether that code might need to be rolled back using automation tools embedded in the CircleCI CI/CD platform, he added.

In the absence of that capability, noted Zuber, it takes DevOps teams, on average, 45 minutes to an hour to manually roll back an application.

Initial platforms supported by CircleCI releases are Kubernetes and Amazon SageMaker. CircleCI releases is also compatible with the open source Argo CD platform that CircleCI integrated its platform with last year.

CircleCI also plans to add support for blue-green deployments later this year, enabling DevOps teams to deploy one application version more gradually to replace another.

The way CD is managed often varies widely from one organization to another. In highly regulated industries, CD is typically managed by a central team to ensure compliance requirements are met. In other instances, developers are directly responsible for building and deploying their code. Regardless of approach, CircleCI enables organizations to define a set of guardrails to ensure best practices are observed, noted Zuber. In effect, each organization can implement a set of checks and balances that are aligned to their specific needs, he added.

As DevOps continues to evolve, more organizations want the ability to control deployments at a more granular level by, for example, only making a new application or an update available to a select number of users. Once deployed, developers need a more holistic view into not just how that application is performing but also how it impacted the business, said Zuber. If, for example, the number of logins declines after an update, a developer needs the information to determine whether that update should be rolled back as quickly as possible, he added.

It’s not clear how many organizations are managing the entire CI/CD workflow on an end-to-end basis. While most DevOps teams make extensive use of CI to build an application, the deployment of applications is still often managed manually. However, as CD capabilities become simpler to employ across platforms with standard interfaces, the number of organizations automating CD should steadily increase. In fact, that capability will soon prove essential as, thanks to the rise of generative artificial intelligence (AI), the pace at which applications are being built only continues to accelerate.

In the meantime, each organization will need to determine how closely it wishes to integrate CI and CD workflows. However, the more tightly integrated these processes become, the simpler it becomes for developers to assume full responsibility for building and maintaining the software they create.

March 14, 2024
Three DevOps Trends to Watch in 2024

The world of DevOps has evolved at an unprecedented rate in the last few years, yet it continues to be a primary driver of efficiency, innovation, and digital transformation for many organizations. Google’s 2023 Accelerate State of DevOps Report shows that a company’s software delivery performance quality predicts organizational performance, team performance and employee well-being.

In 2024, there are several trends that practitioners can expect to have a major impact on the industry. Forward-thinking tech companies will take advantage of the convergence of these trends as the year progresses. Opportunities exist for leaders in the cloud and virtualization arenas to increase operational flexibility, improve disaster recovery measures and leverage the rapid advancement of new technologies to give their company a competitive edge.

As the year rolls on, here are a few key DevOps trends to watch for in 2024:

1. Apple Silicon Release Cycles

The era of Apple Silicon is now in full force. With Apple now creating its own chips, the company can have complete control over its product development timeline. Additionally, each release is increasingly nuanced in how the hardware/software stack is built. In 2023 alone, we saw the introduction of the new M3 chip series, the completion of Mac’s transition to Apple Silicon among many other new capabilities and innovations. As the use of Apple technology in the enterprise continues to expand, companies that can quickly adapt to the faster release cycles of Apple’s new chipsets will see significant performance improvements.

2. Hybrid and Multi-Cloud Adoption

To compete in today’s dynamic business landscape, harnessing the power of cloud technologies is essential. The industry can expect to see a major surge in hybrid and multi-cloud adoption. In addition to the cost-efficient nature of the cloud, enterprises are increasingly moving towards hybrid and multi-cloud models to enhance operational flexibility and improve disaster recovery measures. In fact, global spending on cloud adoption is projected to top $1 trillion in 2024. As the cloud continues to be a key driver of innovation across industries, the ability to serve and integrate solutions across cloud providers or on-premises is increasingly becoming table stakes. Platforms that can operate seamlessly across these diverse cloud environments will offer a significant competitive advantage.

3. Securing Remote Work and Virtualization Environments

As virtual desktop infrastructure (VDI) continues to take the world by storm, security and compliance across complex environments is crucial. The continued advancement of cyberthreats makes the challenge of securing virtual identities more pertinent than ever. Especially as companies look to mitigate the risks of device and intellectual property theft in offshoring and consulting scenarios, the demand for secure remote desktop technologies has never been higher.

Additionally, security and compliance regulations are becoming increasingly robust. And providers should leave no stone unturned to ensure that their platforms are compliant with all security regulations and consistently audited to address any vulnerabilities that may pop up. Ensuring that the solution provided is living up to its promise especially as it relates to security and compliance is crucial to the public’s perception of your business. Solutions that offer robust security features while also enabling scalability and flexibility in operations will have a competitive edge as we enter the new year.

To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.

March 14, 2024
Flox CLI Tool Makes Managing Custom Developer Environments Simpler

Flox today made generally available an open source command line interface (CLI) for a namesake tool that enables developers to more easily spin up multiple custom development environments.

At the same time, the company revealed at the 21st annual Southern California Linux Expo (SCALE 21) that it is now providing access to FloxHub, a free repository for sharing development environments across teams.

Flox is based on the same declarative framework used to invoke the Nix package management and configuration tool. It allows developers to create environments that contain everything they need to build software, including automatic creation of a software bill of materials (SBOM).

Ross Turk, head of marketing and developer relations for Flox, said that approach provides developers with a framework for managing multiple development environments that they can spin up using the same methods commonly used to invoke the package managers versus having to master a less familiar approach to manage development environments, he added.

That capability makes it simpler for developers to both replicate development environments across machines and, when needed, dynamically switch between projects in a way that can be continuously audited by a centralized DevOps team. Instead of requiring a DevOps team to apply the same rigid standard environment across all development projects, each developer is still free to customize their environment based on their personal preferences.

In addition, developers trying to fix a bug can also now easily reproduce the development environment they were using at the time they were writing the code they needed to review.

DevOps teams are increasingly finding themselves caught between competing agendas. Many IT leaders are driving the adoption of platform engineering methodologies to centralize the management of DevOps workflows in the hopes of increasing developer productivity and reducing total costs. Conceptually, developers would prefer to have more backend processes managed by those teams, but not to the point where it forces them to use a development environment that has been defined for them. Each developer typically prefers to customize their development environment as the project evolves.

Arguably, one of the primary reasons developers embrace DevOps is that it gives them more control over their development environments. The challenge is that as each team defines its own DevOps workflow, organizations often later discover they are licensing redundant DevOps tools and platforms.

In the meantime, it’s not clear to what degree platform engineering might boost developer productivity. It no doubt will reduce the amount of time developers spend managing backend processes, but having that extra time doesn’t directly equate to better code being delivered faster. Writing code is as much an art as science, so it creates levels of inspiration that don’t always occur whenever a developer starts to write code. Instead, platform engineering creates more opportunities for inspiration to strike because they are not spending time on other tasks. At the very least, it should provide more time to run tests on the code that is produced.

One way or another, as application development continues to evolve, most organizations will find the middle ground that best suits their needs. The challenge now is reducing the amount of time between now and when that goal is eventually achieved.

March 13, 2024
The IaC Weight on DevOps’ Shoulders
It’s often challenging for established tech companies to adopt modern DevOps practices around infrastructure-as-code (IaC). It’s not so much in adopting the syntax, but rather the DevOps culture that IaC is part of for engineering organizations.

Organizations often struggle to transition from traditional operations models to collaborative DevOps culture, which is a byproduct of transitioning infrastructure operations to code.

In many cases, legacy tendencies persist, while IaC duties stay siloed within ops-centric DevOps teams—which is the exact opposite of the benefits of leveraging powerful technology like infrastructure-as-code. What this ultimately creates is bottlenecks in development velocity and product iteration.

With all this in mind, it begs the question: What changes can enable a smoother IaC journey?

The Pitfalls of IaC Silos

Having DevOps teams shoulder the entire IaC burden is the exact anti-pattern we’re looking to avoid when truly transitioning to DevOps culture in our organizations. What many organizations start to experience is these familiar pain points that start to emerge:

1.) Knowledge Gaps Widen

Developers—by nature of their day job as software engineers—often lack access, visibility and context into infrastructure changes. Yet, with more and more infrastructure requirements shifting left, they are left (pun intended) with insufficient capabilities to manage these growing needs.

This is compounded when they are faced with minimal cloud permissions. Limited permissions make it difficult to experiment and familiarize themselves with cloud tools and capabilities the way they would with code and its parts of the stack.

Meanwhile, DevOps teams—as the exact mirror image and the nature of their role in the organization—don’t grasp application needs and objectives. This often results in a widening chasm of broken communication and collaboration around IaC initiatives.

2.) Collaboration Suffers

A common tool that exacerbates the problem is a ticketing and support platform. Developers, as their modus operandi, lacking a shared understanding of the domain, will often resort to opening tickets to achieve their goals. This creates a culture of inefficient cross-team communication via ticketing systems.

This doesn’t mean the ticketing platforms are the problem, it just results in inefficient communication methods, as developers don’t always know what questions to ask.

What this does mean is that DevOps engineers operate largely on instinct, being mostly in the dark on the real application requirements that can’t all be communicated in a simple Jira ticket. This results in final solutions that rarely satisfy needs on the first attempt and take longer to deliver.

3.) Releases Slow to a Crawl

DevOps teams hold the responsibility for conducting and orchestrating release engineering, on top of provisioning environments, maintenance, monitoring and other tasks that fall in the DevOps engineering domain.

This is why DevOps tasks can become a bottleneck, as they not only dictate deployment timing (due to important considerations of uptime and client disruption, among others) but are also tasked with juggling infrastructure changes across many and multiple environments.

This causes unpredictable delays in provisioning dev environments and other developer-oriented tasks that frustrate developers and stall production updates, and delivery velocity.

The Obstacles of Change

Recognizing these inefficiencies and shifting these responsibilities left became the popular way for DevOps leaders to distribute IaC duties. This allowed developers to gain sufficient autonomy to choose the solutions for their needs, while DevOps teams provided guidance and oversight for how to practically achieve these solutions.

However, like all things shift left, this comes with its own set of unique challenges. Adopting the DevOps paradigm faces fundamental obstacles that add further complications and complexity to these cultural transitions, including:
- Uncontrolled sprawl
- Ballooning costs
- Security concerns
On the one hand, distributing the IaC load lessens the burden on the DevOps teams, but the downside is that it becomes difficult to understand which resources are actually in use and which have been temporarily created for testing purposes.

With many owners creating resources on demand, once they are no longer needed, these leftovers create confusion around dependencies and make cloud platforms disorganized and difficult to maintain.

Just like enabling more hands to touch IaC creates greater sprawl and disorder, more users with less governance invite careless sprawl in terms of costs as well. This often results in duplicate and unused resources accumulating, wasting budgets that are currently tight, and every penny counts. With a lack of automation and oversight, environments grow messy and expensive.

The sprawl issues can also impact security, as expanding permissions raises valid security concerns that are intensified when clouds become disorganized and difficult to maintain.

Well-intentioned developers may misconfigure resources or expose sensitive systems, and without proper methods to manage drift or misconfiguration, this can pose real risks to organizations and systems. Another important aspect that also increases with less oversight is intentional insider risk.

From Silos to Shared Responsibility

OK, that must have been really disheartening—and you’re probably thinking you should run for the hills before attempting any transitions to shared responsibility.

But wait—don’t panic yet. It is possible for teams to redistribute IaC responsibility while avoiding these pitfalls.

Below are some key factors to help you make the transition smoother in the short and the long term:
- Automate everything—If DevOps practices have taught us anything, it is that manual cloud environment maintenance doesn’t scale. Automation has been the backbone of much of the evolution we have undergone when it comes to infrastructure operations, and this has been powered by the programmability and everything-as-code mindset—from IaC to policy-as-code (PaC). This automation-based optimization makes it possible to reduce neglected resources through regular and automated analysis and response. What you get is cost savings and the prevention of runaway sprawl.
- Empower through governance—Remember when we spoke about automation and governance? Well, programmatic policy enforcement is a good way to constrain unwanted resource proliferation without manual intervention. Governance comes in to handle auto-remediation of misconfigurations or other anti-patterns that do not align with automated policies, which simplifies environment management at scale. Importantly, governance, if done right, can be a tool to empower teams, break silos and encourage autonomy. Having the right guardrails in place is how you instill confidence and provide helpful direction.
- RBAC for the win—Role-based access control (RBAC) limits permissions to appropriate needs, establishing important guardrails that can help prevent some of the issues we spoke about above, including sprawl and security concerns.
RBAC will enable developers to gain the necessary freedom while minimizing security risk. Minimum required access encourages responsibility and constraint.

Now that you have a solid grasp of the principles, it’s time to put them into action!

Fortunately, there are numerous solutions at your disposal. You can jump in and build them yourself using capable tools like Jenkins or Atlantis.

Alternatively, for a more comprehensive approach, there are excellent commercial IaC management tools on the market. These can assist you in reaching your objectives swiftly and with greater efficiency.

An Evolutionary Vision for IaC and DevOps

As technology complexity increases, no single team can handle infrastructure alone. Dev and Ops must unite abilities to shepherd modern systems and the underlying code powering them. This requires closing knowledge gaps through collaboration and communication

With a thoughtful process, IaC automation, governance guardrails and the right access policies, tech leaders can guide this cultural transition smoothly without exposing the organization to unnecessary risk and spending.

When done right, Shared IaC responsibility will boost velocity, agility, and autonomy—the trifecta of crucial traits that quantify and qualify elite engineering teams and differentiate companies in a competitive landscape.

Interested in learning more about env0 to manage your IaC at scale and confidently?

Stop by Booth L12 at KubeCon EU to speak with an expert or see a demo!

To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.
March 13, 2024
Applying AI/ML to Continuous Testing

Artificial intelligence (AI) and machine learning (ML) can play a transformative role across the software development lifecycle, with a special focus on enhancing continuous testing (CT). CT is especially critical in the context of continuous integration/continuous deployment (CI/CD) pipelines, where the need for speed and efficiency must be balanced with the demands for quality and security. AI/ML contributes by automating complex tasks, predicting potential issues before they occur and providing actionable insights, thereby reducing manual efforts and enabling more strategic use of human resources.
Moreover, the application of AI/ML extends beyond mere automation of tests. It encompasses the capability to learn from data, adapt to new information, and improve over time. This ability is invaluable for identifying patterns, anticipating vulnerabilities and optimizing testing strategies. In quality assurance, AI-driven tools can predict areas most likely to fail and tailor testing efforts accordingly. In security, ML algorithms can detect anomalies that signify potential threats, while in operations, AI can enhance feedback mechanisms, leading to more resilient and responsive systems.

AI/ML for Continuous Testing

Here’s how AI technologies can be applied to reduce bottlenecks associated with testing activities across various testing activities:

1. Requirements Analysis
• Explanation: Ensures test scenarios align with business requirements and user needs.
• Bottleneck: Misinterpretation or incomplete analysis can lead to inadequate test coverage.
• AI/ML Solution: NLP can automate the extraction and interpretation of requirements, ensuring comprehensive and accurate test coverage.

2. Test Strategy
• Explanation: Outlines the testing approach, objectives, and resources.
• Bottleneck: An unclear strategy may lead to inefficient testing efforts and resource allocation.
• AI/ML Solution: AI can analyze historical data to suggest the most effective test strategies and predict resource needs.

3. Test Plans
• Explanation: Detailed documents guiding the testing process, timelines, and responsibilities.
• Bottleneck: Inflexible plans can struggle to adapt to project changes, causing delays.
• AI/ML Solution: Machine learning algorithms can suggest adjustments to test plans based on ongoing project developments and past outcomes.

4. Test Cases
• Explanation: Specific conditions under which a test is executed.
• Bottleneck: Time-consuming development and significant effort to maintain.
• AI/ML Solution: AI can automate the generation of test cases from requirements documents, improving efficiency and coverage.

5. Test Scripts
• Explanation: Automated scripts that execute test cases.
• Bottleneck: Script development and maintenance can be resource-intensive.
• AI/ML Solution: AI can generate and update test scripts based on changes in the application or test cases, reducing maintenance effort.

6. Test Data
• Explanation: Data sets used during testing to simulate real-world scenarios.
• Bottleneck: Creating, managing, and maintaining accurate test data is challenging.
• AI/ML Solution: AI can automate the generation and management of test data, ensuring relevance and variety.

7. Test Environment
• Explanation: The setup where testing is conducted mirrors production environments as closely as possible.
• Bottleneck: Configuration and maintenance of test environments are complex.
• AI/ML Solution: AI can predict and configure optimal test environments based on test requirements, reducing setup time.

8. Coordination with Dependent Systems
• Explanation: Ensuring the system under test interacts correctly with databases and other applications.
• Bottleneck: Dependency management can cause delays.
• AI/ML Solution: AI can automate the detection and resolution of integration issues, enhancing coordination efficiency.

9. Test Environment Setup
• Explanation: Configuring infrastructure and tools necessary for testing.
• Bottleneck: Setup complexity and resource contention lead to delays.
• AI/ML Solution: AI algorithms can optimize environment setup, automatically adjusting resources as needed.

10. Test Campaign Setup
• Explanation: Organizing and scheduling a series of test executions.
• Bottleneck: Requires careful planning and can be hindered by resource limitations.
• AI/ML Solution: AI can assist in scheduling and prioritizing test campaigns based on risk and impact analysis.

11. Test Execution
• Explanation: The process of running test cases and scripts, both automated and manual.
• Bottleneck: Time-consuming, particularly for manual tests.
• AI/ML Solution: AI can prioritize test execution and identify flaky tests, streamlining the process.

12. Test Verdict Reporting
• Explanation: Determining and reporting the outcome of test executions.
• Bottleneck: Manual verdict determination can be slow.
• AI/ML Solution: AI can automatically interpret test outcomes, speeding up reporting.

13. Logging of Data
• Explanation: Recording data relevant to the test for further analysis.
• Bottleneck: Extensive data collection can overwhelm resources.
• AI/ML Solution: AI can intelligently filter and log pertinent data, reducing noise.

14. Test Result Analysis
• Explanation: Analyzing test outcomes to identify defects and issues.
• Bottleneck: Requires significant time and expertise.
• AI/ML Solution: ML algorithms can quickly identify patterns and anomalies in test results, highlighting potential issues.

15. Test Result Reporting
• Explanation: Communicating findings to stakeholders.
• Bottleneck: Compiling reports is time-intensive.
• AI/ML Solution: Automated reporting tools powered by AI can generate insightful and comprehensive reports quickly.

16. Waiting for Resource to Fix Failed Tests
• Explanation: Downtime while awaiting fixes for identified issues.
• Bottleneck: Halts testing progress.
• AI/ML Solution: AI can predict which areas might fail and propose potential fixes

Challenges

Applying AI/ML to software testing activities offers numerous advantages but also introduces several challenges. Addressing these problems requires a combination of technical solutions, process adjustments, and cultural changes.

1. Lack of Intuitive Understanding of the Application Being Tested
• Problem: AI/ML models may not fully grasp the application’s context or the nuances of its functionalities, leading to less effective test scenarios.
• Solution: Enhance AI models with richer contextual data and incorporate feedback loops where testers can refine and adjust AI-generated test cases. Employing techniques like reinforcement learning can also help AI models better understand application contexts over time.

2. Repeatability and Consistency Between Test Sessions
• Problem: AI-driven tests may generate different outputs for the same input over different sessions, complicating test consistency and traceability.
• Solution: Implement versioning for AI models and their training data, ensuring consistency across test sessions. Use deterministic approaches in conjunction with AI to maintain a core of stable, repeatable tests.

3. Lack of Understanding of the Tests Generated
• Problem: Testers may find it challenging to understand or trust the rationale behind AI-generated test cases, impacting their ability to evaluate test outcomes effectively.
• Solution: Incorporate explanations into AI/ML models to provide insights into their decision-making processes. Foster a culture of trust and understanding through education and transparency regarding how AI models operate.

4. Test Coverage
• Problem: There’s a risk that AI/ML might not adequately cover all testing scenarios, potentially missing critical defects.
• Solution: Combine AI/ML with traditional testing methods to ensure comprehensive coverage. Regularly review and adjust the criteria used by AI/ML models to generate test cases, ensuring they align with evolving application features and risks.

5. Compatibility with Different Test Tools
• Problem: AI/ML models might not seamlessly integrate with existing testing tools and frameworks, limiting their utility.
• Solution: Develop or use AI/ML solutions with extensive API support and integration capabilities. Work with tool vendors or contribute to open source projects to enhance compatibility.

6. Acceptance by Teams
• Problem: Testers and developers may be skeptical or resistant to AI-driven testing due to concerns about job displacement or mistrust in AI’s effectiveness.
• Solution: Educate and involve teams in the development and implementation of AI/ML testing strategies. Demonstrate the value of AI/ML in augmenting their roles rather than replacing them, focusing on AI as a tool to tackle mundane tasks and allowing them to focus on more complex and rewarding work.

7. Data Quality and Availability
• Problem: AI/ML models require large amounts of high-quality data for training. Inadequate or poor-quality data can lead to ineffective testing.
• Solution: Invest in data curation and generation strategies such as synthetic data creation to ensure models are well-trained.

8. Continuous Learning and Adaptation
• Problem: AI/ML models may become outdated as applications evolve.
• Solution: Establish continuous learning mechanisms where models are regularly updated with new data and feedback, ensuring they remain relevant and effective.

9. Ethical and Bias Considerations
• Problem: AI/ML testing models might inherit or amplify biases present in their training data, leading to unfair or discriminatory outcomes.
• Solution: Implement ethical guidelines and bias detection methodologies for AI/ML model development and use. Regularly audit models for biases and correct them as needed.

By addressing these challenges with thoughtful strategies, organizations can maximize the benefits of AI/ML in testing activities while mitigating potential drawbacks, leading to more efficient, effective and trustworthy processes.

Summary

This blog explained use cases of artificial intelligence (AI) and machine learning (ML) within the realm of continuous testing of the software development life cycle. By harnessing the power of AI/ML, we’ve seen how organizations can significantly enhance their development processes, making them more efficient, secure and responsive to user needs. The exploration of specific AI/ML applications across various activities has provided a blueprint for automating and optimizing tasks that traditionally required significant manual effort.

March 12, 2024
What is RISC-V and Why Has it Become Important for Java?

RISC stands for reduced instruction set computer, and V points to its fifth release in 2015. RISC-V is the new processor architecture to watch out for. Arm-based processors, which stayed among the most popular ones in recent years, may capture a significant share of the semiconductor market, and this is why.

Arm and RISC-V are a type of processor architecture. Another well-known processor architecture type is x86.

Arm and RISC-V chips are used by various industries, from tablets to servers, but initially, they targeted the embedded market. Their design specifics allow them to offer sufficient performance while keeping power consumption low.

RISC-V is a novel open source instruction set architecture (ISA) based on the reduced instruction set computer (RISC) model. The project started at the University of California, Berkeley and was taken over by the RISC-V Foundation in 2015.

Using the x86 and arm processors for hardware development implies high fees because you need to pay royalties to the providers of processor IP, making production expensive and delivering bureaucratic complications. The RISC-V project brought liberation in this sense, creating a new model based on the open-source approach, allowing users to use these processors free of charge. In short, RISC-V is getting more and more popular thanks to its improved design and open-source approach to its development.

RISC-V consists of a set of basic instructions. These instructions act as a communication bridge between hardware and software. RISC-V includes 47 instructions, and additional extensions can be added in a modular way. Thanks to that, RISC-V is smaller compared to x86 or ARM.

In contrast to other popular processors, RISC-V is open source, and therefore, anyone can design, manufacture and sell RISC-V chips and software. As such, RISC-V established new opportunities for modern computing devices, from mobile phones to supercomputers and even microchips for the space industry. Space industry technologies are very challenging as they operate in very different environmental factors. Consequently, the recent NASA announcement on selecting SiFive RISC-V as the core CPU for the high-performance spaceflight computing (HPSC) processor is a great example of the power of RISC-V architecture.

RISC-V belongs to so-called “disruptive technologies.” It removes drawbacks of older systems, eliminating complexity together with licensing fees,, as a result delivering new levels of software flexibility and performance and reducing costs of application development.

According to the RISC-V market report by SHD Group, RISC-V SoCs are forecasted to reach $92.7 billion by 2030, representing a 47% annual growth rate. So, we are still in the early stages of adaptation. Considering that RISC-V can be used as a key ingredient for anything from a smartphone chip to advanced processors for artificial intelligence, its market share could grow very quickly.

It should be noted that Asia is a step ahead in using RISC-V architecture today, with Huawei and Alibaba leading the game. But we also see increasing involvement from companies such as Google, IBM and others. At the same time, the early adoption of RISC-V has already caused a political debate on the microchip trade policy between the U.S. and China.

Java on RISC V

Java has always been at the top of the list for enterprise coding languages, and it is also one of the most widespread languages for embedded systems.

However, applying Java to embedded systems requires adapting it to their technical requirements, which is why Oracle and several OpenJDK vendors have created a special edition of Java for embedded systems. Java runtime for embedded should be compiled into the native ISA for the target hardware, supported by highly tailored, relatively small runtime packages, able to execute on multiple processors if needed for capacity expansion.

RISC-V presents a new type of hardware and needs an adapted version of Java too. So, the Java community initiated the RISC-V port, which was successfully integrated into JDK 19.

However, even with a port in place, you still need to build and test the JDK binary correctly to use it with RISC-V. Today, many OpenJDK vendors and Linux distributions look to offer packages for RISC-V, and gradually, most of the vendors will be delivering JDK packages adapted to RISC-V. But it is great to be among the first. BellSoft released Liberica JDK builds for Linux on RISC-V for JDK 21 LTS. BellSoft commits to supporting version 21 until 2032, providing users with all the benefits of an open source Java runtime from a leading OpenJDK contributor. This includes quarterly CPU and PSU updates and additional solutions for Java development. The RISC-V support in Liberica JDK comes with a Standard JDK flavor, offering three VMs: Server VM, Client VM and Minimal VM designed specifically for systems with lower performance so that your Java applications start up faster and utilize less memory.

OpenJDK and RISC-V are both open source projects; Liberica JDK is free for personal and commercial use. The BellSoft team is glad to continue contributing to the secure and easy development process of Java applications and allowing early access to evolving technologies.

RISC-V is changing the hardware manufacturing market and requires the closest attention from software developers too. RISC-V delivers a sustainable approach to development, removing unnecessary barriers, providing new levels of flexibility and scalability and, as a result, reducing the cost of chip production. RISC-V combined with correct application development practice can completely change the tech future.

March 8, 2024
appCD Launches Platform to Securely Provision Cloud Infrastructure

appCD this week launched a namesake platform that analyzes an application about to be deployed to automatically generate the code needed to provision the IT infrastructure required.

Fresh from raising $6 million in seed funding, appCD CEO Sachin Aggarwal said that approach makes it possible to consistently create secure code for provisioning IT infrastructure versus requiring developers with little to no cybersecurity to manually write it.

Instead, appCD reviews Python or Java code to understand intent, identifies dependencies and then infers application programming interface (API), service configuration, ingress/egress and other environment variables. Terraform code or Helm charts are then automatically generated based on the static analysis of the code that has been reinforced using artificial intelligence (AI) learning techniques.

That approach also makes it simpler for platform engineering teams to centrally manage the provisioning of IT infrastructure using a set of pre-defined policies that are automatically implemented each time an application is deployed to ensure security by default, he added.

DevOps teams can also use appCD to visualize the deployment environment before and after an application is deployed, said Aggarwal.

While infrastructure-as-code (IaC) tools have accelerated the rate at which cloud infrastructure can be provisioned, they are also often the root cause of the security issues that plague cloud computing environments. Developers often misconfigure cloud infrastructure by, for example, inadvertently leaving ports open. The appCD platform reduces the potential such mistakes will be made, noted Aggarwal.

That capability also eliminates the need to attach cybersecurity specialists to a DevOps team because issues are automatically resolved by the guardrails enforced by appCD, he added.

In theory, at least, there may soon come a day when AI routinely improves the quality of the code being used to both build and deploy applications. The more the code used to train a large language model (LLM) is vetted, the more reliable the output becomes. In the meantime, platforms such as appCD are combining static analysis with AI reinforcement learning to achieve that same goal in a use case that is specific to generating code to provision IT infrastructure.

Each DevOps team will naturally need to determine the best path forward, but as regulations pertaining to securing software supply chains become more stringent, it’s only a matter of time before organizations will be required to embrace DevSecOps workflows. In the meantime, DevOps teams should assume that given the probability cloud infrastructure has been misconfigured, it’s all but certain there will be multiple cloud security incidents. In fact, cybercriminals are actively scanning for misconfigurations that they have already created playbooks to specifically exploit in a matter of minutes.

In the absence of being able to prevent all those issues from arising, the immediate priority becomes remediating as many issues as possible before cybercriminals exploit those vulnerabilities. However, it’s also important to draw a proverbial line in the sand to ensure that all cloud infrastructure being provisioned in the future is much more secure than it has tended to be thus far.

March 7, 2024
The Ultimate DORA DevOps Guide: Boost Your Dev Efficiency With DORA Metrics

Imagine having a powerful tool that measures your software team’s efficiency, identifies areas for improvement and unlocks the secrets to achieving speed and stability in software development – that tool is DORA metrics.

DORA metrics offer valuable insights into the effectiveness and productivity of your team. By implementing these metrics, you can enhance your dev practices and improve outcomes.

In this blog, we will delve into the importance of DORA metrics for your team and explore how they can positively impact your software team’s processes. Join us as we navigate the significance of these metrics and uncover their potential to drive success in your team’s endeavors.

What are DORA Metrics?

Software teams use DORA metrics in an organization to help improve their efficiency and, as a result, enhance the effectiveness of company deliverables. It is the industry standard for evaluating dev teams and allows them to scale.

The metrics include deployment frequency, lead time for changes, mean time to recovery, and change failure rate. They have been identified after six years of research and surveys by the DORA (DevOps Research and Assessments) team.

To achieve success with DORA metrics, it is crucial to understand them and learn the importance of each metric. Here are the four key DORA metrics

The Four DORA Metrics

1. Deployment Frequency: Boosting Agility

Organizations need to prioritize code deployment frequency to achieve success and deliver value to end users. However, it’s worth noting that what constitutes a successful deployment frequency may vary from organization to organization.

Teams that underperform may only deploy monthly or once every few months, whereas high-performing teams deploy more frequently. It’s crucial to continuously develop and improve to ensure faster delivery and consistent feedback. If a team needs to catch up, implementing more automated processes to test and validate new code can help reduce recovery time from errors.

Why is deployment frequency important?

● Continuous delivery enables faster software changes and quicker response to market demands.
● Frequent deployments provide valuable user feedback for improving software efficiently.
● Deploy smaller releases frequently to minimize risk. This approach reduces the impact of potential failures and makes it easier to isolate issues. Taking small steps ensures better control and avoids risking everything.
● Frequent deployments support agile development by enabling quick adaptation to market changes and facilitating continuous learning for faster innovation.
● Frequent deployments promote collaboration between teams, leading to better outcomes and more successful projects.

Use Case:

In a dynamic market, agility is paramount. Deployment frequency measures how frequently code is deployed. Infrequent deployments can cause you to lag behind competitors. Increasing deployment frequency facilitates more frequent rollouts, hence, meeting customer demands effectively.

2. Lead Time for Changes: Streamline Development

The time it takes to implement changes and deploy them to production directly impacts their experience, and this is the lead time for changes.

If we notice longer lead times, which can take weeks, it may indicate that you need to improve the development or deployment pipeline. However, if you can achieve lead times of around 15 minutes, you can be sure of an efficient process. It’s essential to monitor delivery cycles closely and continuously work towards streamlining the process to deliver the best experience for customers.

Why is the lead time for changes important?

● Short lead times in software development are crucial for success in today’s business environment. By delivering changes rapidly, organizations can seize new opportunities, stay ahead of competitors and generate more revenue.
● Short lead times help organizations gather feedback and validate assumptions quickly, leading to informed decision-making and aligning software development with customer needs. Being customer-centric is critical for success in today’s competitive world, and feedback loops play a vital role in achieving this.
● By reducing lead time, organizations gain agility and adaptability, allowing them to swiftly respond to market changes, embrace new technologies and meet evolving business needs. Shorter lead times enable experimentation, learning, and continuous improvement, empowering organizations to stay competitive in dynamic environments.
● Reducing lead time demands collaborative teamwork, breaking silos, fostering shared ownership and improving communication, coordination and efficiency.

Use Case:

Picture your software development team tasked with a critical security patch. Measuring lead time for changes helps pinpoint the duration from code commit to deployment. If it goes for a long run, bottlenecks in your CI/CD pipeline or testing processes might surface. Streamlining these areas ensures rapid responses to urgent tasks.

3. Change Failure Rate: Ensuring stability

The change failure rate measures the code quality released to production during software deployments. Achieving a lower failure rate than 0-15% for high-performing dev teams is a compelling goal that drives continuous improvement in skills and processes. Establishing failure boundaries tailored to your organization’s needs and committing to reducing the failure rate is essential. By doing so, you enhance your software solutions and deliver exceptional user experiences.

Why is change failure rate important?

● It enhances user experience and builds trust by reducing failures; we elevate satisfaction and cultivate lasting positive relationships.
● It protects your business from financial risks, and you avoid revenue loss, customer churn, and brand damage by reducing failures.
● Reduce change failures to allocate resources effectively and focus on delivering new features.

Use Case:

Stability is pivotal in software deployment. The change failure rate measures the percentage of changes that fail. A high failure rate could signify inadequate testing or insufficient quality control. Enhancing testing protocols, refining code reviews and ensuring thorough documentation can reduce the failure rate, enhancing overall stability.

4. Mean Time to Recover (MTTR): Minimizing Downtime

Mean time to recover (MTTR) measures the time to recover a system or service after an incident or failure in production. It evaluates the efficiency of incident response and recovery processes. Optimizing MTTR aims to minimize downtime by resolving incidents through production changes. The goal is to build robust systems that can detect, diagnose, and rectify problems. Organizations ensure minimal disruption and work towards continuous improvement in incident resolution.

Why is mean time to recover important?

● Minimizing MTTR enhances user satisfaction by reducing downtime and resolution times.
● Reducing MTTR mitigates the negative impacts of downtime on business operations, including financial losses, missed opportunities and reputational damage.
● Helps meet service level agreements (SLAs) that are vital for upholding client trust and fulfilling contractual commitments.

Use Case:

Downtime can be detrimental, impacting revenue and customer trust. MTTR measures the time taken to recover from a failure. A high MTTR indicates inefficiencies in issue identification and resolution. Investing in automation, refining monitoring systems and bolstering incident response protocols minimizes downtime, ensuring uninterrupted services.

Key Use Cases

Development Cycle Efficiency:

Metrics: Lead time for changes and deployment frequency, high deployment frequency, swift lead time:

Teams with rapid deployment frequency and short lead time exhibit agile development practices. These efficient processes lead to quick feature releases and bug fixes, ensuring dynamic software development aligned with market demands and ultimately enhancing customer satisfaction.

Low deployment frequency despite swift lead time:

A short lead time coupled with infrequent deployments signals potential bottlenecks. Identifying these bottlenecks is vital. Streamlining deployment processes in line with development speed is essential for a software development process.

Code review excellence:

Metrics: Comments per PR and change failure rate, few comments per PR, low change failure rate:

Low comments and minimal deployment failures signify high-quality initial code submissions. This scenario highlights exceptional collaboration and communication within the team, resulting in stable deployments and satisfied end-users.

Abundant comments per PR, minimal change failure rate:

Teams with numerous comments per PR and a few deployment issues showcase meticulous review processes. Investigating these instances ensures review comments align with deployment stability concerns, ensuring constructive feedback leads to refined code.

Developer Responsiveness:

Metrics: Commits after PR review and deployment frequency, frequent commits after PR review, high deployment frequency:

Rapid post-review commits and a high deployment frequency reflect agile responsiveness to feedback. This iterative approach, driven by quick feedback incorporation, yields reliable releases, fostering customer trust and satisfaction.

Sparse commits after PR review, high deployment frequency:

Despite few post-review commits, high deployment frequency signals comprehensive pre-submission feedback integration. Emphasizing thorough code reviews assures stable deployments, showcasing the team’s commitment to quality.

Quality Deployments:

Metrics: Change failure rate and mean time to recovery (MTTR), low change failure rate, swift MTTR:

Low deployment failures and a short recovery time exemplify quality deployments and efficient incident response. Robust testing and a prepared incident response strategy minimize downtime, ensuring high-quality releases and exceptional user experiences.

High change failure rate, rapid MTTR:

A high failure rate alongside swift recovery signifies a team adept at identifying and rectifying deployment issues promptly. Rapid responses minimize impact, allowing quick recovery and valuable learning from failures, strengthening the team’s resilience.

Code Collaboration Efficiency:

Metrics: comments per PR and commits after PR is raised for review

In collaborative software development, optimizing code collaboration efficiency is paramount. By analyzing comments per PR (reflecting review depth) alongside commits.

After PR is raised for review, teams gain crucial insights into their code review processes.

High comments per PR, low post-review commits:

Thorough reviews with limited code revisions post-feedback indicate a need for iterative development. Encouraging developers to iterate fosters a culture of continuous improvement, driving efficiency and learning.

Low comments per PR, high post-review commits:

Few comments during reviews paired with significant post-review commits highlight the necessity for robust initial reviews. Proactive engagement during the initial phase reduces revisions later, expediting the development cycle.

Impact of PR Size on Deployment:

Metrics: Large PR size and deployment frequency

The size of pull requests (PRs) profoundly influences deployment timelines. Correlating large PR size with deployment frequency enables teams to gauge the effect of extensive code changes on release cycles.

High deployment frequency despite large PR size:

Maintaining a high deployment frequency with substantial PRs underscores effective testing and automation. Acknowledge this efficiency while monitoring potential code intricacies, ensuring stability amid complexity.

Low deployment frequency with large PR size:

Infrequent deployments with large PRs might signal challenges in testing or review processes. Dividing large tasks into manageable portions accelerates deployments, addressing potential bottlenecks effectively.

PR Size and Code Quality:

Metrics: Large PR size and change failure rate

PR size significantly influences code quality and stability. Analyzing Large PR size alongside change failure rate allows engineering leaders to assess the link between PR complexity and deployment stability.

High change failure rate with large PR size:

Frequent deployment failures with extensive PRs indicate the need for rigorous testing and validation. Encourage breaking down large changes into testable units, bolstering stability and confidence in deployments.

Low change failure rate despite large PR size:

A minimal failure rate with substantial PRs signifies robust testing practices. Focus on clear team communication to ensure everyone comprehends the implications of significant code changes, sustaining a stable development environment.

Leveraging these correlations empowers engineering teams to make informed, data-driven decisions — a great way to drive business outcomes— optimizing workflows, and boosting overall efficiency. These insights chart a course for continuous improvement, nurturing a culture of collaboration, quality and agility in software development endeavors.

Help Your Team With DORA Metrics!

In the ever-evolving world of software development, harnessing the power of DORA metrics is a game-changer. By leveraging them, your software team can achieve remarkable results. These metrics are vital to enhancing user satisfaction, mitigating financial risks, meeting service-level agreements and delivering exceptional software solutions.

March 7, 2024
7 API Observability Anti-Patterns to Avoid
Observability has become a buzzword, and the API world is slowly catching up. But be careful when looking for content about API observability- there are many outdated best practices and irrelevant pieces of content.

I was recently invited as a guest to Stephen Townshend’s SlightReliability podcast, and his brilliant blog post on “Bad Observability” inspired me to start ranting about the topic.

Here is a follow-up to our discussion: The observability anti-pattern to avoid when working with APIs. Thanks, Stephen, for the inspiration!

Anti-Pattern: You Forgot Your Users

It’s easy to lose sight of the fact that APIs are built for people to use. APIs are not just a technical interface; they are a bridge that connects digital experiences and enables people to accomplish tasks efficiently.

There are two different kinds of users for your API:
- The developers who are integrating your API into their system.
- Your customers’ users, who will ultimately interact with the applications or services that rely on your API.
If you can’t understand how they interact with your APIs, you’re missing half the picture.

When developers integrate with your APIs, how long does it take them to make their first successful API call? What are typical errors slowing them down? Observability plays an essential role in surfacing insights to improve developer experience.

Once your customers’ applications or services relying on your APIs are in production, can you track how your releases impacted their requests? Which version of your APIs are they using? What are the most popular endpoints? Are all customers getting the same performance? Same error rates?

Anti-Pattern: Relying Only on API Monitoring

API monitoring typically involves sending automated test requests to API endpoints and verifying the responses. It helps you keep track of the health of your APIs even when there is no customer activity at all, and can be used to report uptime.

But API monitoring is nothing more than black box testing; you don’t see what is happening inside. You just know that your requests reach the API and receive responses, but you can only create and maintain tests for some possible use cases. You will miss something.

Anti-Pattern: Using API Access Logs for Troubleshooting

API access logs are records of requests and responses made to an API, capturing details such as the requestor’s IP address, HTTP method, requested endpoint, status codes and other relevant information for monitoring, debugging and security purposes.

Using access logs to troubleshoot API issues is like searching for a needle in your microservices haystack. One of our users had a performance issue with their APIs; they spent days searching for the root cause using access logs. Is it the gateway? Does it need more CPU? Is it a configuration issue? They enabled distributed tracing and, within seconds, they could see what was happening: All the time was spent in the authentication server issuing JWT tokens.

To really understand what is happening in your system, you need to be able to track a single user request from the API gateway to the upstream services and their dependencies for effective troubleshooting.

As Martin Thwaites puts it: “Logs are a poor man’s observability tool.” You need distributed tracing.

Anti-Pattern: Different Teams, Different Tools, Different Data

Different teams have different needs. The tool that works for your DevOps team might not fit your product team best. By forcing everyone to use the same tool, you’re not promoting unity but inefficiency.

However, while promoting tool diversity might be the right move for your organization, there should be some coordination and integration between different tools and data sources. Open standards like OpenTelemetry help ensure that information can be shared and leveraged across teams and tools.

The DevOps and the product teams should rely on the same error rate when assessing the performance and reliability of the newest API product launched.

Anti-Pattern: One Size Fits All

Different API architecture styles have different needs when it comes to observability.

REST APIs might be today’s default, but newer styles like GraphQL and gRPC are gaining popularity. Each of these styles has distinct characteristics and requirements regarding observability. Just think about GraphQL’s way of handling errors, which differs significantly from REST APIs. Web Socket and event-driven APIs have their own unique observability challenges, as well.

Different API architectures have different needs, and your observability strategy needs to reflect that to improve the overall reliability of your application.

Anti-Pattern: Observability for Production Only

Why aren’t you using observability in pre-production?

By using distributed tracing during the API development life cycles, developers can trace the flow of requests and responses across different services, providing a comprehensive view of how the API behaves under various scenarios.

With observability, they can identify performance bottlenecks and even detect architectural problems like N+1 problems in GraphQL queries.

For teams leveraging APIOps practices, observability data can be used as an additional validation before promoting an API to the production stage.

Anti-Pattern: Starting the Trace at the API Gateway is Overrated

Tutorials about observability usually start the instrumentation process at a microservice level. It is great to have detailed insights into your microservices. But a lot will happen at the API gateway level when dealing with APIs.

You want to capture all your user transactions, including those that never reach your microservices because of rate-limiting rules, an authentication problem or a caching mechanism.

Starting the trace at the API gateway gives you a clear entrance point and a complete picture of the journey of all your users. That’s why it’s important to use a modern API gateway with built-in support for OpenTelemetry.

So there you have it: Seven API observability pitfalls to avoid. Keep your APIs observable and your users happy!

To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.
March 6, 2024