Tag: CI/CD

  • Harness Launches Open Source Gitness Platform

    Harness Launches Open Source Gitness Platform

    At its unscripted conference today, Harness unveiled a streamlined open source git platform, dubbed Gitness, that is significantly lighter weight than the platforms DevOps teams traditionally rely on to build increasingly complex applications.

    At the same time, the company is adding four additional modules to its continuous integration/continuous delivery (CI/CD) platform to improve developer productivity and make it simple to build and deploy secure applications.

    Finally, Harness is launching an Engineering Excellence Collective, a consortium of 300 senior engineering leaders. The collective is developing a maturity model based on 11 pillars for achieving software engineering excellence; a FinOps Excellence community initiative was also announced to advance best practices for controlling cloud computing costs.

    Harness CEO Jyoti Bansal said the need for a modern open source Git platform—as an alternative to legacy platforms developed a decade ago—has never been more apparent. Gitness provides all the collaborations, performance, analytics and security capabilities DevOps teams require in a streamlined platform that reduces overall bloat by seamlessly integrating, for example, code hosting and automated pipelines, he added.

    Other core capabilities include automated status checks and mandatory reviews, branch protection rules, user access management tools and a semantic search engine that has been infused with generative artificial intelligence (AI) capabilities that invoke multiple large language models (LLMs).

    Collectively, the approach reduces the amount of time spent when attempting to integrate and manage disparate tools on legacy platforms that inevitably lead to the creation of additional bottlenecks, said Bansal. Rather than bolting on additional tools, the time has come to simplify software engineering by migrating to a platform specifically designed to address modern software engineering requirements, he added.

    Designed to be installed using Docker contains in a matter of minutes, DevOps teams can also take advantage of tools to automatically migrate their existing repositories and pipelines from platforms such as GitHub and GitLab.

    Gitness is also at the core of one of the four additional modules that Harness is adding to its portfolio. The Harness Repository is a premium tool, available in beta, that provides governance and compliance-as-code capabilities embedded within the repository.

    The three other modules being added are a Harness Internal Developer Portal based on open source Backstage software, Harness Infrastructure as Code Management that makes it simpler to implement GitOps workflows at scale and Harness Software Supply Chain Assurance to provide visibility into open source software components and manage the creation of software bills of materials (SBOMs).

    It’s not clear whether DevOps teams are ready to rip and replace existing platforms, but the pressure to improve developer productivity has never been higher. Organizations increasingly realize that developers are spending the bulk of their time managing development environments tied to legacy platforms that create bottlenecks and are becoming more costly to manage and sustain.

    Harness, in effect, is now calling for the establishment of an alternative approach based on open source software that promises to streamline software engineering workflows. As applications incorporate microservices, security controls and AI models, they are only going to become more challenging to build and deploy at scale.

  • Buildkite Acquires Packagecloud to Streamline DevOps Workflows

    Buildkite Acquires Packagecloud to Streamline DevOps Workflows

    Buildkite Pty Ltd today revealed it has acquired Packagecloud as part of an effort to tightly integrate package management tools with its continuous integration/continuous delivery (CI/CD) platform.

    Buildkite CEO Keith Pitt said the goal is to reduce the level of friction that exists between providers of third-party package management tools and CI/CD platforms that DevOps teams are today required to address themselves.

    There is no shortage of package management tools used to configure, install and update software, but the level of integration between these tools and CI/CD platforms has not historically been very high. The acquisition of Packagecloud will result in a package manager becoming a first-class citizen within the context of a CI/CD platform, said Pitt.

    Buildkite will maintain the existing support Packagecloud provides for platforms such as Jenkins, but the acquisition will streamline DevOps workflows and improve overall developer productivity, he added.

    In the months ahead, Buildkite will also add a range of tools to better enable DevOps teams to achieve that goal, said Pitt. In addition, the company plans to add tools to streamline the management of compliance processes as regulations pertaining to application development are becoming more stringent, he added.

    It’s not clear how much IT organizations are reevaluating their CI/CD platforms to improve developer productivity, but platform engineering as a methodology to centralize the management of DevOps workflows is gaining traction. Those efforts promise to reduce the total cost of implementing DevOps at scale and improve developer productivity by making the scaffolding required to build applications more readily accessible. Today, too many developers are spending time managing developer environments rather than writing actual code.

    The challenge, of course, is convincing developers—who tend to jealously guard their tool prerogatives—to embrace a platform engineering methodology and cede control over the development environment to a centralized team.

    One way or another, developer productivity needs to improve as more organizations encounter economic headwinds. Developers are the most expensive resource any organization has, so improving the rate at which high-quality code is created will always be imperative, noted Pitt.

    Today, organizations depend on software to drive revenue. Investors are assessing senior leaders within those organizations on their ability to deliver quality software in a timely fashion. Each new application and subsequent software update now has a direct impact on the profitability of the organization. As such, tolerance for legacy approaches to building software that require more time and effort is dropping.

    The issue, as always, is that switching from one software development platform to another is an expensive proposition, so many organizations may find themselves supporting a mix of legacy and modern platforms. The challenge becomes defining the pace at which that migration occurs as new applications are created and legacy applications are continuously updated.

  • CloudBees Unfurls Dual CI/CD Strategy at DevOps World Event

    CloudBees Unfurls Dual CI/CD Strategy at DevOps World Event

    During a DevOps World event, CloudBees took the wraps off the first major update to the open source Jenkins continuous integration/continuous delivery (CI/CD) platform made in the past several years.

    At the same time, the company also unveiled a proprietary DevSecOps platform based on Kubernetes that is optimized for building and deploying cloud-native applications based on portable Tekton pipelines.

    Shawn Ahmed, chief product officer for CloudBees, said the updates to Jenkins make it possible to horizontally scale Jenkins controllers while at the same time ensuring high availability. The latest update also provides a workspace caching mechanism to simplify sharing them across multiple jobs in a way that significantly reduces the amount of compute and storage resources consumed, he added.

    CloudBees also added a Baseline capability to the Jenkins repository to improve startup times without causing build storms.
    Collectively, those capabilities will increase the number of jobs that can be simultaneously executed to improve overall productivity, said Ahmed.

    CloudBees has also added a Pipeline Explorer tool in the cloud-based edition of Jenkins that enables DevOps teams to visually debug and troubleshoot pipeline jobs. That tool will also provide the foundations for self-healing and artificial intelligence (AI) capabilities that CloudBees plans to add to the Jenkins platform later this year and into 2024, added Ahmed. For example, AI tools will surface suggestions on how to fix a problem in a pipeline, he noted.

    The DevSecOps platform, meanwhile, leverages Tekton pipelines to provide DevOps teams with more granular control over how pipelines can be orchestrated, said Ahmed. It also provides access to GitHub Actions-style domain-specific language (DSL), support for feature flagging, security, compliance, pipeline orchestration capabilities, analytics and value stream management (VSM) tools that can be deployed by an internal IT team or accessed via a single or multi-tenant managed cloud service.

    Those built-in capabilities will make it simpler for platform engineering teams to centrally manage building and deployment of cloud-native applications by enabling developers to self-service their own engineering requirements via an internal developer platform (IDP), said Ahmed. The overall goal is to provide a more integrated approach that breaks down software engineering silos that exist in many existing DevOps workflows, said Ahmed.

    Each DevOps team will need to decide for themselves to what degree to move forward using Jenkins to build and deploy applications versus opting for a more modern alternative in the form of the DevSecOps platform. CloudBees, however, is committed to providing plug-ins and integrations across both platforms to enable DevOps teams to protect their investment in existing tooling, noted Ahmed.

    The status of Jenkins X, an open source CI/CD platform based on Kubernetes that CloudBees previously donated to the Cloud Delivery Foundation (CDF), is not immediately clear. Regardless of the approach, however, there is a clear opportunity to revamp DevOps workflows by upgrading Jenkins or switching to a more modern platform such as DevSecOps. The challenge, as always, is determining the pace at which any such transition might be made.

  • CDF Report: Steady Growth in Number of Jenkins Jobs Run

    CDF Report: Steady Growth in Number of Jenkins Jobs Run

    The Jenkins project that operates under the auspices of The Continuous Delivery Foundation (CDF) reported this week that over a two-year period, usage of pipelines grew 79%, with total workloads defined in the continuous integration/continuous delivery (CI/CD) platform growing 45% over the same period.

    Based on analysis conducted from June 2021 to June 2023, the report found the number of jobs defined increased from 27,105,176 jobs to 48,625,398 jobs per month. The number of all jobs defined on Jenkins increased from 50,785,205 to 73,746,418 jobs per month (45%).

    Sacha Labourey, a member of the governing board for the CDF and chief strategy officer at CloudBees, a provider of commercially supported instances of the open source Jenkins CI/CD platform, said it’s clear that more applications are being constructed by more organizations than ever using DevOps workflows.

    At present, the CDF reported there are now more than 600 active contributors to the Jenkins project. That level of support has made it possible for Jenkins to become a standard CI/CD platform. DevOps teams know there is a substantial open source community they can count on for support, said Labourey.

    The challenge now is how to centrally observe DevOps workflows to accelerate application development while at the same time giving developers more freedom to innovate. Platform engineering principles represent an opportunity to achieve that goal, as long as developers still have an opportunity to innovate and collaboratively address bottlenecks in DevOps workflows as they occur.

    The platform engineering team should be observing processes and proactively making recommendations and define a set of guardrails that ensure DevSecOps best practices are followed, he added.

    Otherwise, application developers are likely to view platform engineering as another effort by centralized IT teams to take power away from the developer, noted Labourey.

    Striking that balance is especially critical as the number of cloud-native applications being built and deployed alongside classic monolithic applications has significantly increased, he added. DevOps teams need to be able to centralize observability of what are two distinct styles of application development that tend to occur at a significantly different pace from one another, Labourey said.

    Hopefully, advances in artificial intelligence (AI) will make it easier for DevOps teams to strike the right balance between management and innovation. In the meantime, however, as the pace of application development continues to accelerate, each DevOps team will need to determine how best to evolve their existing workflows. That’s especially critical at a time when economic headwinds are requiring organizations to find ways to increase existing developer productivity rather than hire or contract additional developers.

    In the meantime, it’s apparent that DevOps, as a methodology for building and deploying applications faster, is here to stay. The next major challenge is finding a way to achieve that goal at levels of scale that require automation everywhere from the network edge to the cloud.

  • Datadog Updates Aim to Improve Logging, Testing Efficiency

    Datadog Updates Aim to Improve Logging, Testing Efficiency

    At its recent DASH 2023 conference, Datadog revealed Flex Logs and Intelligent Test Runner, two major new capabilities aimed at improving logging and testing efficiency for engineering teams.

    Omri Sass, director of product management at Datadog, said Flex Logs introduces a new middle tier of log storage between real-time querying and cold archives. As Sass explained, this provides a cheaper option for retaining logs beyond the typical two weeks for occasional analytics and investigations. By decoupling storage from compute for reading logs, teams can independently scale each based on their needs. This enables retaining rich log data for months or years to uncover long-term trends, he said.

    In addition, Sass noted that Flex Logs helps address common log analysis limitations around storage costs and rehydrating archives. The feature better aligns log pricing with specific use cases beyond just troubleshooting. According to Sass, this evolution in log storage tiers will likely shift more logs from legacy on-premises systems to its cloud platform.

    The second capability, Intelligent Test Runner, leverages observability signals to optimize CI test suites. As Sass explained, comprehensive test automation in CI pipelines often increases deployment times and flakes. Intelligent Test Runner inspects code changes to determine associated tests, ignoring irrelevant ones, which helps avoid long wait times for minor changes just to exercise unaffected areas.

    By tapping into existing telemetry around code commits, test health and runtimes, the feature adapts testing to each code modification. Sass explained that observability data has matured enough to enable real-time optimization like this. The end result is faster CI validation, so engineering teams can ship updates more rapidly.

    Sass highlighted how Flex Logs and Intelligent Test Runner are Datadog’s attempts to address real pain points developers face around managing log data life cycles and CI speed bumps. Sass said he believes this combination of depth in core disciplines like logs and application performance monitoring (APM), along with expanding into adjacent developer workflows, is key to solving these issues through innovation.

    Looking to the future, the observability space continues to evolve rapidly as emerging technologies and practices expand monitoring beyond traditional IT infrastructure. Companies handling modern workloads expect their observability platform to provide comprehensive visibility as development toolchains and architectures grow more complex, with the next generation of observability likely incorporating advanced analytics, automation and AI to help engineers tame systems at ever-increasing scale and speed. 

  • Palo Alto Networks Extends CNAPP to CI/CD Platforms

    Palo Alto Networks Extends CNAPP to CI/CD Platforms

    Palo Alto Networks has extended the reach of its cloud-native application protection platform (CNAPP) to include the ability to secure continuous integration/continuous delivery (CI/CD) platforms.

    Daniel Krivelevich, CTO for application security for Prisma Cloud at Palo Alto Networks, said this extension of the Prisma Cloud platform would make it easier to better secure software supply chains as software builds are continuously updated by application development teams.

    Previously, Palo Alto Networks provided the ability to secure runtimes and secrets and scan code for vulnerabilities as it is being developed. This latest extension now adds the ability to secure the CI/CD platforms that drive DevOps workflows, he noted.

    While more organizations than ever are shifting application security responsibility further left toward developers, many of them overlook the need to secure the CI/CD platform itself. Most developers are not cybersecurity experts, so the chances they will miss something that can find its way into a software build are still high. Extending the reach of Prisma Cloud to the CI/CD platform will ensure the build that forms the foundation of the code is free of vulnerabilities when it finds its way into a production environment.

    That’s critical, because cybercriminals are becoming more adept at injecting malware into software supply chains in the hopes of being able to eventually compromise multiple downstream applications. In fact, this issue has prompted governments around the world to hold more organizations legally responsible for the security of the applications they build and deploy.

    There are, of course, multiple approaches to securing software supply chains, but Palo Alto Networks is making a case for an integrated platform that provides a comprehensive approach to securing the entire IT environment. Rather than integrating multiple point products, a CNAPP ultimately reduces the total cost of cybersecurity on an end-to-end basis, noted Krivelevich.

    It’s not clear how quickly organizations are transitioning to CNAPPs, but as that transition occurs, the overall state of application security should improve. One of the issues that has historically plagued application security is no one in an organization has been specifically tasked with ensuring it. Application development teams assumed cybersecurity professionals were allocating funds to it. Cybersecurity teams, however, tended to focus on areas they directly controlled, such as network firewalls. A CNAPP makes it simpler to deploy a cybersecurity framework in a more extensible way.

    Of course, there’s still much work to be done when it comes to prioritizing vulnerability remediation efforts. The truth is developers only have so much time to allocate to developing patches, so the focus needs to be on identifying the vulnerabilities that are the most severe in terms of the amount of damage that might be inflicted. At the very least, however, a CNAPP should provide the means by which the context required to identify those vulnerabilities could be more easily shared.

    One way or another, however, the number of vulnerabilities finding their way into production environments should steadily decline as more focus is put on securing software supply chains.

  • Shift Left With DAST: Dynamic Testing in the CI/CD Pipeline

    Shift Left With DAST: Dynamic Testing in the CI/CD Pipeline

    Dynamic application security testing, or DAST, is a process used to evaluate an application’s security while it’s running rather than while it’s in a static state. This method involves simulating attacks on an application, identifying vulnerabilities and analyzing the responses. DAST focuses on the exposed HTTP and HTML interfaces of web-enabled applications, making it an excellent tool for spotting security vulnerabilities that may occur during an application’s active runtime.

    DAST is a ‘black-box’ testing method, meaning it tests the application from the outside in—much like a potential attacker might. It doesn’t have access to the source code and instead focuses on the application as a whole rather than its individual components. This external perspective enables DAST to discover potential security threats that static testing methods might miss.

    While DAST is an effective security testing tool, it becomes even more potent when paired with a shift left approach.

    Understanding the Concept of ‘Shift Left’ in Software Testing

    Shift left is a practice in software development where testing is conducted earlier in the life cycle—that is, it’s ‘shifted left’ on the project timeline. Traditionally, testing was seen as the last phase before the release of a software product. However, with the shift left approach, testing becomes an integral part of every stage of the development process from planning to deployment.

    This approach has several advantages. It allows for the identification and resolution of issues earlier in the development process, reducing the cost and effort required to fix them later. It also encourages collaboration between developers and testers, fostering a more cohesive, efficient development environment.

    Shift left is not just a testing strategy but a philosophy that promotes preventive measures over reactive solutions. It’s about building quality and security into software from the get-go, rather than tacking it on at the end.

    DAST in the Context of Shift Left

    Importance of DAST in Early Stages of Development

    The integration of DAST in the early stages of development is crucial for several reasons. First, by conducting dynamic security testing from the onset, teams can identify vulnerabilities earlier, making them easier and less costly to fix. This proactive approach helps to prevent security issues from becoming ingrained in the code, which can lead to significant problems down the line.

    Second, early integration of DAST encourages a security-focused mindset from the beginning of the project, promoting a culture of security within the team. This cultural shift is crucial in today’s cybersecurity climate, where threats are increasingly sophisticated, and the stakes are higher than ever.

    How DAST Complements Other Testing Methods

    DAST doesn’t replace other testing methods; rather, it complements them. While DAST focuses on external vulnerabilities, other testing methods like static application security testing (SAST) can target internal issues. By combining these methods, teams can achieve a more comprehensive view of their application’s security.

    In a shift left approach, this combination of testing methods can be very powerful. By conducting these tests early and often, teams can ensure that both the external and internal aspects of their application are secure. This layered approach to security testing can help to catch any vulnerabilities that might otherwise slip through the cracks.

    Incorporating DAST Into the CI/CD Pipeline

    1. Understanding the Application Structure

    Before we can integrate DAST into the CI/CD pipeline, it’s crucial to understand the structure of the application we’re working with. This involves understanding the layers of the application, the technologies used and the data flows. By analyzing the application’s architecture, you can identify the critical components that need more attention during the security testing phase.

    Understanding the application structure is not just about knowing the technology stack. It’s also crucial to understand the business logic and the functionality of the application. This will allow you to create better test cases during the DAST process. Additionally, understanding the application structure will enable you to make better decisions when configuring the DAST tool, leading to more efficient and effective security testing.

    Finally, understanding the application structure will help you prioritize the vulnerabilities found during the DAST process. Not all vulnerabilities are created equal; some might be more critical due to the data they expose or the potential damage they could cause to the application. 

    2. Configure the DAST Tool

    The next step is to configure the DAST tool. This involves setting up the scanning parameters, defining the test cases, and tuning the tool to reduce false positives.

    When configuring the DAST tool, it’s important to strike a balance between thoroughness and speed. You want the tool to be able to identify all potential vulnerabilities, but you also want the scanning process to be fast enough not to slow down the development process. This balance can be achieved by carefully selecting the scanning parameters and continuously tuning the tool based on the results of the scans.

    Additionally, when configuring the DAST tool, you should consider the unique characteristics of your application. For example, if your application uses a specific technology or framework that is known to have certain vulnerabilities, you should ensure that the DAST tool is configured to test for these vulnerabilities. Similarly, if your application handles sensitive data, you should prioritize testing the components that process and store this data.

    3. Integrating DAST Into the CI Pipeline

    The next step is to integrate DAST into the CI pipeline. The CI pipeline is a sequence of steps that developers follow to merge their changes into the main codebase. By integrating DAST into this pipeline, we can ensure that every code change is automatically tested for security vulnerabilities.

    Integrating DAST into the CI pipeline can be achieved using various tools and plugins. Most DAST tools provide APIs that allow you to automate the testing process. By using these APIs, you can trigger a DAST scan every time a new code change is pushed to the repository. This allows you to catch vulnerabilities early in the development process, reducing the cost and time needed to fix them.

    However, integrating DAST into the CI pipeline is not a one-time task. It requires continuous monitoring and tuning to ensure that the testing process remains effective. You will need to update your DAST configurations as the application evolves, add new test cases as new features are developed, and fine-tune the testing process to reduce false positives. 

    4. Scan for Vulnerabilities

    After integrating the DAST tool, you can start scanning for vulnerabilities. The scanning process involves executing the test cases defined during the configuration phase and identifying potential security issues in the application. Depending on the size and complexity of your application, this process can take from a few minutes to several hours.

    During the scanning process, the DAST tool will interact with the application just like a real user would. It will send inputs to the application, observe the responses and analyze them for signs of security vulnerabilities. This dynamic approach allows the tool to identify vulnerabilities that static analysis tools might miss, such as runtime vulnerabilities and issues related to the application’s business logic.

    5. Analyze and Prioritize Findings

    Once the scanning process is complete, the next step is to analyze and prioritize the findings. The DAST tool will provide a report of the identified vulnerabilities, including their type, location and potential impact. This information should be used to prioritize the vulnerabilities based on their risk to the application and the business.

    Analyzing and prioritizing the findings is not an easy task. It requires a deep understanding of the application, the business, and the threat landscape. You need to consider various factors, such as the potential impact of the vulnerability, the likelihood of exploitation, the sensitivity of the affected data, and the complexity of the remediation process. By considering these factors, you can prioritize the vulnerabilities and focus your efforts on those that pose the highest risk.

    In addition, analyzing and prioritizing the findings should involve all relevant stakeholders. Developers, security teams, and business leaders should all participate in this process, as they each bring unique perspectives and insights that can help make better prioritization decisions.

    6. Remediate and Retest

    After analyzing and prioritizing the findings, the next step is to remediate the vulnerabilities and retest the application. Remediation involves modifying the application’s code or configuration to eliminate the identified vulnerabilities. Depending on the complexity of the vulnerabilities, this process can involve simple code changes or significant architectural modifications.

    After remediation, it’s crucial to retest the application to ensure that the vulnerabilities have been effectively addressed. This involves running the DAST tool again and verifying that the previously identified vulnerabilities are no longer present. Additionally, retesting allows you to check for new vulnerabilities that might have been introduced during the remediation process.

    Remediation and retesting is a continuous process. As new vulnerabilities are discovered and new features are developed, you will need to repeat this process to ensure the ongoing security of your application. This continuous effort is a critical component of the shift left with DAST strategy.

    7. Continuous Monitoring and Updates

    The final step in incorporating DAST into the CI/CD pipeline is continuous monitoring and updates. This involves regularly checking the effectiveness of the DAST tool, updating its configurations as the application evolves, and tuning it to reduce false positives.

    Continuous monitoring and updates are crucial to maintaining the effectiveness of your shift left with DAST strategy. As the application evolves and the threat landscape changes, the DAST tool needs to be updated to remain effective.

    Conclusion

    In conclusion, incorporating DAST into the CI/CD pipeline is a powerful strategy to enhance the security of your software. By understanding the application structure, integrating DAST into the CI pipeline, configuring the DAST tool, scanning for vulnerabilities, analyzing and prioritizing findings, remediating and re-testing, and continuously monitoring and updating, you can unleash the power of shift left with DAST. This comprehensive approach will allow you to identify and address security vulnerabilities early in the development process, leading to more secure and reliable software.

  • Reimagining CI/CD: AI-Engineered Continuous Integration

    Reimagining CI/CD: AI-Engineered Continuous Integration

    In my recent article Revolutionizing the Nine Pillars of DevOps with AI-Engineered Tools, I explained that the continuous integration (CI) Pillar is about developers frequently merging code changes into a central repository. After that, automated builds and tests are run, successful builds are containerized and release candidate artifacts are signed. The main objectives of CI are to detect problems early, reduce integration issues and provide high-quality, secure release candidates. In this article, I explain how AI can enhance each step in the CI phase of DevOps CI/CD pipelines.

    Code Branching: In DevOps, managing multiple branches can be challenging, especially with large teams. This can lead to merge conflicts, confusion about where code should be committed and difficulty tracking changes.

    • AI Solution: AI can help by automatically managing branches and offering recommendations on merging. AI-based systems can analyze code patterns and project management styles to suggest optimal branching strategies, minimizing conflicts and maintaining organization.
    • AI Tool Example: Microsoft’s IntelliCode offers AI assistance for software development which could potentially help manage branches by suggesting optimal strategies.

    Code Commits: Commits need to be made regularly to maintain the codebase, but they can introduce errors. Miscommunication and the lack of a unified commit policy can lead to problems.
    • AI Solution: AI can analyze commit histories to identify patterns and recommend best practices. Additionally, AI tools can provide real-time feedback on code changes, suggesting potential improvements before the code is committed.
    • AI Tool Example: Kite is a coding assistant that uses AI to help with code changes and improvements before the commit stage.

    Merges: The process of merging code changes can lead to merge conflicts, particularly in large, complex codebases.
    • AI Solution: AI tools can help predict potential merge conflicts and offer solutions before they occur. They can analyze code structures and change patterns to predict and prevent these conflicts.
    • AI Tool Examples: Tools like DeepCode or Sider use AI to review and analyze your code to help prevent merge conflicts.

    Static Scanning: This process can be time-consuming and complex with a high potential for false positives.
    • AI Solution: AI can improve static analysis tools by learning from past scans to improve accuracy, reduce false positives, and even suggest potential fixes for issues identified.
    • AI Tool Example: DeepCode is an AI tool that analyzes your code, learns from past scans and suggests fixes for issues identified.

    Builds: Build failures can occur due to a number of issues, including poor code quality, incompatible changes, and environment configuration issues.
    • AI Solution: AI can predict build failures by analyzing historical data and identifying patterns that lead to such failures. AI can also automate the build process and manage environment configurations to reduce human error.
    • AI Tool Example: CircleCI Insights uses machine learning to analyze and predict the success of builds, helping to mitigate build failures.

    Scanning for Quality Errors and Security Vulnerabilities: This process can be slow and yield false positives, and the sheer number of potential issues can be overwhelming to manage.
    • AI Solution: Machine learning can be trained on previous scans to recognize patterns, improving the speed and accuracy of scans. AI can also prioritize issues based on potential impact, helping teams focus their efforts.
    • AI Tool Example: ShiftLeft scans for vulnerabilities in the code and uses AI to prioritize the issues found.

    Unit Testing: Writing and maintaining unit tests can be time-consuming and it’s often difficult to achieve good coverage.
    • AI Solution: AI can assist in generating unit tests based on code analysis and historical data, ensuring good coverage and saving development time.
    • AI Tool Example: Diffblue Cover uses AI to automatically write unit tests, aiming to improve coverage and save developer time.

    Containerization: Misconfiguration of containers can lead to issues such as application failures, security vulnerabilities or inefficient resource usage.
    • AI Solution: AI can help solve this challenge by automating and optimizing the configuration process.
    • AI Tool Example: Magalix can analyze historical configuration data and current application requirements to suggest optimal configuration settings for a container. It can flag potential misconfigurations or inefficiencies and suggest improvements based on best practices learned from past data.

    Functional Integration Testing: These tests often require significant resources and time to write and maintain.
    • AI Solution: AI can generate functional integration tests based on system behavior analysis, reducing the time and effort required to maintain these tests.
    • AI Tool Example: The test.ai tool uses AI to generate and execute functional tests, reducing the need for manual test case creation.

    Determining Whether a Build is Suitable as a Release Candidate Artifact: Manual analysis of build quality can be slow and subjective.
    • AI Solution: AI can analyze build artifacts, test results and other data to predict the suitability of a build as a release candidate, reducing the time and subjectivity involved in this decision.
    • AI Tool Example: Google’s Cloud Build can be leveraged with machine learning to predict the success of builds.

    Digitally Signing Artifacts in an Artifact Repository: This process can be cumbersome and prone to human error.
    • AI Solution: AI can manage the process of digitally signing artifacts, ensuring that only validated, authorized code is signed and reducing the potential for human error.
    • AI Tool Example: While no specific AI tool exists for this task, automated DevOps tools such as JFrog Artifactory or Sonatype Nexus could be combined with AI technology to automate the signing process, ensuring that only validated, authorized code is signed.

    Challenges of CI Transformation to AI

    The transition from traditional continuous integration (CI) to AI-engineered CI presents several challenges:

    Integration with Existing Systems: When incorporating AI into an existing CI toolchain, ensuring seamless integration can be a challenge. Organizations need to verify that AI tools can work effectively with current systems and don’t disrupt existing workflows.

    Skillset and Understanding: Adopting AI requires a certain level of understanding of how AI works, including its strengths and weaknesses. Teams need to be trained on how to use AI tools effectively and interpret their outputs.

    Data Privacy and Security: AI tools often rely on analyzing large amounts of data, which can include sensitive or proprietary information. Organizations need to ensure that using these tools doesn’t compromise their data security and privacy.

    Cost: Implementing AI tools can be costly, both in terms of purchasing and deploying the tools and in the potential need for additional hardware resources. Organizations need to consider the return on investment (ROI) and determine if the benefits of using AI outweigh the costs.

    Tool Maturity: Many AI tools in the DevOps space are relatively new and still evolving. Organizations need to consider the maturity and stability of the tools they’re planning to use.

    Over-Reliance on AI: While AI can greatly enhance CI processes, over-reliance on AI without proper understanding can lead to misinterpretations and errors. It’s important to balance AI usage with human oversight and expertise.

    Benefits

    Transitioning to AI-engineered CI toolchains can have several significant benefits for organizations:

    Improved Efficiency: AI can automate various aspects of the CI process, such as code review, testing and environment configuration. This can greatly reduce the time taken for these tasks, speeding up the overall CI process and allowing teams to deliver software more quickly.
    Increased Quality: AI tools can perform in-depth code analysis and identify potential issues with a level of detail that might be missed by humans. This can lead to improved code quality, fewer bugs, and more stable software.
    Predictive Analytics: AI can analyze historical data to predict outcomes, such as the likelihood of a build failure. This can help teams proactively address issues, reducing the risk of failures and improving the reliability of the CI process.
    Improved Decision Making: AI can provide valuable insights based on data analysis, helping teams make more informed decisions. This can relate to things like identifying the most suitable release candidate or deciding when and how to scale resources.
    Consistency and Standardization: AI-driven CI can ensure consistent application of rules and standards, as there’s less reliance on individual developers’ judgment. This can lead to a more standardized, reliable development process.

    Summary

    Overall, AI has the potential to greatly improve the efficiency, accuracy and effectiveness of DevOps continuous integration practices. However, it’s also important to note that these solutions require careful management and oversight to ensure they’re working as intended and improving rather than hindering processes. Organizations should start with small, manageable projects to gain an understanding and experience with AI before scaling up. They should also invest in training and support to help their teams adapt to these new tools.

  • Mendix Embraces DevOps for Next Era of Low-Code App Development

    Mendix Embraces DevOps for Next Era of Low-Code App Development

    Mendix, a subsidiary of Siemens, today updated its namesake low-code application development platform to add support for DevOps workflows and make it simpler for developers to compose applications and embed artificial intelligence (AI) capabilities.

    Sheryl Koenigsberg, vice president of product marketing for Mendix, said Mendix 10 makes it possible to use Git repositories for version control along with Mendix Pipelines, currently in beta. Mendix Pipelines is a continuous integration/continuous delivery (CI/CD) environment that will be embedded in the low-code platform. In addition, Mendix is adding support for Webhooks that can be used to integrate Mendix with existing CI/CD platforms.

    Mendix has also added a Solutions Kit, now generally available, to make it possible to customize applications in a way that ensures those changes will be preserved as application environments are upgraded.

    There is also now a Portfolio Manager tool to track projects and a connector for integration with the Jira project management software from Atlassian.

    Later this year, Mendix will also make generally available Adaptation Insights to provide insights into application extensions.

    In the future, developers will be able to leverage an extensibility framework to tailor the Mendix Studio Pro integrated development environment (IDE) by adding their own plugins, wizards, editors and third-party integrations. Later this year, Mendix 10 will also add, in beta, a version of its Studio Pro IDE for MacOS.

    The Mendix approach to composability is also being extended to incorporate AI. The company is adding Mendix Assist, a bot that generates validation logic for data entry. This module is the second AI capability that Mendix has developed after previously previewing Mendix Chat, a tool based on a large language model created using Mendix data that will be integrated into the Mendix integrated development environment (IDE) later this year. Mendix Assist will be incorporated into Mendix Chat.

    Mendix is also developing additional LLMs for creating applications and will as part of its commitment to composability plans to make it possible to integrate third-party AI services and LLMs into the Mendix Runtime to create “smart apps.” AI models will be converted to the Open Neural Network Exchange (ONNX) format, an open standard for machine learning models, to provide that capability.

    In addition, Mendix is adding support for event-driven architectures via Business Events and an Event Broker, to build more complex applications.

    Other capabilities being added to Mendix 10 include updated REST and database connectors, support for visual data filters, rich properties and design controls, a Start from Spreadsheet tool, end user survey tools, a PDF document generator and a revamped set of governance and risk assessment tools.

    Finally, Mendix is adding a Terraform module to make it simpler to deploy its low-code platform on the Amazon Elastic Kubernetes Service (Amazon EKS) in addition to providing a reference implementation for building private clouds to develop applications using the Mendix Cloud service.

    As AI-infused low-code applications are developed faster, it’s not clear to what degree organizations are prepared to embrace the inevitable disruption that will ensue. However, the one thing that is clear is that the DevOps workflows used to deploy these applications at scale will soon be put to the ultimate test.

  • Harness Brings Generative AI Capabilities to DevOps Workflows

    Harness Brings Generative AI Capabilities to DevOps Workflows

    Harness today added a raft of generative artificial intelligence (AI) capabilities to its continuous integration/continuous delivery (CI/CD) platform as part of a larger effort to reduce the level of manual toil that today conspires to create DevOps bottlenecks.

    The added generative AI capabilities include an AI Development Assistant (AIDA), which analyzes log files and correlates error messages with known issues. It also includes the ability to use natural language to define policies for governing cloud assets and costs and an ability to automatically identify security vulnerabilities and generate code fixes.

    Other generative AI capabilities that will be added to AIDA over the next three to six months include automated code reviews along with tools for creating pipelines and launching chaos engineering experiments.

    Harness CEO Jyoti Bansal said these are the first of a series of generative AI capabilities that will be applied across the entire software development life cycle (SDLC) using large language models (LLMs) that the company has trained using publicly available sources. AIDA, at its core, provides a tool for creating custom prompts for launching queries against LLMs and a vector database that Harness has incorporated into its platform.

    That approach means no customer data was used to create the model and none of the data used will find its way into a generative AI platform that is available to anyone, said Bansal. As a result, organizations don’t need to be concerned they light run afoul of licensing issues because proprietary code was inadvertently included in an LLM, he added.

    Most of the existing generative AI capabilities that have been brought to market thus far are designed primarily to improve developer productivity, noted Bansal. Harness is now bringing the benefit of generative AI to DevOps teams that today rely on too many manual processes to manage the automations they build on behalf of application development teams, he added.

    Harness is making AIDA available at no extra charge at a time when many DevOps teams will soon be evaluating the AI capabilities provided to them. Most DevOps teams are not going to want to manage workflows without being able to rely on AI to reduce toil and eliminate bottlenecks. Overall, Harness is forecasting generative AI will drive a 30% to 50% increase in DevOps productivity. In addition, Harness is forecasting there will be a 50% to 75% reduction in the time spent remediating vulnerabilities.

    Bansal said that as that process occurs, Harness is betting more organizations will move to replace legacy CI/CD platform with more modern alternatives.

    Previously, Harness has invested in machine learning algorithms to automate a range of DevOps tasks. Generative AI, in combination with machine learning, will transform DevOps workflows in ways that will ultimately improve both the rate at which applications can be built and their overall quality.

    It’s not clear whether organizations might be willing to replace CI/CD platforms that they have been customizing for decades, but at the very least, that option is now more widely considered than ever.