devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • Copado Expands Beta of Generative AI Tools for Salesforce DevOps Platform

    Copado Expands Beta of Generative AI Tools for Salesforce DevOps Platform

    Copado today announced it has expanded a beta program through which it provides application development teams building applications for the Salesforce software-as-a-service (SaaS) platform access to generative artificial intelligence (AI) capabilities enabled by ChatGPT.

    David Brooks, SVP of Evangelism for Copado, said the company, via the usage of vector databases, has exposed instances of ChatGPT to everything in its knowledge base that has been collected over the last 10 years. That approach enables Copado to extend the large language model (LLM) used to train ChatGPT without exposing any data that might be used to one day train the next version of the generative AI platform.

    The goal is to enable DevOps teams to generate code and tests faster while simultaneously reducing the time required to handle support calls, said Brooks.

    In addition, Copado is using ChatGPT to enable impact analysis and conflict detection to reduce the level of friction encountered when deploying applications in a production environment. That latter capability is critical because, in many instances today, DevOps teams don’t have a firm understanding of how code needs to be deployed, resulting in unnecessary rollbacks of application deployments, noted Brooks.

    Generative AI will also automate the creation of documentation, such as release notes for every application deployed, to reduce the overall level of toil DevOps teams experience, noted Brooks.

    Copado also plans to provide governance tools that will enable DevOps teams to ensure the code written by an LLM is not based on a hallucination that results when trying to reconcile conflicting data, added Brooks. There will, of course, always be a need for a human to review code before it is deployed in a production environment, he added.

    In the longer term, Copado will make it possible for organizations to invoke multiple LLMs to address various functions as they best see fit, said Brooks. In fact, DevOps teams should expect to soon be employing multiple copilots trained to address specific tasks such as writing code, deploying applications, assessing cybersecurity and addressing compliance mandates, he added.
    Ultimately, application development and deployment efforts that today require weeks to complete will soon be accomplished within the same day, said Brooks.

    At the same time, organizations will also be able to better manage application environments that today operate at levels of scale few IT teams can manually track. Generative AI capabilities, in effect, will make it simpler for DevOps teams to effectively manage application environments at much higher levels of scale, said Brooks.

    Each organization will naturally need to determine to what degree they are prepared to absorb that level of innovation, but it’s clear the pace at which applications are built and deployed is about to exponentially increase. The issue organizations will increasingly encounter is that business processes will not be able to absorb the rate at which software can be built and updated.

    It’s not clear just yet to what extent generative AI will transform DevOps, but as advances continue to be made, many of the manual tasks and bottlenecks that today conspire to slow down application development and deployment are about to be eliminated in ways that few would have thought possible a little more than a year ago.

  • Survey Surfaces Significant Platform Engineering Gains

    A survey of 100 software engineers working at organizations in the U.S. and Western Europe published today found nearly all (99%) have embraced platform engineering to some degree as a methodology to manage DevOps workflows at scale, with more than half (53%) having done so in the last year.

    Conducted by Port, a provider of a proprietary internal developer portal (IDP) platform, the survey also found 83% of respondents have also incorporated GitOps workflows into their platform engineering practices.

    The survey also found 85% are working for organizations that have either started deploying IDPs or plan to do so in the coming year. However, what constitutes an IDP is not exactly clear, with more than a third (35%) that have implemented one reporting they are relying on spreadsheets to manage their IDP. A little over half (53%) of respondents, in contrast, are using an IDP to provide developers with self-service capabilities for setting up environments to write code that either was custom built, is based on open source Backstage software originally developed by Spotify or a commercial IDP platform.

    Among the 85% of respondents using or planning to use an IDP, the main measure of success is to improve developer productivity (56%), followed by reduced time to deployment (25%). Despite those initiatives, only slightly more than a third (34%) said IDPs are helping DevOps teams enforce software engineering standards.

    Roni Floman, chief marketing officer for Port, said overall, the survey suggests developers are wasting a lot of time on tasks that have little to do with writing code. For example, the survey finds that prior to embracing platform engineering, developers spent three to four hours a day on non-core activities, resulting in weeks being required to eventually deploy software in a production environment.

    The biggest challenge organizations face when embracing platform engineering is convincing developers that a centralized team has their best interests at heart. Many application development teams embraced DevOps as a best practice in the first place to escape the tyranny of a centralized IT team. The issue is that, as a result, many of them now find themselves managing tasks that have little to do with writing code. Platform engineering seeks to strike a balance between giving developers the ability to adopt new tools when they see fit and the benefits of having a dedicated platform engineering team that automates processes on their behalf.

    Unless platform engineering teams find a way to get developers to buy into the effort, however, it’s unlikely they will succeed, noted Floman.

    Each organization will need to determine for itself what level of platform engineering makes sense for them to implement, given their unique culture. In most cases, a platform engineering initiative will start small before increasing in size and scope as momentum is gained. In the meantime, the most important thing any platform engineering team clearly needs to do in the short term is engage developers. After all, the best way to get support for any change to a DevOps workflow is, as always, to convince developers it was their idea in the first place.

  • DevOps and Databases: The Key to Faster Deployment

    DevOps and Databases: The Key to Faster Deployment

    With DevOps, we learned that the biggest factor in streamlining deployments is to make things automated and minimize communication. However, we didn’t apply that principle to other areas. While security gets some traction with DevSecOps and machine learning has MLOps, we lack a solution for database management. In this article, we propose a new approach that unlocks the power of DevOps engineers to own their databases and lets them self-serve deployments, maintenance and troubleshooting.

    Why Database Deployment and Maintenance is Slow

    In the last few years, we moved from deployment being a big and challenging event to deploying many times daily. We moved from manual deployments to automated CI/CD pipelines. We built tools for deploying infrastructure and configuration changes. This led to a tremendous increase in the complexity of our systems. We somewhat learned how to handle that in deployments, infrastructure, configuration, security or even machine learning.

    However, database complexity increased significantly as well. We now use many database engines with each application. We use SQL and NoSQL databases as well as databases tuned for specific scenarios like machine learning or vector search. Since we deploy changes often and asynchronously, modifications to database schema or background tasks may happen at any time and affect the performance. When this happens, the business is directly impacted as the database issues quickly spread throughout the ecosystem. However, developers and DevOps engineers can’t fix the issues on their own and need help from other teams, specifically operations and database administrators. This makes the process slow and error-prone as things are not automated but require manual human intervention. Recommended reading: Why DevOps should own their databases and how to do it.

    The problem with database maintenance is the lack of solutions that would present the full and actionable story. We have many tools that can alert us about spiking CPU or increased latency, but we don’t know what’s the reason behind these issues. To find out, developers need to ask database administrators to check the logs and metrics to identify slow queries and whether the CPU spikes are caused by changes in the environment (like updating the operating system), slow background tasks that can be postponed (like vacuuming or index rebuilding) or due to inefficient application code that doesn’t use an index and performs a full table scan.

    Since we don’t know the reason, we need to involve multiple teams to investigate. We need to first identify the points of contact in each team, assign tickets or work items, sometimes page people at night, and coordinate the troubleshooting process. To make things faster, we need to fix two aspects: reduce the amount of manual work and reduce the number of people involved in the investigation. Let’s see how.

    Reducing the Manual Work with Observability

    Many things can go wrong when dealing with databases. Let’s see some of the areas that can break and are not automatically fixed today.

    Observability for the Code Changes

    Every time developers change their codebase, there is a risk of breaking the correctness and performance. We can verify the correctness with automated tests (unit tests, integration tests, end-to-end tests, etc.) and do it regularly in our CI/CD pipelines. However, we can’t verify the performance impact. Our tests focus on what we extract from the database and not how we do it. We may use N+1 queries, we may scan tables instead of using indexes, and we may have slow queries with CTEs or not use searchable arguments (a query that is not sargable). All these things may result in CPU spikes or latency degradation and we can’t automatically explain what happens. We need to have observability that gives the story like “the query execution within a given REST endpoint doesn’t use an index and leads to slow performance.”

    Performance of Schema Changes

    When we modify a schema, we risk that tables will be locked for rewriting. This may take minutes or hours and effectively make our systems unavailable. However, our CI/CD pipelines do not test the performance of schema migrations. Tests run against the latest schema and migrations are fast because databases in pre-production environments are small. We don’t check how long these migrations will take in production and if they’ll lock tables. We need to have tools that can warn us if a slow migration is being deployed and will indicate how to make it faster.

    Understanding of Execution Plans

    Databases grow over time. Queries change their execution plans because statistics get outdated or because indexes are modified. However, automated tests won’t capture these issues as they focus on the correctness of the data and they don’t run on the production database. We don’t see how things evolve and that the query degrades gradually over time. We need to have observability systems that provide an understanding of all the moving pieces around. They need to give us a story explaining that “the query we deployed six months ago stopped using an index because the statistics are out of date as we injected a big batch of data last night”.

    We need a story explaining what happened, like SQL stopped using an index or SQL query is not sargable. Other reasons can be schema changes, mismatched query fields, index fragmentation or significant changes in data volume affecting the query optimizer’s decision-making.

    How to Reduce the Work

    Taking all we saw above, we need to have systems that can understand how our environment evolves and how these changes affect performance. They need to alert us when some slow changes are being deployed to production, understand how the deployments affect the performance and automatically explain the issues. Troubleshooting should be as simple as just getting the notification with links to the places that we need to fix (like configuration, a particular line of code in our application, or the cloud console) with an explanation of how to do it. Fixing should be merely approving a pull request or confirming the configuration change.

    Reducing the Number of People Involved With Self-Servicing

    Having database observability is the first step. The second step is the ownership. Developers and DevOps engineers need to own their databases the same way they own their code and deployments. They need to be able to self-serve the issues and do that without any help from operations or database administrators. This will reduce communication and make things much smoother. There are three things that we need to introduce to let developers and DevOps engineers build their ownership.

    First, we need the tools. Just like we explained in the previous section, we need tools that can explain the story. We can’t just put more responsibilities on developers and DevOps engineers without making the work simple and automated.

    Second, we need new processes. We need to have playbooks and operational procedures explaining how to use the tools, how to self-serve the maintenance, and when to escalate. Developers must be familiarized with the procedures and must know how to follow them even outside of working hours. This way we can replace some work (communication, coordination of teams, manual investigation) with some other work (self-servicing) that we can do much faster.

    Finally, we need to change the mindset. Developers and DevOps engineers need to take on new responsibilities and become true owners of their databases. Even though this sounds like more work, we can make it much less time-consuming by building good observability tools and replacing slow human communication with automated and well-defined self-servicing. It’s not about shifting the work from one place to another. It’s about making the work much smaller and much more automated so the developers and DevOps teams can do it without involving other teams. Since we reduce the cross-team collaboration, the increase in the scope of work of developers and DevOps engineers will lead to doing the work faster.

    Summary

    There are three aspects that we need to work on now. First, tools like Metis must be created. Second, Platform Engineers must use these tools to define processes. Third, developers and DevOps engineers need to take ownership. While it may seem challenging at first, we already saw with the DevOps movement that minimizing communication and owning the full scope of the system is beneficial for the business and for the engineers. We need to restructure our teams and reassign responsibilities to unlock the potential and make organizations work faster.

  • Atlassian Delivers on Generative AI Promise for DevOps and ITSM

    Atlassian Delivers on Generative AI Promise for DevOps and ITSM

    During a virtual Unleash event, Atlassian today announced its generative artificial intelligence (AI) capabilities are now generally available across its Jira and Confluence suite of tools for managing IT and DevOps workflows.

    In addition, Atlassian also committed to adding generative AI capabilities to its Bitbucket continuous integration/continuous deployment (CI/CD) platform to enable DevOps teams to review pull requests and leave comments automatically with suggested changes around syntax and code conventions via a natural language interface. Developers will also be able to autogenerate pull request descriptions from commit messages.

    Matt Schvimmer, head of products for the Agile and DevOps division of Atlassian, said the capabilities provided by Atlassian Intelligence will reduce the level of toil and stress DevOps teams experience at a time when the velocity at which applications are being developed and deployed only continues to increase.

    Atlassian reported nearly 10% of its more than 265,000 customers have already leveraged Atlassian Intelligence since the launch of the beta program. Organizations, for example, can instantly create user stories within Jira Software tickets, change the tone of a customer response within Jira Service Management, create summaries, launch workflow requests via prompts and generate a starting point for a test plan in Confluence.

    Natural language capabilities, currently available for Confluence, will shortly be generally made available for Jira. An ability that promises to help demystify company-specific concepts, jargon or acronyms is also available now in beta, with support for Jira Software and Jira Service Management to shortly follow.

    There is little doubt at this juncture that AI will be pervasively applied across both ITSM and DevOps workflows. As those advances are made, it should also become feasible to reduce the level of toil these teams regularly encounter. At the same time, the level of scale at which ITSM and DevOps workflows can be effectively managed should also increase substantially.

    Each organization will need to decide how quickly to incorporate AI into those workflows, but hopefully, the level of burnout experienced by IT personnel will be sharply reduced as more tasks are automated. Less clear is the impact AI will have on the size of IT teams required to provide those services, but for the foreseeable future, there will always be a need for some level of human supervision.

    In the longer term, the war for IT and DevOps talent that has been waged now for multiple decades should diminish as managing these workflows becomes more accessible, noted Schvimmer. Organizations will not need to rely so much on the expertise of a handful of specialists, he added.

    Organizations, however, would be well-advised to make sure AI is embraced from both the bottom up and the top down to ensure optimal adoption, said Schvimmer.

    In the meantime, IT teams should take an inventory of the processes that are likely to be automated by AI today with an eye toward restructuring teams as more tasks are automated. Ultimately, the goal should be to let machines handle the tasks they do best to enable the IT staff to provide services that deliver more value to the business.

  • Sauce Labs Adds Visual Testing Tool to Platform

    Sauce Labs Adds Visual Testing Tool to Platform

    Sauce Labs has added native visual regression testing capabilities to its testing platform to enable DevOps teams to streamline workflows.

    Marcus Merrell, vice president of technology strategy for Sauce Labs, said Sauce Visual provides the ability to test user interfaces without requiring DevOps teams to acquire, maintain and integrate a separate tool.

    The overall goal is to make it possible to test functions and user interfaces in parallel by adding a single line of code to a DevOps workflow, he added. Updates to the visual presentation of a page are automatically highlighted, with any visual inconsistencies automatically surfaced. In comparison, a standalone visual testing tool increases the time it takes to execute a test suite, noted Merrell.

    Sauce Visual natively uses the same tools as the rest of the Sauce test automation platform, and can be integrated into any continuous integration/continuous delivery (CI/CD) workflow. At launch, Sauce Visual is compatible with Java/JUnit, Java/TestNG, WebDriverIO, Selenium, Appium (web) and Cypress testing tools with support for Playwright, Puppeteer, TestCafe, Espresso, and XCUITest forthcoming.

    The Sauce Labs approach also serves to reduce the total cost of application testing at a time when the number of applications organizations are building and deploying only continues to increase exponentially, said Merrell.

    In general, organizations are attempting to strike a balance between shifting responsibility for testing too far left toward developers that typically lack testing expertise and the need to verify application experiences are delivered as expected. Many organizations have shifted more responsibility for testing toward developers in the hope of improving code quality and reducing the number of vulnerabilities that might find their way into a production environment. The challenge is those efforts have also increased the level of cognitive load developers need to carry at the expense of having more time to write application code.

    Of course, in time, artificial intelligence (AI) tools will make it easier to generate test scripts, but there is still going to be a need for testers who were not involved in the writing of code to validate it, noted Merrell. The difference is those testers will not necessarily need to be rocket scientists as the process of creating tests becomes more automated, he added.

    Testing user interfaces is especially problematic for developers because many of them don’t have a lot of expertise in the business functions that the application they are building enables. In addition, there are often minute changes to user interfaces that developers are not always going to discern.

    As always, one of the first things that tends to be reduced whenever a developer falls behind schedule is testing. In a recent Sauce Labs survey found more than two-thirds (67%) admitted they pushed code into a production environment without testing, with more than a quarter (28%) acknowledging they do so on a regular basis.

    The assumption is that if any issues do arise, they can be addressed via the next update to the application. End users, however, are less tolerant of that approach when, in the age of digital services, an alternative application is readily available.

  • DevOps Strategies for Connected Car Development

    DevOps Strategies for Connected Car Development

    The concept of smart, connected vehicles is no longer science fiction but a day-to-day reality. These vehicles, empowered by internet connectivity and a suite of digital technologies, offer unprecedented convenience, safety and entertainment features. 

    But as the cars on our roads become smarter and more interconnected, the complexity of developing their software systems multiplies. DevOps is emerging as an indispensable strategy for organizations in the connected car industry, bringing together software development and IT operations to streamline, optimize and accelerate the development life cycle. 

    This article briefly reviews connected car technology and explores several DevOps strategies that can support connected car development. We’ll discuss how DevOps can manage the complexity of automotive software, accelerate development speed without compromising safety and provide the essential tools for efficient building, testing and deployment in a highly regulated environment.

    What Are Connected Cars?

    Connected cars are vehicles equipped with internet access, allowing them to share data with devices both inside and outside the car. Essentially, they offer a multitude of features that make the driving experience more convenient, safe and enjoyable. While this technology is still in its nascent stages, it’s rapidly evolving, and soon, connected cars will become a staple on our roads.

    These hyper-connected vehicles are part of a broader ecosystem that encompasses a network of physical objects—everything from other vehicles and traffic lights to pedestrian crosswalks and parking spaces—all connected by technology.

    The potential of connected cars is immense. They promise to transform our lives in numerous ways, from reducing traffic congestion to improving road safety and revolutionizing the way we travel.

    Key Features and Systems of Connected Cars

    Infotainment Systems

    An integral part of connected cars is the infotainment system. This comprehensive system provides both entertainment and information tailored to the driver’s needs. From navigation and music streaming to voice commands and hands-free phone calls, these systems are designed to enhance the driving experience while ensuring safety.

    Infotainment systems also offer a wide range of services, such as real-time traffic updates, weather forecasts, and points of interest. They can connect with a smartphone, allowing users to access apps, make calls and send messages without taking their eyes off the road.

    Telematics

    Telematics uses GPS and onboard diagnostics to record and transmit data about the car’s performance, location and usage. It allows for real-time tracking of the vehicle, making it particularly useful in fleet management and stolen vehicle recovery.

    Telematics data can also be used to assess driver behavior, monitor fuel efficiency, and predict maintenance needs. For instance, if the car’s engine starts showing signs of trouble, the telematics system can alert the driver and recommend a nearby service center.

    Advanced Driver-Assistance Systems (ADAS)

    Advanced driver-assistance systems (ADAS) are systems that help the driver in the driving process. These systems use sensors, cameras and radar to monitor the vehicle’s surroundings and assist the driver in various ways.

    ADAS can provide features like adaptive cruise control, lane-keeping assist and automatic emergency braking. These systems not only enhance driver comfort but also significantly improve road safety by reducing the risk of accidents.

    Vehicle-to-Everything (V2X) Communication

    Vehicle-to-everything (V2X) communication is a key technology in connected cars. It enables vehicles to communicate with any entity that may affect the vehicle and vice versa. This includes other vehicles (V2V), infrastructure (V2I), pedestrians (V2P) and networks (V2N).

    V2X communication can improve traffic flow, reduce accidents and enhance fuel efficiency. For instance, connected cars can receive information about road conditions, traffic jams or accidents ahead in real-time, allowing the driver to take appropriate action.

    Over-the-Air (OTA) Updates

    Over-the-air (OTA) updates are another key feature of connected cars. Similar to how you receive updates on your smartphone, OTA updates allow car manufacturers to update software, fix bugs or add new features remotely.

    OTA updates can improve vehicle performance, enhance security and extend the vehicle’s lifespan. They can also add new functionality, allowing the car to stay up-to-date with the latest technology without the need for a physical visit to a service center.

    Why DevOps is Essential for Connected Car Development

    Addressing the Complexity of Car Software Systems

    The connected car is a complex ecosystem of software systems. These vehicles have numerous systems that communicate with each other, the driver and the outside world. Managing the development of these systems can be a daunting task, and this is where DevOps strategies come in.

    DevOps aims to shorten the system development life cycle and provide continuous delivery with high software quality. This methodology is particularly suited to the complex software systems of connected cars, as it encourages a holistic view of the development process, ensuring that all components work together seamlessly.

    Moreover, DevOps helps to manage the complexity of car software systems by automating tasks, reducing errors and improving efficiency. The use of automated tools for configuration management, deployment and monitoring means less manual work, fewer mistakes, and quicker problem resolution.

    Enhancing Development Speed Without Compromising Safety

    One of the greatest challenges in connected car development is the need for speed. In this fast-paced industry, companies are under pressure to develop and deploy new features quickly to stay competitive. However, this must not come at the expense of safety, as any faults in the system can have dire consequences.

    With DevOps strategies for connected car development, it’s possible to enhance speed without compromising safety. The DevOps approach emphasizes a culture of rapid and frequent deployments, coupled with rigorous testing to ensure quality. By integrating development and operations, it enables teams to work together more effectively, leading to faster development cycles.

    Furthermore, the use of automation in DevOps allows for quicker detection and resolution of issues. Automated testing tools can run thousands of tests in a fraction of the time it would take a human, ensuring that any faults are identified and fixed before they can cause harm.

    Building, Testing and Deploying Efficiently in a Regulated Environment

    The automotive industry is heavily regulated, with strict safety and emission standards that must be met. These regulations add an extra layer of complexity to the development process, as they must be considered at every stage of design, testing, and deployment.

    DevOps strategies for connected car development are well-equipped to handle this challenge. By implementing a continuous integration/continuous deployment (CI/CD) pipeline, teams can ensure that all changes are tested and validated against regulatory requirements before they are deployed. This not only ensures compliance but also saves time and resources by preventing issues from reaching production.

    Additionally, DevOps encourages transparency and visibility across the development process, making it easier to demonstrate compliance to regulators. Through the use of version control systems, every change can be tracked and traced, providing a clear audit trail.

    DevOps Strategies for Connected Car Development

    Version Control for Connected Car Development

    In any software development project, version control is vital. It allows developers to track and manage changes to their code, making it easier to collaborate and preventing conflicts. In the context of connected car development, version control becomes even more critical due to the complex and interconnected nature of automotive software systems.

    Version control enables multiple teams to collaborate on projects without stepping on each other’s toes, using techniques like branching and merging. Furthermore, version control systems provide a history of changes, which is invaluable for debugging and understanding how the system has evolved over time. This can be particularly useful in a regulated industry like automotive, where there may be a need to provide an audit trail.

    Continuous Integration and Continuous Deployment (CI/CD)

    CI/CD is a core DevOps practice that involves integrating changes to the project regularly and automatically testing those changes to detect and fix problems early. The aim is to provide rapid feedback so that if a defect is introduced into the code base, it can be identified and corrected as soon as possible.

    For connected car development, a robust CI/CD pipeline is essential. It allows for fast-paced development without compromising quality or safety. By automating the build, testing and deployment processes, teams can ensure that new features and updates are delivered quickly and reliably.

    Automated Testing and Quality Assurance

    Given the complexity of connected car systems and the potential consequences of a malfunction, rigorous testing is of paramount importance. However, manual testing is time-consuming and prone to error. That’s where automated testing comes in.

    Automated testing is another key DevOps strategy for connected car development. By writing test scripts that can be run automatically, teams can save time, reduce the likelihood of human error and ensure comprehensive coverage of the system.

    Moreover, automated testing tools can be integrated into the CI/CD pipeline, allowing tests to be run every time changes are made. This provides immediate feedback on the impact of those changes and ensures that any issues are identified and addressed as early as possible.

    Infrastructure-as-Code (IaC) for Connected Car Environments

    The final piece of the DevOps puzzle is infrastructure-as-code (IaC). IaC is the practice of managing and provisioning computing infrastructure through machine-readable definition files rather than physical hardware configuration or interactive configuration tools.

    In the context of connected car development, IaC can be a powerful tool. It allows teams to create and manage their development, testing and production environments in a consistent and repeatable way. This not only enhances efficiency but also ensures that the environments are configured correctly and securely.

    Moreover, by using IaC, teams can automate the creation and teardown of environments, saving time and reducing the risk of human error. IaC also supports version control, enabling teams to track changes and roll back to a previous state if something goes wrong.

    Conclusion

    In conclusion, connected cars represent a significant advancement in the automotive industry, but they also bring a host of development challenges. By adopting DevOps strategies for connected car development, companies can navigate these challenges and unlock the full potential of this exciting technology. From managing complex software systems to ensuring rapid and safe development, DevOps offers a robust and flexible approach that is tailored to the needs of the connected car industry.

  • Vulnerability Management for DevOps Teams: A Practical Guide

    Vulnerability Management for DevOps Teams: A Practical Guide

    Vulnerability management is a process that includes the identification, categorization, prioritization and resolution of security vulnerabilities in software systems. It involves a continuous cycle of protection to ensure that your digital assets are secure and your operations are running smoothly.

    With the rapid evolution of technology, fast-moving cloud-native systems and evolving cyber threats, it’s critical to stay ahead of the game. The goal is to close the gap between the discovery of vulnerabilities and their resolution, thereby minimizing the window of opportunity for potential attacks.

    Vulnerability management is not just about finding and fixing vulnerabilities. It’s also about understanding the nature of the vulnerabilities and the potential impact they could have on your network and systems. This understanding informs the prioritization process, helping teams decide which vulnerabilities need immediate attention and which ones can be addressed later. See this blog post for an in-depth review of vulnerability management.

    Why is Vulnerability Management Important for DevOps Teams?

    With the DevOps approach, development and operations teams work together throughout the software life cycle. This collaboration results in faster development times and more reliable software. However, the fast development cycle also introduces new security challenges. This has given rise to the DevSecOps paradigm, where developers, operations teams, and security experts collaborate during all stages of the development lifecycle.

    Here are a few reasons vulnerability management is a critical part of DevSecOps practices:

    Early Detection of Vulnerabilities

    DevOps teams are under constant pressure to deliver high-quality software quickly. However, the speed of delivery should not compromise the security of the software. Vulnerability management helps DevOps teams identify vulnerabilities early in the development process. This early detection allows teams to address security issues before they become serious problems.

    Early vulnerability detection is not just about finding vulnerabilities; it’s also about understanding the potential impact of those vulnerabilities. It involves analyzing the vulnerabilities to determine their severity and the risk they pose to the system. This analysis informs the prioritization process, ensuring that the most critical issues are addressed first.

    Building Continuous Security Into CI/CD Pipelines

    Integrating vulnerability management directly into continuous integration/continuous deployment (CI/CD) pipelines is essential for maintaining continuous security. CI/CD pipelines automate the steps in software delivery, from code integration to deployment, ensuring faster and more consistent releases. By embedding vulnerability management into these pipelines, DevOps teams can automatically scan for and address security vulnerabilities at each stage of the software development life cycle.

    Incorporating vulnerability management tools into CI/CD pipelines allows for automatic scanning of code repositories, container images, and even infrastructure-as-code (IaC) configurations for known vulnerabilities. This integration means that any security issues are identified as soon as code is committed or containers are built, not just at the deployment stage. The advantage of this approach is twofold: It reduces the risk of deploying vulnerable software, and it allows developers to fix issues in their context of development, making the process more efficient.

    Cost-Effective Development and Operations

    Implementing vulnerability management can lead to cost savings for DevOps teams. By identifying and resolving vulnerabilities early in the development process, teams can avoid the high costs associated with fixing security issues after the software has been deployed. In addition, vulnerability management can help prevent costly security breaches that could damage the company’s reputation and result in financial loss.

    Moreover, vulnerability management can contribute to more efficient operations. By systematically managing vulnerabilities, teams can avoid wasting time and resources on low-priority issues. Instead, they can focus their efforts on addressing the most critical vulnerabilities.

    Enhancing Cross-Team Collaboration

    Vulnerability management can also enhance cross-team collaboration within the DevOps environment and between DevOps and security teams. By providing a clear framework for identifying, categorizing, prioritizing and resolving vulnerabilities, it fosters a shared understanding of security issues among team members. This shared understanding can facilitate communication and cooperation, ultimately improving the software product’s security posture.

    Strategies for Integrating Vulnerability Management Into DevOps Processes

    Adopt a ‘Shift-Left’ Approach

    The ‘shift-left’ approach is a strategy that involves shifting the security measures to the early stages of the DevOps pipeline. Traditionally, security checks and vulnerability assessments were conducted toward the end of the development cycle, often leading to late discovery of security flaws and vulnerabilities.

    By shifting these security measures to the left, i.e., to the earlier stages of the development cycle, DevOps teams can catch and fix security vulnerabilities early on. This approach aligns well with the DevOps philosophy of early error detection and continuous improvement.

    Automation of Vulnerability Scanning and Detection

    Automation is at the heart of DevOps. It allows for rapid, reliable and consistent processes, freeing up the team to focus on what they do best – delivering quality code and features. Automation should also extend to vulnerability management.

    Automating vulnerability scanning and detection means integrating security tools into the DevOps pipeline that can automatically scan the code, detect vulnerabilities and even suggest fixes. This approach ensures that security checks are not skipped or forgotten and that vulnerabilities are dealt with promptly.

    Security as Code

    Another strategy for integrating vulnerability management into DevOps processes is embracing the principle of security-as-code. This principle treats security infrastructure and configurations as code, allowing them to be part of the DevOps pipeline.

    By adopting security-as-code, DevOps teams can ensure that the security configurations and infrastructure are always up-to-date and follow the best practices. Like the rest of the code, security configurations can be version-controlled, reviewed, tested and automatically deployed.

    Collaboration Between Security and DevOps Teams

    One of the key tenets of DevOps is collaboration between different teams. In a DevSecOps paradigm, this collaboration extends to the security team as well. Integrating vulnerability management into DevOps processes fosters close collaboration between the security and DevOps teams.

    The security team can provide valuable insights into the security landscape, vulnerability management tools and best practices. On the other hand, the DevOps team can ensure that these insights and tools are integrated into the DevOps pipeline.

    Continuous Monitoring and Feedback Loop

    Continuous monitoring is crucial for effective vulnerability management. Continuous monitoring ensures that the security state of the system is always known and that any changes or vulnerabilities are promptly detected. The feedback loop ensures that the insights gained from monitoring and vulnerability assessments are fed back into the development process, leading to continuous improvement in security.

    Regular Training and Awareness

    Finally, regular training and awareness are crucial for integrating vulnerability management into DevOps processes. DevOps teams should be trained in security best practices, threat landscape, and use of vulnerability management tools.

    Regular training ensures that the team stays up-to-date with the latest security trends and vulnerabilities. Awareness sessions can help the team understand the importance of security and vulnerability management, encouraging them to take an active role in securing the system.

    Conclusion

    In conclusion, integrating vulnerability management into DevOps processes is not only possible but crucial for secure and reliable systems. Adopting a shift-left approach, automating vulnerability scanning and detection, embracing security-as-code, fostering collaboration, implementing continuous monitoring and feedback and ensuring regular training and awareness are some of the strategies that can help achieve this integration. 

    Remember, vulnerability management is an ongoing process that requires commitment, collaboration and continuous improvement.

  • PagerDuty Previews Generative AI Copilot for ITSM

    PagerDuty Previews Generative AI Copilot for ITSM

    Under an early access program, PagerDuty, Inc. is making available a tool that brings generative artificial intelligence (AI) capabilities to its IT service management (ITSM) platform.

    Jonathan Rende, senior vice president of products for PagerDuty, said PagerDuty Copilot for the PagerDuty Operations Cloud extends previous investments in machine learning algorithms the company has made as part of an ongoing effort to apply AI to ITSM.

    Designed to be invoked via Slack, PagerDuty Copilot makes use of multiple large language models (LLMs) to automate tasks ranging from providing summarization of IT incidents to creating code to automate workflows. PagerDuty plans to transparently swap LLMs in and out of its platforms as AI advances continue to be rapidly made, noted Rende.

    PagerDuty Copilot provides a level of abstraction for invoking AI models along with appropriate guardrails that make it simpler to manage IT operations without IT teams needing to have AI expertise, said Rende.

    The overall goal is to boost the productivity of IT operations teams by eliminating much of the drudgery and toil that conspires to make working in IT tedious, noted Rende. AI technologies are not likely to replace the need for IT personnel as much as they will enable IT teams to focus on tasks that add more value to the business, he added.

    It’s now only a matter of time before generative AI capabilities are pervasively applied across both ITSM and DevOps workflows. Less clear is the impact those capabilities will have on the best practices currently relied on to manage those workflows as more tasks are automated. Ultimately, however, AI should make it easier for more organizations to embrace those best practices as the level of skill and expertise required to manage IT at scale is reduced.

    In addition, the whole concept of issuing tickets to manage tasks tracked by a central system of record may need to evolve simply because AI has automated requests for service. There will naturally need to be some system of record for tracking requests. Still, ultimately that process will be managed via copilots rather than by a ticket created by an end user that is then tracked via an ITSM platform.

    Savvy IT teams, in the meantime, are already moving to determine which tasks and workflows will be automated in anticipation of AI becoming more widely embedded in ITSM and DevOps platforms. Roles and responsibilities within IT teams will inevitably evolve as AI increasingly automates mainly mundane tasks, such as creating reports that many IT professionals would rather not spend time writing.

    The biggest IT management platform challenge in the future might not necessarily be adjusting to AI as much as it will be orchestrating requests that are likely to be generated by multiple types of copilots that have been embedded into almost every application.

    One way or another, AI is about to transform how IT operations are managed. As disruptive as those advances will be, AI, more importantly, will also enable organizations to manage IT at levels of scale that were previously unimaginable.

  • Synopsys Report Sees Steady Application Security Gains

    Synopsys Report Sees Steady Application Security Gains

    An analysis of the security automation practices of 130 organizations published today by Synopsys suggests significant progress has been made toward securing software supply chains in the last year. However, much work still remains to be done.

    The annual Building Security In Maturity Model (BSIMM) report found there has been a 22% increase in the number of organizations creating software bills of materials (SBOMs) this year. Additionally, there has been a 10% increase in the number of organizations tracking open source software risks.

    At the same time, there has been a 25% increase in the number of organizations that are fixing all defects found in software and a 15% increase in the number of organizations that can identify potential attackers and track attack patterns.

    The report also sees significant gains in activities related to software development practices (44%), penetration testing (35%) and compliance and policy controls (21%). Usage of application behavior monitoring and diagnostics grew 64% and monitoring automated asset creation grew 45%, respectively.

    The finding and publishing of secure design patterns, the required use of approved security features and frameworks and the use of application containers to support security goals all grew about 25%, according to the report.

    However, the report also noted that usage of possible attack lists dropped by 31%, while expert-driven tasks such as building and applying adversarial security tests declined 25% and use of centralized defect reporting dropped 18%.

    The most widely used processes are implementing security checkpoints and associated governance (91%), creating or interfacing with incident response (90%), identifying privacy obligations (88%), using external penetration testers to find problems (88%), ensuring host and network security basics are in place (87%) and automated code review tools (86%).

    Jamie Boote, associate principal security consultant for the Synopsys Software Integrity Group, said the report suggests more responsibility for cybersecurity is shifting everywhere as organizations rely more on automation to apply security controls. Application development and IT operations are, as a result, able to address vulnerabilities more proactively either before an application is deployed or as soon as a vulnerability is discovered, he noted.

    The challenge organizations will encounter when it comes to automating security processes is there is a tendency to set and forget. Once a process becomes automated, not enough organizations will reevaluate that process as the cybersecurity threat landscape continues to evolve, noted Boote.

    Of course, the rate at which security processes are being automated varies widely from one organization to the next, but the more automated they become, the less dependent organizations become on application developers who typically don’t have a lot cybersecurity expertise. All too often, application developers are rushing to meet software delivery deadlines at the expense of security best practices. Unfortunately, it’s usually only a matter of time before an issue that was avoided is later discovered in a production environment and is considerably more problematic to update, given all the dependencies between software and infrastructure that exist.

    One way or another, however, a pending wave of regulations will soon force organizations to pay more attention to securing software supply chains. The only thing left to determine in the absence of automation is how painful the reengineering of DevOps workflows to embrace DevSecOps best practices is going to be.