devopsy.kinsta.cloud

Tag: DevOps workflows

Broadcom Virtual Cloud Foundation Update Streamlines DevOps Workflows

Broadcom at the VMware Explore 2024 conference today updated its VMware Cloud Foundation (VCF) platform to provide both deeper levels of integration, multitenancy capabilities and a single console through which all VCF components can be managed.

Paul Turner, vice president of products for the VCF division of Broadcom, said VCF 9 will substantially reduce the total cost of ownership (TCO) by, for example, eliminating the need for more than a dozen consoles that were previously required.

The overall goal is to make it simpler for IT teams to manage private clouds that can be deployed either in an on-premises IT environment or on a public cloud service, said Turner.

As part of that effort, Broadcom is also making virtual private cloud (VPC) maturity assessment, training and jumpstart services.

Additionally, Broadcom is adding an Advanced Memory Tiering with NVMe capability to VCF 9 that makes it more cost-effective to run data-intensive applications. VCF 9 also makes it possible to manage fleets of virtual machines, including security operations augmented by generative artificial intelligence (AI), and deploy virtual machines as much as 61% faster.

At the same time, network virtualization elements of the platform based on VMware NSX are better integrated with VMware ESXi to streamline operations, noted Turner. Virtual private clouds (VPCs) can also now take advantage of services such as VMware vDefend to enable security groups and VMware Avi Load Balancer to enable one-click deployment of load-balancing capabilities. The VMware hypervisor can now also serve as a threat for vDefend.

A VCF Import tool will make it easier to import VMware NSX, VMware vDefend, VMware Avi Load Balancer and more complex storage topologies into existing VCF environments. VCF 9 also now supports native vSAN-to-vSAN Data Protection with Deep Snapshots to improve resiliency.

Finally, Broadcom is making it easier to deploy VMware Private AI Foundation with NVIDIA capabilities on top of VCF 9 by adding support for virtual graphics processor unit (vGPU) profile visibility, GPU reservations, a data indexing and retrieval service, and an AI agent builder service.

It’s not clear how many enterprise IT organizations are signing up for VCF. Broadcom is providing additional capabilities to make VCF attractive, but many IT organizations have already invested in other networking, storage and security platforms. In addition, many DevOps teams have adopted alternative platforms for accelerating the building and deployment of applications that don’t necessarily depend on what virtual machine is employed.

Many IT organizations are also still trying to determine what their total costs under a subscription license are compared to the perpetual licensing model for VMware which is no longer an option. The primary issue is not so much migrating workloads but rather the degree to which it makes economic sense to deploy additional workloads on VMware platforms.

At a time when IT organizations are increasingly sensitive to the total cost of IT, however, there is a case to be made for consolidation. The challenge is convincing IT teams they will not become beholden to a single vendor that might in the future increase licensing fees after they no longer have any other viable alternatives.

August 27, 2024
Implementing Threat Modeling in a DevOps Workflow
Security remains paramount in the contemporary digital landscape, especially within the DevOps workflow. DevOps practices aim to streamline the software development lifecycle (SDLC) by fostering collaboration between development and operations teams, ensuring a secure SDLC. However, the rapid pace of DevOps can sometimes overlook critical security concerns. Integrating threat modeling into the DevOps workflow is essential to identify and mitigate potential security threats. This comprehensive guide explores how threat modeling can be effectively implemented within a DevOps framework to enhance security.

Understanding Threat Modeling

Threat modeling is a structured approach to identifying, evaluating and mitigating security threats to a system. It involves anticipating potential attacks, understanding their impact and devising strategies to defend against them. This proactive approach to security ensures that vulnerabilities are identified early in the development process, reducing the risk of security breaches post-deployment.

Key Elements of Threat Modeling
- Assets Identification: Determining what needs to be protected, including data, systems and services
- Threat Identification: Recognizing potential threats that could exploit vulnerabilities
- Vulnerability Identification: Identifying weaknesses in the system that could be exploited
- Mitigation Strategies: Developing countermeasures to address identified threats and vulnerabilities.
Integrating Threat Modeling into the DevOps Workflow

Integrating threat modeling into a DevOps workflow involves embedding security practices throughout the development and operations lifecycle. This approach ensures continuous security assessment and improvement, aligning with the DevOps principles of continuous integration and continuous deployment (CI/CD).

1. Shift-Left Security: Integrating Early in the SDLC

One of the fundamental principles of DevOps is the concept of ‘shift-left’ security, which means incorporating security practices early in the SDLC. By integrating threat modeling at the initial stages, potential security threats can be identified before they become deeply embedded in the codebase. This early integration supports a secure SDLC, ensuring that security measures are considered from the beginning and throughout the development process.

Benefits of Shift-Left Security
- Early Detection of Vulnerabilities: Identifying security issues early reduces the cost and effort required to fix them later
- Improved Code Quality: Encouraging developers to write secure code from the beginning
- Reduced Time-to-Market: Minimizing the risk of delays caused by security issues discovered late in the development process.
2. Continuous Threat Modeling

Continuous threat modeling aligns with the DevOps principle of continuous integration and deployment. This practice involves regularly updating threat models to reflect changes in the application, infrastructure and threat landscape. Automated tools can assist in maintaining up-to-date threat models, ensuring that security assessments are conducted consistently.

Key Aspects of Continuous Threat Modeling
- Automated Security Scans: Integrating security scanning tools into the CI/CD pipeline to detect vulnerabilities continuously
- Regular Updates: Revisiting and updating threat models as the application evolves
- Monitoring and Feedback: Utilizing feedback loops to improve threat models based on new information and detected threats.
3. Collaboration Between Development, Operations and Security Teams

Effective threat modeling in a DevOps environment requires close collaboration between development, operations and security teams. This cross-functional collaboration ensures that security is considered at every stage of the development lifecycle.

Strategies for Enhancing Collaboration
- Security Champions: Appointing security champions within development and operations teams to advocate for security best practices
- Regular Security Training: Providing ongoing security training to developers and operations personnel to raise awareness and build skills
- Shared Responsibility: Promoting a culture of shared responsibility for security across all teams.
4. Integrating Threat Modeling Tools

Numerous tools are available to facilitate threat modeling within a DevOps workflow. These tools can automate various aspects of threat modeling, making it easier to identify and mitigate potential security threats.

Popular Threat Modeling Tools
- OWASP Threat Dragon: An open-source tool for creating threat models and visualizing potential threats
- Microsoft Threat Modeling Tool: Provides a systematic approach to threat modeling with pre-defined templates
- IriusRisk: A platform that integrates with CI/CD pipelines to automate threat modeling and risk assessment
5. Case Studies and Statistics

Integrating threat modeling into a DevOps workflow is essential for identifying and mitigating potential security threats before they become significant issues. Real-world case studies and statistical evidence can illuminate the tangible benefits of this approach. This section delves into notable case studies, such as the Capital One and Equifax data breaches, to underscore the critical importance of threat modeling and continuous monitoring in DevOps practices.

Capital One Data Breach

In 2019, Capital One experienced one of the most significant data breaches in recent history, exposing the personal information of over 100 million customers. The breach was caused by a misconfigured web application firewall (WAF), which allowed an attacker to access Capital One’s Amazon Web Services (AWS) cloud environment. This incident highlights several critical points about the importance of continuous threat modeling and monitoring in a DevOps workflow.

Capital One’s breach underscores the necessity of rigorous configuration management and monitoring. A misconfiguration, such as the one that occurred in this case, can have devastating consequences if not detected and corrected promptly. Continuous threat modeling could have identified this misconfiguration early, allowing Capital One to address the issue before it was exploited. Furthermore, regular security assessments and automated monitoring tools could have alerted the security team to the vulnerability in time to prevent the breach.

The Capital One breach also illustrates the importance of a comprehensive approach to security that includes not only technical measures but also organizational practices. By fostering a culture of security awareness and implementing robust security practices throughout the DevOps lifecycle, organizations can reduce the risk of similar incidents. Continuous threat modeling ensures that security is an ongoing consideration, rather than a one-time task, and helps to create an environment where potential threats are identified and addressed proactively.

Equifax Data Breach

The 2017 Equifax data breach exposed the personal information of approximately 147 million people, making it one of the largest and most damaging breaches in history. The root cause of the breach was an unpatched vulnerability in Apache Struts, a widely used web application framework. This incident highlights the critical need for integrating automated security scanning tools into the CI/CD pipeline, a key aspect of threat modeling in DevOps.

The Equifax breach could have been prevented if the vulnerability had been identified and patched in a timely manner. Automated security scanning tools, integrated into the CI/CD pipeline, can continuously monitor for known vulnerabilities and alert developers to the need for patches. By integrating such tools, organizations can ensure that their applications are always up to date with the latest security fixes, reducing the risk of exploitation by attackers.

Moreover, the Equifax breach emphasizes the importance of a proactive security posture. In a DevOps environment, where rapid development and deployment are the norms, it is crucial to have automated systems in place that can keep up with the pace of change. Continuous threat modeling and automated security scanning provide a means to do this, enabling organizations to identify and mitigate vulnerabilities as they arise, rather than after they have been exploited.

Statistics Supporting Threat Modeling in DevOps

Statistical evidence further reinforces the value of integrating threat modeling into DevOps practices. According to Gartner, “By 2022, 90% of software development projects will claim to be following DevSecOps practices, up from 40% in 2019.” This significant increase reflects a growing recognition of the importance of integrating security into the DevOps workflow. Organizations that adopt DevSecOps practices, including continuous threat modeling, are better positioned to identify and address security threats early, reducing the risk of breaches and other security incidents.

The State of DevOps Report 2021 by Puppet provides additional insights. The report indicates that organizations integrating security into their DevOps practices are 2.4 times more likely to detect security incidents before they cause significant damage. This finding underscores the effectiveness of proactive security measures, such as continuous threat modeling, in identifying and mitigating threats before they can have a detrimental impact.

A survey by the SANS Institute revealed that 64% of respondents experienced improved security posture after implementing DevSecOps practices. This improvement is likely due to the continuous nature of threat modeling and security assessments in a DevOps environment, which helps organizations stay ahead of emerging threats. By continuously updating threat models and integrating automated security tools, organizations can maintain a robust security posture that adapts to new challenges and vulnerabilities.

Benefits of Integrating Threat Modeling into DevOps

The case studies and statistics discussed highlight several key benefits of integrating threat modeling into a DevOps workflow:
- Early Detection of Vulnerabilities: Continuous threat modeling and automated security scanning tools enable organizations to identify vulnerabilities early in the development process, reducing the risk of exploitation.
- Proactive Security Posture: By continuously assessing and updating threat models, organizations can maintain a proactive security posture that adapts to new threats and vulnerabilities.
- Improved Security Awareness: Integrating threat modeling into DevOps fosters a culture of security awareness, encouraging all team members to prioritize security in their work.
- Reduced Risk of Breaches: Continuous threat modeling helps organizations identify and address security threats before they become significant issues, reducing the risk of data breaches and other security incidents.
Statistics
- According to a report by Gartner, “By 2022, 90% of software development projects will claim to be following DevSecOps practices, up from 40% in 2019.”
- The State of DevOps Report 2021 by Puppet indicates that organizations that integrate security into their DevOps practices are 2.4 times more likely to detect security incidents before they cause significant damage.
- A SANS Institute survey revealed that 64% of respondents experienced improved security posture after implementing DevSecOps practices.
6. Best Practices for Implementing Threat Modeling in DevOps

Implementing threat modeling within a DevOps workflow is essential to ensuring that security is embedded into the SDLC from the outset. By following best practices, organizations can maximize the effectiveness of their threat modeling efforts, leading to more secure and resilient applications. The following sections expand on the best practices for implementing threat modeling in a DevOps environment.

Early and Continuous Integration

Integrate Threat Modeling Early

The concept of ‘shift-left’ security emphasizes the importance of integrating security measures at the earliest stages of the SDLC. By incorporating threat modeling during the initial phases of development, potential security issues can be identified before they become deeply ingrained in the codebase. Early integration of threat modeling ensures that security considerations are a fundamental part of the design and architecture process.

Continuous Updating of Threat Models

Threat models should not be static documents that are created once and forgotten. Instead, they need to be living artifacts that are continuously updated to reflect changes in the application, infrastructure and the evolving threat landscape. Continuous updating ensures that the threat models remain relevant and effective in identifying and mitigating potential security threats.

Utilize Automated Tools

Automated tools play a crucial role in facilitating continuous threat modeling and security assessments. Tools such as OWASP Threat Dragon, Microsoft Threat Modeling Tool and IriusRisk can automate various aspects of threat modeling, making it easier to integrate these practices into the CI/CD pipeline. Automation helps ensure that threat modeling is performed consistently and efficiently, reducing the burden on development and security teams.

Collaborative Approach

Foster Cross-Functional Collaboration

Effective threat modeling requires close collaboration between development, operations and security teams. This cross-functional approach ensures that security is considered from multiple perspectives and throughout the entire development lifecycle. Collaboration can be fostered through regular meetings, joint workshops and shared documentation.

Appoint Security Champions

Appointing security champions within development and operations teams can help promote a security-first mindset. These individuals act as advocates for security best practices, ensuring that security considerations are prioritized and integrated into everyday workflows. Security champions also serve as points of contact for security-related questions and issues, bridging the gap between different teams.

Provide Regular Security Training

Ongoing security training is essential to keeping all team members informed about the latest threats, vulnerabilities and best practices. Training should cover fundamental security principles, specific threats related to the technologies being used and hands-on exercises in threat modeling and mitigation. Regular trainings help build a culture of security awareness and ensure all team members have the knowledge and skills needed to contribute to threat modeling efforts.

Leverage Automated Tools

Automate Threat Identification and Mitigation

Threat modeling tools can automate the identification and mitigation of potential security threats, making the process more efficient and consistent. Tools such as OWASP Threat Dragon, Microsoft Threat Modeling Tool, and IriusRisk offer features such as visualizing threat models, suggesting mitigations and integrating with CI/CD pipelines. Automation reduces the manual effort required for threat modeling and helps ensure that security assessments are performed regularly.

Integrate Security Scanning Tools into the CI/CD Pipeline

Integrating security scanning tools into the CI/CD pipeline allows for continuous vulnerability detection. Tools such as Snyk, Veracode and Checkmarx can automatically scan code, dependencies and configurations for vulnerabilities as part of the build and deployment process. Continuous scanning helps identify security issues early, providing developers with immediate feedback and allowing them to address vulnerabilities before they reach production.

Regular Reviews and Updates

Regularly Review and Update Threat Models

Regular reviews and updates of threat models are essential to ensuring their continued effectiveness. As applications evolve and new features are added, threat models must be revisited to account for these changes. Regular reviews should be scheduled at key milestones in the SDLC, such as before major releases or after significant architectural changes.

Implement Monitoring and Feedback Loops

Monitoring and feedback loops are critical to continuously improving threat models. By collecting data on security incidents, vulnerabilities and threat model effectiveness, organizations can gain insights into areas for improvement. Feedback loops should involve regular debriefs and post-mortem analyses to understand what worked well and what could be improved. This iterative approach helps refine threat models and ensures that they remain relevant and effective.

Metrics and Reporting

Track Key Security Metrics

Tracking key security metrics is essential for measuring the effectiveness of threat modeling efforts. Metrics such as the number of vulnerabilities identified and remediated, time to resolve security issues and the impact of security incidents can provide valuable insights into the security posture of the organization. These metrics should be tracked over time to identify trends and measure progress.

Use Metrics to Identify Areas for Improvement

Metrics can be used to identify areas for improvement in threat modeling and overall security practices. For example, if certain types of vulnerabilities are consistently identified, it may indicate a need for additional training or changes to development practices. Metrics can also help prioritize security efforts, ensuring that resources are focused on the most critical areas.

Demonstrate Value to Stakeholders

Reporting on key security metrics helps demonstrate the value of threat modeling to stakeholders. By showing the tangible benefits of threat modeling, such as reduced vulnerabilities and faster resolution times, organizations can build support for continued investment in security practices. Regular reporting also helps keep stakeholders informed about the security posture of the organization and any emerging threats.

What’s Next?

Implementing threat modeling in a DevOps workflow is crucial for proactively identifying and mitigating potential security threats. By following best practices — such as early and continuous integration, fostering cross-functional collaboration, leveraging automated tools, regularly reviewing and updating threat models and tracking key security metrics — organizations can enhance their security posture and build more resilient applications. As the digital landscape continues to evolve, the importance of proactive security measures like threat modeling cannot be overstated. By embedding security into every stage of the SDLC, organizations can ensure that their applications remain secure and resilient against emerging threats.
July 18, 2024
Fauna Adds Declarative Tool to Update Namesake Database

Fauna has added an ability to maintain strict enforcement of schema to its document-relational database using a declarative language that makes it simpler to update databases within the context of a DevOps workflow.

While Fauna has enabled IT teams to extend a document database to add tables and rows typically requiring a relational database, Fauna Schema adds the structure, enforcement and reliability of a schema typically associated with traditional relational databases.

Fauna CEO Eric Berg said the addition of the approach strengthens the case of using a Fauna database that makes it simpler to streamline the management of databases that can be used to support multiple types of applications. In addition, Fauna now enables database schemas to evolve as the needs of the business evolve using a new Document Types capability that allows developers to define and enforce schema structures directly within the database. Document Types can support both static and dynamic typing, enabling developers to start with a schemaless document database before adding stricter type controls in a way that doesn’t require a database to be taken offline, said Berg.

At the core of that capability is the Fauna Schema Language (FSL), which enables developers to declaratively define their domain models, access controls, and business logic in a human-readable language in a set of files that can be managed alongside their application code.

Using a code repository such as GitHub, DevOps teams can now coordinate schema and application code updates via a continuous integration/continuous delivery (CI/CD) pipeline. Additionally, IT teams can leverage infrastructure-as-code (IaC) tools such as Pulumi and Terraform to automate the deployment and management of their Fauna database configuration.

Fauna also added a Computed Fields capability that allows the values of fields in documents to be dynamically generated based on expressions defined by the user. That approach allows developers to create fields whose values are calculated at the time of query to enable a dynamic composition of objects and field values. These fields can be indexed, providing even greater flexibility in query patterns so developers can accurately define their data model in the database versus requiring them to be defined in mapping layers running outside of the database.

Encoding that database logic within the database allows development teams to define cross-collection joins and subqueries as an element of their schema design to dynamically generate data, and object models in real-time. That capability avoids the performance disadvantages inherent in a data denormalization strategy required by other document databases.

It’s not clear to what degree IT organizations are moving to consolidate databases, but with the rise of document databases, it has become relatively simple for developers to add a database to just about any type of application. The challenge organizations encounter is that as document databases evolve, they become more challenging to manage. It’s not uncommon for developers to pass the responsibility for managing document databases on to a DevOps team that often has a vested interest in consolidation.

After all, the more databases based on disparate architectures there are to manage the more challenging it becomes to manage DevOps workflows at any level of meaningful scale.

July 1, 2024
Shreds.AI Launches LLM Platform Trained for Software Engineering

Shreds.AI today unveiled a namesake generative artificial intelligence (AI) platform based on a large language model (LLM) it trained to specifically automate software engineering tasks.
Available in beta, the Shreds.AI platform can assign tasks to as many as eight other LLMs by invoking the application programming interfaces (APIs) they expose.

Shreds.AI CEO Soufiane Amar said rather than producing small amounts of code the platform has been trained to create the tens of thousands of lines and files of code needed to drive complex software engineering workflows. The Shreds.AI platform will also accurately orchestrate the integration of various software components to build an application because it has been trained using tools that developers regularly employ, he added.

A developer inputs a simple natural language description of the software that they want Shreds.AI to create, and the platform then generates architectural diagrams and the code for independent and isolated features called shreds. DevOps teams only need to validate the code before employing it, a process Shreds.AI makes simpler via a network of independent developers that organizations can contract to review code, said Amar.

Shreds.AI is a Meta AI, in that in addition to generating code and reasoning across processes, is able to rank third-party LLMs based on their ability to perform specific tasks, he noted.

The Shreds.AI platform is already being tested by the automotive conglomerate Stellantis and Réseau de Transport d’Électricité (RTE), the electricity transmission system operator of France. Shreds.AI estimates that an application that previously might have cost $1 million can now be created for less than $30,000. The company claims Shreds.AI slashes the time to market for software, along with team sizes and costs, by over 80% compared to traditional software development methods. By enabling automatic maintenance, it also solves the software obsolescence problem. It increases software lifespan by more than 60% by, for example, making it easier to convert the programming language used to build that application into another language that more developers know.

It’s still early days so far as the incorporation of AI into DevOps workflows is concerned and the code generated by these platforms still needs to be managed. The challenge is that thanks to the rise of AI it’s expected that the amount of software that will be built and deployed in the next two years will exceed the amount of software that has been deployed in the past two decades. The only way to keep pace with that rate of development will be to also apply AI to the management of DevOps workflows as well.

In the hope of eliminating as much existing toil as possible, DevOps teams should be identifying which manual processes they regularly perform today, with an eye towards soon applying AI to them tomorrow. After all, the whole point of embracing DevOps in the first place was to ruthlessly automate as many software engineering processes as possible, to enable more applications to be built and deployed as quickly as possible.

June 13, 2024
CircleCI Extends CI/CD Reach to AI Models

CircleCI this week revealed it is extending the reach of its continuous integration/continuous delivery (CI/CD) platform to make it simpler to incorporate artificial intelligence (AI) models into DevOps workflows.

In addition to providing access to the latest generation of graphical processor units (GPUs) from NVIDIA via the Amazon Web Services (AWS) cloud, Circle CI has added inbound webhooks to access AI model curation services from providers such as Hugging Face and integrations with LangSmith, a debugging tool for generative AI applications and the Amazon SageMaker service for building AI applications.

CircleCI CEO Jim Rose said while there is clearly a lot of enthusiasm for incorporating AI models into applications, the processes being used are still immature, especially in terms of automating workflows that include testing of probabilistic AI models.

Most AI models are built by small teams of data scientists that create a software artifact that needs to be integrated within a DevOps workflow just like any other artifact, noted Rose. The challenge is that most data science teams have not yet defined a set of workflows for automating the delivery of those artifacts as part of a larger DevOps workflow, he added.

DevOps teams will also need to make adjustments to a version control-centric approach to managing applications to trigger pipelines to pull AI software artifacts that exist outside of traditional software repositories. For example, the inbound webhooks provided by CircleCI now make it possible to automatically create a pipeline whenever an AI model residing on Hugging Face changes.

It’s still early days as far as the deployment of AI models in production environments is concerned, but there is no doubt generative AI will have a major impact on how software is developed. AI models are a different class of software artifacts that are retrained instead of being updated, a process that occurs intermittently. As such, DevOps teams need to keep track of each time an AI model is being retrained to ensure applications are updated.

At the same time, generative AI will also increase the pace at which other software artifacts are being created and deployed. Many of the manual tasks that today slow down the rate at which applications are built and deployed will be eliminated. That doesn’t mean there will be no need for software engineers, but it does mean the role they play in developing and deploying software is about to rapidly evolve. DevOps teams need to evaluate both how generative AI will impact the tasks they manage as well as the way the overall software development life cycle (SDLC) process needs to evolve.

Each organization, as always, will need to decide for itself how best to achieve those goals depending on the use cases for AI, but the changes that generative AI will bring about are now all but inevitable. The longer it takes to adjust, the harder it will become to overcome the cultural and technical challenges that will be encountered along the way.

November 30, 2023
OpsCanvas Generates Code to Automate DevOps Workflows

OpsCanvas, Inc. today emerged from stealth to unveil a Draw & Deploy platform that generates code to provision cloud infrastructure.

OpsCanvas CEO Brian Kathman said as an alternative to traditional infrastructure-as-code (IaC) tools, this offering makes DevOps practices more accessible to a wider number of organizations.

The OpsCanvas platform analyzes application architecture diagrams to deterministically create known-good code that the company has vetted to replace scripts DevOps teams would otherwise have to create. The platform then, via a single click, makes it possible to automate common tasks for deploying, cloning, updating, rolling back, extending and decommissioning resources.

Unlike generative artificial intelligence (AI) platforms, OpsCanvas does not use a large language model (LLM) to generate code. Instead, vetted code is created based on the diagrams that the platform creates. OpsCanvas also encodes best practices, standardized network topology for applications and provides integration with multiple continuous integration/continuous delivery (CI/CD) platforms.

As a result, DevOps teams are provided with a comprehensive view of diagrams, including the number of stored versions and deployment status, for every application deployed.

The goal is to enable organizations of any size to manage entire DevOps workflows using a platform that can generate more than 10,000 lines of already-tested, custom configurations in less than 30 seconds, said Kathman.

In effect, OpsCanvas aims to democratize DevOps by eliminating the need for specialized skills to provision and deploy applications. In addition to accelerating the pace at which applications are built and deployed, that approach will also make it feasible for more small-to-medium businesses (SMBs) to embrace DevOps, noted Kathman.

Regardless of the path chosen to make DevOps more accessible, the days when software engineers wrote scripts to provision cloud infrastructure are now numbered. Either using code that has been vetted by platforms such as OpsCanvas or relying on co-pilots to generate code using large language models (LLMs), provisioning and deploying applications is about to become both simpler and more secure.

Many of the cybersecurity issues that organizations experience today can be traced back to misconfigured infrastructure provisioned by developers using IaC tools. As generative capabilities become more widely used, the quality of the code used to provision infrastructure should substantially improve. As that transition occurs, DevOps engineers should be able to allocate more of their time to address more complex challenges.

One way or another, DevOps is about to become less tedious in ways that should help reduce burnout that leads to turnover. At the same time, the number of organizations capable of embracing DevOps best practices should increase as it becomes simpler for IT administrators who lack programming skills to provision infrastructure.

In fact, the velocity at which software is built and deployed today is about to exponentially increase in a way that most DevOps professionals never imagined might be possible.

October 24, 2023
OpsMX Adds Deployment Firewall to Ensure Compliance

OpsMx today added a deployment firewall to its Deploy Shield portfolio for securing continuous integration/continuous delivery (CI/CD) platforms.

The deployment firewall automatically applies rules to ensure DevOps teams comply with mandates such as the payment card industry data security standard (PCI DSS) or the Health Insurance Portability and Accountability Act (HIPAA). It then evaluates a release against a checklist of policies and blocks any deployments into a production environment unless a specific exception has been defined. In those instances, DevOps teams can also set an alert to address that issue at a later date.

In addition, DevOps teams can extend those rules defined by OpsMX to address specific compliance requirements.

The checklist includes making sure vulnerability scams have been run, tracking when the manifest was last updated, confirming that operational controls are being followed, checking whether images have been modified and evaluating the infrastructure being employed and performing an analysis of the performance impact.

A simulation feature also allows developers to check their release for compliance before it is time to deploy.

OpsMx CEO Gopal Dommety said the goal is to streamline the effort that is required to meet multiple compliance mandates as the pace at which applications are being developed and deployed continues to increase.

Today there are simply too many data silos for DevOps teams to manage, with each one presenting yet another opportunity to run afoul of a compliance mandate, he added. The overall goal is to reduce the cognitive load being placed on developers as more responsibility for compliance is shifted left, noted Dommety.

The deployment firewall currently supports DevOps platforms such as Jenkins, Argo and Spinnaker, with support for GitHub Actions and GitLab to follow.

OpsMX claims its overall customer base has increased by more than 60% in the first half of this year, with customers including Google, Cisco and Western Union. In addition to Deploy Shield, the company provides a Delivery Bill of Materials (DeliveryBOM) tool to capture a consolidated record of every step in the application software delivery and deployment process. Security checks, approvals, policy enforcement and audits are tracked via one tool. In addition, the company provides tools for managing the deployments of software via either the Argo or Spinnaker CD platforms.

It’s now only a matter of time before various regulations that apply to the building and deploying applications become more stringent. Tolerance for sloppy application development processes is dropping as legislative bodies increasingly determine that software development practices are a big cybersecurity problem that threatens national security and the global economy.

Organizations that fail to comply with those regulations will be subject to even heftier fines than the ones that already exist. In many cases, that may slow down the rate at which applications are deployed unless DevOps teams find ways to manage compliance-as-code more effectively as part of a larger DevSecOps workflow.

When it comes to DevSecOps, progress tends to be slow in most organizations, but as more tools become available to automate various tasks, it’s now only a matter of time before software supply chains are more tightly secured. Hopefully, the pace at which those advances are being made will occur faster than cybercriminals can exploit the vulnerabilities that today are too numerous to manually track.

October 23, 2023
Dynatrace Survey Surfaces DevOps Automation Gains and Challenges

A global survey of 450 DevOps and security practitioners published this week by Dynatrace found investments in automation have improved software quality (61%), reduced deployment failures (57%) and decreased IT costs (55%).

However, only 38% of respondents said their organization had a clearly defined DevOps automation strategy. On average, organizations have succeeded in automating just over half (56%) of their end-to-end DevOps life cycle, the survey found.

Saif Gunja, director of product marketing for Dynatrace, said the survey makes it clear that automation is still being applied to isolated workflows rather than centrally managed. That is changing, however, as more organizations embrace platform engineering to manage DevOps workflows at scale, he added.

Overall, organizations are investing in DevOps automation over the next 12 months to improve security and compliance management (55%), infrastructure provisioning and management (52%) and performance optimization (51%).

The biggest barriers preventing organizations from automating new DevOps use cases are security concerns (54%), difficulty operationalizing data (54%) and toolchain complexity (53%). The average organization relies on more than seven different tools for DevOps automation, the survey finds.

As a result, the automation that has been put in place is often too brittle to effectively scale beyond a small DevOps team, noted Gunja.

A full 71% of respondents also used observability data and insights to drive automation decisions and improvements in DevOps workflows. However, 85% also admitted they faced challenges using observability and security data to drive DevOps automation. The top three challenges facing organizations included inaccessible data (51%), siloed data (43%) and the need for data to flow through many systems (41%).

Well over half (54%) said their organization is investing in platforms to enable easier integration of tools and collaboration between teams involved in automation projects.

Finally, 59% also anticipated large language models (LLMs) such as ChatGPT and Bard would have a significant impact on DevOps, with the top three benefits expected including increased productivity and reduced manual effort (57%), improved development, security and operations collaboration (56%) and enabling teams to generate code automatically (48%).

Many organizations that have embraced DevOps are now looking to strike a balance between the concept of developers owning every application or service they developed and the need to reduce the amount of toil developers now encounter, noted Gunja. Developers typically only spend about a quarter of their time writing code so there is a clear need to reduce the current level of cognitive load developers encounter as they maintain application development environments, he added.

The challenge is finding a way to reduce that load in a way that doesn’t overly restrict the ability of developers to embrace new tools and platforms that will enable them to build applications in a more innovative fashion. One of the primary reasons many development teams embraced DevOps in the first place was to not have the way they build applications dictated to them by a centralized IT team.

One way or another, DevOps workflows will continue to be increasingly automated. Less clear at the moment, however, is exactly who within an organization will be driving those initiatives.

September 28, 2023
Buildkite Acquires Packagecloud to Streamline DevOps Workflows

Buildkite Pty Ltd today revealed it has acquired Packagecloud as part of an effort to tightly integrate package management tools with its continuous integration/continuous delivery (CI/CD) platform.

Buildkite CEO Keith Pitt said the goal is to reduce the level of friction that exists between providers of third-party package management tools and CI/CD platforms that DevOps teams are today required to address themselves.

There is no shortage of package management tools used to configure, install and update software, but the level of integration between these tools and CI/CD platforms has not historically been very high. The acquisition of Packagecloud will result in a package manager becoming a first-class citizen within the context of a CI/CD platform, said Pitt.

Buildkite will maintain the existing support Packagecloud provides for platforms such as Jenkins, but the acquisition will streamline DevOps workflows and improve overall developer productivity, he added.

In the months ahead, Buildkite will also add a range of tools to better enable DevOps teams to achieve that goal, said Pitt. In addition, the company plans to add tools to streamline the management of compliance processes as regulations pertaining to application development are becoming more stringent, he added.

It’s not clear how much IT organizations are reevaluating their CI/CD platforms to improve developer productivity, but platform engineering as a methodology to centralize the management of DevOps workflows is gaining traction. Those efforts promise to reduce the total cost of implementing DevOps at scale and improve developer productivity by making the scaffolding required to build applications more readily accessible. Today, too many developers are spending time managing developer environments rather than writing actual code.

The challenge, of course, is convincing developers—who tend to jealously guard their tool prerogatives—to embrace a platform engineering methodology and cede control over the development environment to a centralized team.

One way or another, developer productivity needs to improve as more organizations encounter economic headwinds. Developers are the most expensive resource any organization has, so improving the rate at which high-quality code is created will always be imperative, noted Pitt.

Today, organizations depend on software to drive revenue. Investors are assessing senior leaders within those organizations on their ability to deliver quality software in a timely fashion. Each new application and subsequent software update now has a direct impact on the profitability of the organization. As such, tolerance for legacy approaches to building software that require more time and effort is dropping.

The issue, as always, is that switching from one software development platform to another is an expensive proposition, so many organizations may find themselves supporting a mix of legacy and modern platforms. The challenge becomes defining the pace at which that migration occurs as new applications are created and legacy applications are continuously updated.

September 19, 2023
JFrog Extends DevOps Reach to Manage and Secure AI Models

At its swampUP 2023 conference, JFrog today added additional security capabilities to its continuous integration/continuous development (CI/CD) environment as well as making it possible to manage machine learning models within the context of a DevOps workflow.

The latest edition of the JFrog Software Supply Chain Platform adds an integrated static application security testing tool (SAST) to scan source code alongside the existing JFrog Xray tool for scanning binaries.

JFrog has also added an Open-Source Software (OSS) Catalog to the JFrog Curation service to make it simpler for DevOps teams to discover specific packages that have been determined to be secure. JFrog is also adding Release Lifecycle Management (RLM) capabilities that enable aggregating software artifacts into an immutable software package that serves as the single source of truth as multiple iterations of an application are developed. JFrog RLM uses anti-tampering systems, compliance checks and evidence capture to collect data and insights to ensure the immutability of signed binaries.

Finally, ML Model Management capabilities, available in beta, are being added to the platform to make it possible to store AI models, run AI models from Hugging Face, a provider of open source AI models, as a proxy in cache via an application programming interface (API) and apply DevSecOps best practices to govern, secure and ensure licensing compliance.

JFrog CTO Yoav Landman said that as DevOps teams become more familiar with AI models, it’s becoming increasingly clear they are managed like any other software artifact incorporated into a DevOps pipeline. He noted that the approach provides the added benefit of helping to ensure AI models have not been poisoned by malicious data and prompts that would deliberately cause a hallucination.

Most AI models today are created by data scientists using machine learning operations (MLOps) best practices that each organization defines. In some instances, those processes include what is known as a feature store that enables data scientists to track which components are being used at any given time. However, once a model is completed, it needs to be integrated into an application—a process that, from a DevOps perspective, is best facilitated by storing an AI model in a Git repository alongside other components that need to be managed.

It’s still early days as far as the convergence of DevOps and MLOps practices are concerned. Going forward, as it becomes apparent almost every application will be invoke some type of AI model, the need to integrate these workflows is becoming a more pressing concern.

It’s now only a matter of time before AI models become a pervasive element of the software development life cycle as they are continuously retrained and updated. It will generally fall to DevOps teams to manage the deployment and updating of those AI models in collaboration with the data science teams that train them. The immediate challenge, of course, will be to bridge the significant cultural divide between two very different types of IT professionals.

September 13, 2023