devopsy.kinsta.cloud

Author: Mike Vizard

Foundation Proposes Advancing eBPF Adoption Across Multiple OSes

The eBPF Foundation, which proposes to advance adoption of an approach that enables sandbox programs to run faster at the kernel level, is being launched today as an arm of the Linux Foundation.

The technology known as eBPF was originally developed for Linux. The eBPF Foundation is now committed to expanding use of eBPF across all operating systems. Members of the eBPF Foundation include Facebook, Google, Isovalent, Microsoft and Netflix.

Thomas Graf, CTO and co-founder of Isovalent, a provider of networking and security tools that leverage eBPF, said IT organizations should expect to see the technology become part of Windows, along with other operating systems, as the core technology is tested and validated across a much wider range of platforms.

In effect, eBPF changes the way operating systems are designed. It bridges the boundary between kernel and user space by enabling developers to combine and apply logic across multiple subsystems that, historically, were completely independent of one another. That approach enables, for example, a security tool to scale to the point where it can identify threats at much higher levels of throughput to improve overall scale at a time when the volume of cybersecurity attacks being simultaneously launched continues to increase.

Currently, eBPF is widely employed by web-scale companies such as cloud service providers. Facebook is using it as the primary software-defined load balancer in its data centers while Google is using open source Cilium networking software within its managed Kubernetes offerings.

However, adoption within on-premises IT environments that have deployed Linux has been more limited simply because the number of vendors that have optimized networking, security and storage offerings to take advantage of eBPF is still fairly limited.

Sysdig, for example, recently launched made available an instance of the open source Falco container security platform available that employs eBPF. Tigera has also made available an instance of its container networking platform that leverages eBPF at the kernel level of Linux.

Ultimately, Graf notes the biggest benefit of eBPF is efficiency. The total cost of processing for security, networking and storage platforms should decline as more vendors take advantage of a capability that will one day be widely employed across multiple operating systems.

In the meantime, IT organizations would be well-advised to ask their vendors when they plan to support eBPF. Cloud service providers generally require it to deliver their own managed services more efficiently, so any vendor that hopes to sell a platform to those providers will need to support eBPF to drive performance at scale. The issue then becomes determining which platforms that have embraced the technology are also available in on-premises IT environments looking for similar benefits.

Regardless of approach, it’s clear there will soon be a reason to upgrade not just networking, storage and security platforms but also instances of operating systems that don’t yet support eBPF. DevOps teams would be well-advised to plan accordingly because the number of platforms that will ultimately benefit from eBPF spans the entire enterprise.

August 12, 2021
Google Unveils Tool to Better Secure GitHub Repos

Google today launched a GitHub app that provides automated continuous enforcement of security best practices for GitHub projects.

Kim Lewandowski, a product manager for open source software security at Google, said the Allstar application enables IT teams to assess any project on GitHub to check for security policy adherence. In addition, Allstar sets desired enforcement actions and automatically applies those rules when triggered by a setting or file change in a repository.

The goal is to provide the open source community with a tool that makes it possible for organizations to have more confidence in the open source software that is being employed within a software supply chain, said Lewandowski.

Allstar is intended as a companion application for Security Scorecards, a tool that Google made available last year to assess whether, for example, an open source project employs branch protection to ensure that malware isn’t inadvertently committed to a project. Allstar continuously checks expected GitHub API states and repository file contents against security policies and the enforcement actions defined by an IT organization. After it detects a policy violation, the tool can be configured to simply send an alert or to automatically enforce a specific policy to remediate the issue, said Lewandoski. Specifically, she said the options available are to log the security policy violation without taking any additional action, open a GitHub issue or modify the GitHub setting to match the original Allstar configuration.

At present, Allstar provides a limited number of security policy checks, but more will be made available over the coming months, said Lewandowski. Current security policy checks include branch protection, vulnerability disclosures, access controls and detection of binary artifacts that can’t be scanned. Additional checks that will be added include automatic dependency updates and frozen dependencies.

Lewandowksi said Allstar and Security Scorecards are part of an ongoing Google effort to give back to the open source community that is being targeted by cybercriminals attempting to compromise software supply chains. Just about every application is now dependent on open source components, to some degree. The maintainers of those projects, however, don’t always have the tools or expertise required to check for malware that has been injected into a codebase, she noted.

In general, Lewandoski noted that achieving DevSecOps best practices will require a lot more automation of application security. It’s not possible for each developer to become a world-class cybersecurity expert. The challenge is finding a way to introduce that automation into the application development process as early as possible, which, she noted, in many cases starts with open source components that are incorporated in applications.

It may take a while for every maintainer of an open source project to revamp the processes through which code is added to their project. However, the easier it becomes to continuously review that software, the more confidence organizations that use it will have in its security. In fact, the maintainers of those projects should expect the downstream users of that software to be asking some pointed questions about how security is being managed in the wake of a series of recent high-profile breaches of software supply chains.

August 11, 2021
Datadog Cloud Security Platform Advances DevSecOps

Datadog has launched a Datadog Cloud Security Platform that promises to advance DevSecOps by identifying potential threats using the same agent software many DevOps teams already rely on to instrument applications.

Renaud Boutet, senior vice president of product for Datadog, said organizations that rely on the Datadog software-as-a-service (SaaS) platform can now more easily invoke security services based on the same data model used to observe applications.

The security services provided by Datadog include a cloud security posture management (CSPM) offering that identifies misconfigurations and applications that are not compliant with industry standards such as the Payment Card Industry Data Security Standard (PCI DSS).

A cloud workload security (CWS) offering, meanwhile, detects threats to production workloads by monitoring file and process activity, while a monitoring tool identifies threats to cloud environments by analyzing operational and security logs. It also provides integrations and threat detection rules that are easy to extend and customize.

Finally, there is an application security tool, currently in beta, that identifies and blocks attacks that target code-level vulnerabilities, such as SQL injections and cross-site scripting (XSS) exploits.

Boutet noted that many DevOps teams are reluctant to roll out security platforms that require them to deploy, manage and update an additional set of agents across their application environment. Datadog is now effectively unifying application observability and security in a way that makes it simpler for those DevOps teams to embrace DevSecOps best practices, added Boutet.

In some cases, the dashboards surfaced by Datadog will be employed by DevOps teams that have assumed responsibility for cybersecurity as part of an effort to shift more responsibility for application security further left, noted Boutet. In other instances, cybersecurity teams will be able to view the same alerts shared with their DevOps colleagues, he added.

Pressure to address DevSecOps is mounting in the wake of a spate of high-profile software supply chain breaches. While the majority of organizations have not been breached, questions are now being asked at the boardroom level as concerns over application security increase. Many organizations are wondering if they should slow down the pace of application development to ensure better security and to what degree.

Regardless of the path forward, awareness of application security has never been higher. An executive order issued by President Biden calling for improvements to security will likely have a cascading effect across not just government agencies but also on organizations that do business with those agencies. One way or another, DevOps teams and cybersecurity professionals will be required to work more closely together. The challenge is striking the right balance between security and productivity. Most developers are never going to become world-class security experts. Conversely, most security professionals know next to nothing about coding. That doesn’t mean, however, that efforts shouldn’t be made to educate both camps as much as possible. After all, even a small amount of education can have a profound impact when most of the attacks being made against applications are not especially sophisticated.

August 10, 2021
Sophos Acquires Refactr to Advance DevSecOps

Sophos this week revealed it has acquired Refactr, a provider of an automation platform that makes it simpler to add static and dynamic security scanning and application testing to a DevOps pipeline. Terms of the deal were not disclosed.

Joe Levy, Sophos CTO, said Sophos will extend the Refactr DevSecOps automation platform to add security orchestration automation and response (SOAR) capabilities. This will provide DevOps teams with access to its managed threat response (MTR) and extended detection and response (XDR) platforms. The goal is to make it simpler for DevOps and cybersecurity teams to collaborate once those integrations are achieved in early 2022, added Levy. Sophos will also continue to develop the Refactr platform as a standalone offering in addition to maintaining the current community edition of the platform, noted Levy.

Sophos has acquired a range of cybersecurity companies since it rearchitected its core platform to create an extensible security framework known as the Adaptive Cybersecurity Ecosystem. That platform is built on a modern microservices-based architecture that simplifies integration of acquisitions. Collectively, those acquisitions enable Sophos to collect a wider range of data that can be used to train artificial intelligence (AI) models and continuously better automate DevSecOps processes, noted Levy.

Recent Sophos acquisitions include Capsule8, a provider of a platform for securing containers and BrainTrace, a provider of managed security services. Sophos also acquired Rook Security, another managed security services provider, in 2019.

In general, Sophos is making the case for a security platform that both DevOps and cybersecurity teams can navigate. In addition to integrating with continuous delivery and continuous deployment (CI/CD) workflows via application programming interfaces (APIs), the Refactr platform can also be accessed via visual tools that can be employed to create launch tests using a drag-and-drop tool.

In the wake of a spate of high-profile software supply chain breaches, there’s now more focus on implementing DevSecOps best practices. The challenge organizations face is finding a way to achieve that goal without unduly slowing down the rate at which applications are built and deployed. Achieving that goal requires finding more ways to automate, for example, scans of code during the build process.

In the meantime, however, a recent Sophos Active Adversary Playbook 2021 illustrates how organizations find themselves outgunned by their cybercriminal adversaries. The report details how 37 different attack groups used more than 400 different tools between them to launch cyberattacks. The attackers’ median dwell time before the attacks were detected was 11 days.

It’s clear that without advances in automation and AI, the odds of securing software supply chains are slim to none. The challenge organizations face is finding a way to affordably tap into AI and automation technology and expertise. Individual organizations are not really in a position to build, for example, AI models capable of automating a cybersecurity process. Most organizations will wind up using APIs to invoke advanced security services delivered via the cloud within the context of a DevSecOps workflow. The issue is determining which service provider offers the most integrated and robust capabilities at the lowest cost.

August 4, 2021
Harness Updates Cloud Cost Control Tool for DevOps Teams

Harness today announced that it has updated its cloud cost management (CCM) module for its namesake continuous integration/continuous delivery (CI/CD) platform to add support for Kubernetes at the node level in addition to providing business intelligence tools to analyze overall cloud costs.

Harish Doddala, senior director of product management for Harness, said the update surfaces more granular insights into cloud costs that DevOps teams are incurring because cloud resources are being wasted. The latest version of CCM enables DevOps teams to create their own customized views of those resources.

As the overall economy continues to recover from the COVID-19 pandemic, more organizations are looking to exercise greater control over IT costs. In the immediate aftermath of the pandemic, many DevOps teams consumed cloud resources without much regard to cost to enable organizations to advance various digital business transformation initiatives. Now, finance teams are requiring DevOps teams to account for those cloud costs.

The goal is not only to reduce costs whenever possible—by, for example, quickly identifying a sudden spike in cloud resources consumption—but to accurately forecast budget costs on a monthly basis so that both IT and finance teams can collaboratively track spending. The typical finance team is just as concerned about accurately predicting costs as they are the total amount spent.

Ultimately, Doddala says Harness’ CCM enables organizations to better assess which software development projects to prioritize based on the cloud resources allocated. It’s not uncommon for organizations to suddenly have to shift software development priorities as business conditions change and evolve. A project that may be a top priority today can easily be downgraded because, for example, a rival has been able to launch some new capability that an organization now needs to quickly match.

DevOps teams, unfortunately, don’t always pay as much attention to infrastructure costs. Harness’ CCM helps reduce those costs by automatically stopping idle resources running on Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP) or any Kubernetes cluster, said Doddala.

There are, of course, any number of existing tools that cloud service providers make available to enable IT teams to reduce costs. CCM, however, provides that capability within the context of a DevOps platform that can span multiple cloud computing environments, noted Doddala. DevOps teams, for example, can create and enforce cost policies for cloud infrastructure into a CI/CD pipeline and prevent an organization from being surprised by a bill from their cloud service provider at the end of the month.

It’s not clear the degree to which cost control modules will influence the selection of one CI/CD platform over another. However, it’s apparent the days when organizations consumed cloud resources with little regard to cost are ending. In fact, there may soon be more cost controls imposed on software development projects than there were prior to the pandemic. The challenge, of course, is finding a way to automate the implementation of those controls and minimize the impact on software development velocity.

August 3, 2021
Report: Lots of Room for Continuous Delivery Improvement

A report published by the Continuous Delivery (CD) Foundation suggests that, in terms of achieving that goal, most organizations still have a long way to go. Based on analysis of the Developer Nation survey data collected by SlashData, nearly two-thirds of developers said it takes at least one week to go from code committed to code successfully running in production. Only one in 10 developers are deploying code in a production environment multiple times per day.

On the plus side, half of developers reported that they restore service from an unplanned outage in less than a day.

The survey identified the top programming languages being employed as shell scripting languages, Go/Golang, JavaScript, PHP, Scala, Dart, C#, Ruby, Python and Java.

Surprisingly, the vertical industry furthest along in terms of achieving continuous delivery is the retail sector that has embraced digital business transformation, while the furthest behind is the telecommunications sector.

Tracy Miranda, executive director for the CD Foundation, noted that even after more than 10 years of advocacy, DevOps adoption of CD best practices remains nascent. Many organizations have adopted so-called continuous integration/continuous delivery (CI/CD) platforms, but they are mainly used to drive a set of CI best processes. CD, in comparison, has proven to be a more elusive goal to achieve mainly because each platform on which an application is deployed tends to have unique attributes. Automating the delivery of applications across multiple platforms has, as a result, been difficult to achieve and maintain, she noted.

Nevertheless, with the arrival of cloud-native container-based applications destined to be deployed on Kubernetes clusters that expose a consistent set of application programming interfaces (APIs), the prospects for more widely achieving CD are improving. Each platform that has a Kubernetes cluster running will expose the same API to a CD platform.

In the short term, however, Miranda noted that many IT professionals are still somewhat intimated by both Kubernetes and CD best practices. Too many organizations still tend to think of CD as a platform to acquire rather than a set of processes and skills to be defined and nurtured, she added.

One way or another, however, the CD challenge will need to be addressed as more organizations realize how dependent they are on software to deliver a differentiated customer experience. The rate at which that customer experience improves will determine the level of customer affinity that organizations will be able to attain and retain. That’s doesn’t mean every organization has to be as proficient as, for example, Netflix when it comes to implementing DevOps best practices, Miranda added. However, she said, it’s already apparent most organizations will need to improve how they deliver software to drive multiple digital business transformation initiatives. The opportunity lies in reviewing those processes now with an eye toward creating a culture squarely focused on continually improving the customer experience, she said.

It may still be a while before most organizations truly master CD, but, as is the case with most journeys, the most important thing is to simply take that first big step.

August 2, 2021
Cycloid Tool Can Reverse Engineer Provisioned IaC

Cycloid this week unveiled a tool that makes it possible to reverse engineer code used to manually provision cloud infrastructure. The Infra Import tool is part of an effort to create a more consistent and reliable version of that code using open source Terraform software.

Benjamin Brial, Cycloid CEO, said Infra Import is based on Cycloid’s open source TerraCognita software, which enables IT teams to reverse engineer infrastructure-as-code (IaC) created using open source Terraform tools.

Infra Import extends that capability to cloud platforms such as Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform or an IT environment employing the open source OpenStack cloud platform that may have been manually provisioned. IT teams can then create a configuration based on that Terraform code that could be employed across multiple clouds.

Infra Import is designed to connect to a cloud provider and then automatically reverse engineer Terraform files and IaC stacks based on the existing deployment. It also ensures that the latest edition of Terraform is being used to create that configuration code, noted Brial. That capability eliminates the need for DevOps teams to keep track of what version of Terraform might need to be updated as new versions of the IaC tool become available.

Just as significantly, Infra Import also makes it a lot easier to onboard new members to a DevOps teams because the tools uses valid secure configurations they know can be relied on. Misconfiguration of cloud services is one of the biggest security issues that IT teams today are wrestling with, simply because most developers lack the security expertise required to configure cloud services properly.

Cycloid is in the process of rolling out a comprehensive suite of management tools that complement a lightweight framework it created to make DevOps best practices more accessible. The challenge is many IT organizations have already manually provisioned cloud infrastructure in a way that is, at the very least, often suboptimal if not downright insecure.

Misconfigurations are rife across cloud computing environments, but most organizations are not prepared to manually address the issue. Automating the reverse engineering of those configurations is the first step toward enabling IT teams to address the problem at scale.

It’s not clear to what degree configuration concerns have impacted the number of applications that would otherwise have been deployed in the cloud. Security is always listed as the top concern IT leaders have when employing cloud platforms. The issue, however, is not the security of the platforms themselves but rather the processes employed to deploy applications on the platforms. Cloud service providers have promoted a shared responsibility model for security under which developers are accountable for both securely configuring services and ensuring the overall security of the software environment. The trouble is, not every developer understands the implications of that shared responsibility model much less how to actually secure an application workload.

It’s not likely security concerns will slow down the rate at which applications are deployed in the cloud. However, as more cloud security issues are encountered, more scrutiny will be applied.

July 30, 2021
Logz.io Allies with Microsoft on Open Source Observability

Logz.io this week announced an alliance with Microsoft that enables DevOps teams to deploy its observability platform—based on open source platforms such as ElasticSearch, Logstash, Kibana (ELK), Prometheus and Jaeger—on the Azure public cloud.

Jonah Kowall, Logz.io’s CTO, said the multi-year effort enables those teams to deploy the Logz.io platform via the Azure Console with the click of a button. The Azure platform has a data collection capability that aggregates operating system events in a way that makes it simpler for the Prometheus monitoring solution to consume, he added.

DevOps teams, however, will still have to collect data from the applications they instrument on the Azure platform, he added.

Logz.io integrates multiple open source monitoring tools based on metrics, logs and traces within a single observability platform that otherwise would require considerable time and effort from internal IT teams on their own.

Interest in observability is rising as organizations attempt to provide internal IT teams with more context when troubleshooting issues. Whenever there is an IT issue, most teams convene a “war room” where they attempt to discover the root cause of an issue by comparing analytics generated by a wide range of monitoring tools. Observability platforms promise to enable a DevOps team to replace many of those tools in a way that provides more context by integrating log data, metrics and distributed traces within a single platform.

In many instances, existing application performance management (APM) platforms are being transformed into observability platforms by adding tools to monitor IT infrastructure. Conversely, platforms for managing IT infrastructure are adding support for tools to monitor applications. Elsewhere, a range of startups have emerged to offer observability platforms they have developed from the ground up.

Regardless of the platform employed, the rise of microservices-based applications within many organizations is forcing the observability issue. The dependencies that exist within microservices-based applications are too complex to manage without a platform that provides some level of increased observability. Of course, most existing monolithic applications are not being retired any time soon, so the need for an observability framework for both legacy and modern cloud-native applications has only become more pressing.

The Logz.io platform can, of course, be deployed on other cloud platforms as well as within on-premises IT environments. As hybrid cloud computing continues to evolve, it is apparent that observability platforms will need to extend their reach from the cloud to the network edge. Over time, artificial intelligence (AI) capabilities will be added to make it easier to, for example, streamline the number of alerts that are generated across complex IT environments.

In the meantime, the most important thing with regard to observability may be simply getting started. Most IT teams start to analyze logs before moving on to metrics and then traces using the open source Jaeger agent software. After all, the only thing more difficult to instrument than an application before it is deployed is, arguably, one that is already running in a production environment.

July 29, 2021
WhiteSource Tool Automatically Fixes Code Vulnerabilities

WhiteSource today announced that it has developed the first-ever tool that automatically remediates vulnerabilities discovered in custom code.

Rami Sass, WhiteSource CEO, said WhiteSource Cure surfaces recommendations for fixing security vulnerabilities in code that developers can then apply with a click of a button. WhiteSource has a long history of providing tools that discover vulnerabilities in open source software that it tracks via a database it manages, but Sass said customers are making it clear they need a way to automatically remediate those issues in a way that doesn’t adversely impact developer productivity.

The average developer spends, on average, half a day fixing a single vulnerability, so the impact security issues are having on the rate applications are being developed is significant, added Sass. WhiteSource Cure gives each developer the equivalent of their own personal security expert, noted Sass.

Developers can either apply the remediation suggested by WhiteSource Cure—that manifests itself as a pull request in a DevOps pipeline—or they can fine-tune it as they see fit, said Sass.

WhiteSource Cure arrived as friction and tensions between developers and cybersecurity teams rise in the wake of a series of high-profile attacks against software supply chains. As a result, organizations are instituting more thorough security reviews of application development projects. The concern is those reviews will dramatically reduce the rate at which applications are being delivered.

Of course, many organizations are looking to shift responsibility for application security further left toward developers to minimize the number of vulnerabilities that might need to be remediated either just before an application is deployed in an on-premises IT environment or, worse yet, after it is deployed. Trying to achieve that goal within the context of DevSecOps best practices, however, is challenging. Organizations first have to acquire vulnerability scanning tools that developers will actually use, and then provide developers with the training required to identify various classes of vulnerabilities.

The other big challenge is the time it takes to train developers to recognize vulnerabilities. WhiteSource Cure eliminates the need for developers to become security experts, said Sass. In addition to a commercial offering, WhiteSource is also making available a Community Edition of WhiteSource Cure for open source projects that will forever be available for free to developers working on those projects.

Sass said that rather than relying on machine learning algorithms and other forms of artificial intelligence (AI) to build WhiteSource Cure, the team that developed the tool found that more traditional approaches to software composition analysis (SCA) provided more reliable results.

It’s unclear whether the ability to automatically remediate known vulnerabilities in code will render much of the DevSecOps debate moot. However, it’s clear that the simplest path forward toward achieving DevSecOps is, as always, to automate as much of the remediation process as possible. The challenge will be waiting to see how much confidence developers and cybersecurity teams will have in the recommendations generated.

July 28, 2021
CircleCI Survey Shows the Need to Prioritize DevOps

A CircleCI survey of 2,013 business leaders in the U.S. and UK published today finds nearly all respondents (97%) recognize that the success of their business within the next year depends on their software development teams. Despite those findings, well over one-third of respondents (39%) admit their organization is challenged when it comes to innovation and acquiring and maintaining developer skills.

A full 89% of respondents said they also have a good understanding of how to measure the performance of their engineering teams. However, 40% of companies are still using antiquated metrics such as the number of lines of code written, while another 37% measure story points.

More troubling still, less than one-third of respondents (30%) said they plan to make adoption of DevOps best practices a priority. On the plus side, only 15% of respondents said they are implementing a continuous integration/continuous delivery (CI/CD) platform for the first time.

The CircleCI survey suggests that improvements in how software is delivered could be worth, on average, up to $126 million per year. Nearly two-thirds of respondents (65%) said their business is involved in digital business transformation initiatives of one kind or another. Another 20% said they have completed one.

Jim Rose, CircleCI CEO, said it appears that while more organizations now realize how dependent they are on software, many of them have yet to address issues that fundamentally impact developer productivity. For example, more than half (52%) of respondents don’t allow their software developers to choose their own tools, noted Rose. While there is a clear need for a consistent approach to building and deploying applications, Rose noted that DevOps best practices should not require developers to employ a specific set of tools. Developers are only able to truly innovate when they are able to experiment with different tools, said Rose.

In general, Rose said DevOps adoption tends to go through various cycles. Many organizations today are looking to reduce the number of tools that make up a toolchain. However, those efforts shouldn’t curtail the ability of developers to innovate. A CI/CD platform that provides the foundation for DevOps workflows needs to be easily extensible, noted Rose.

It’s not clear to what degree adoption of DevOps best practices have stagnated, but the number of organizations that have implemented DevOps across their entire enterprise remains limited. Most organizations have individual teams that might have adopted DevOps to one degree or another. In many cases, those teams are using disparate tools and platforms.

Most organizations clearly want to be able to build and deploy applications faster. The challenge isn’t so much mastering DevOps platforms as overcoming many of the cultural issues that conspire to limit DevOps adoption. In time, however, the need to rely more on digital processes will force the DevOps issue. As one organization makes advances—thanks in part to their ability to effectively employ DevOps processes—the more keen rival organizations will be to make sure they don’t lose ground. It may be a while before business executives connect the dots between their ability to compete and the degree to which organizations have mastered DevOps, but it’s increasingly apparent that day is not too far off.

July 28, 2021