devopsy.kinsta.cloud

Category: Features

  • Dynatrace Applies AI to Surface App Vulnerabilities

    Dynatrace Applies AI to Surface App Vulnerabilities

    Dynatrace has enhanced the security module to its observability platform that leverages its Davis artificial intelligence (AI) engine to automatically identify the software libraries and open source packages that represent the greatest security risk.

    Ajay Gandhi, vice president of product marketing for Dynatrace, said the Davis Security Advisor, being added to the Dynatrace Application Security Module, makes it easier for IT teams to understand which vulnerabilities need to be remediated first.

    Davis Security Advisor aggregates vulnerability data in real-time and prioritizes remediation based on the number of vulnerabilities, severity, impact to sensitive data, public exposure, how widely known the vulnerability is, and whether the code impacted is facing the internet. That approach enables Davis Security Advisor to monitor all software libraries used in preproduction and production environments in a way that sharply reduces the number of false positives that would otherwise be generated, noted Gandhi.

    Dynatrace

    As application environments become more complex—in part due to increased adoption of microservices that have lots of dependencies—it’s become impossible for IT teams to implement DevSecOps best practices in workflows without some form of AI augmentation, added Gandhi.

    In fact, a recent Dynatrace survey found 89% of chief information security officers (CISOs) noted cloud-native architectures and container runtime environments, used primarily to construct microservices, have made it more difficult to detect and manage software vulnerabilities.

    In theory, application developers are taking more responsibility for remediating security vulnerabilities within their applications. The challenge is they often inadvertently use an older version of a library that has known vulnerabilities. Hopefully, that issue gets addressed before an application gets deployed. There are, however, plenty of cases where, for one reason or another, a library with known vulnerabilities gets deployed anyway.

    Making matters more challenging, many vulnerabilities are discovered after an application has been deployed. Developers don’t always have access to a software bill of materials that would make it easier to determine what libraries have been employed in what application or, for that matter, which libraries are dependent upon one another.

    Gandhi said the Davis Security Advisor resolves that issue via continuous topology mapping of dependencies to identify those potential security issues. Once identified, the updates that need to be made can then be slipstreamed into a DevOps workflow.

    Analyzing code for vulnerabilities is, of course, not a new idea. Training an AI model to perform that task on behalf of DevOps teams does promise to greatly accelerate the process at a time when organizations are under more pressure than ever to secure their software supply chains in the wake of a series of high-profile breaches.

    It’s just not clear yet who inside organizations might drive adoption for AI platforms capable of identifying vulnerabilities in a way that helps prioritize remediation efforts. In some cases, cybersecurity teams have a vested interest in reducing the number of potential security incidents.

    The one thing that is apparent is shipping insecure code faster only leads to more negative outcomes. There may not be such a thing as a ‘great’ application that happens to be insecure. Instead, the assumption will be that any applications with known vulnerabilities, no matter how great, are too flawed to be deployed given all the security risks most businesses currently face.

  • What Chaos Engineering Is (and Isn’t)

    What Chaos Engineering Is (and Isn’t)

    The birth of chaos engineering happened somewhat accidentally in 2008 when Netflix moved from the data center to the cloud. The move didn’t go as planned.

    The thinking at the time was that the data center locked Netflix into an architecture of single points of failure, like large databases and vertically scaled components. Moving to the cloud would necessitate horizontally scaled components, which would decrease the single points of failure. But the move to horizontally scaled cloud deployment practices did not produce the boost to uptime of the streaming service that they expected.

    The specific methods for making a system robust enough to handle instances disappearing was not important. It might even be different depending on the context of the system. The important thing was that it had to be done, because the streaming service was facing availability deficits due to the high frequency of instance instability events. In a way, Netflix had simply multiplied the single-point-of-failure effect.

    Enter Chaos Monkey. Chaos Monkey gave the company a way to proactively test everyone’s resilience to a failure, and do it during business hours so that people could respond to any potential fallout when they had the resources to do so, rather than at 3 a.m. when pagers typically go off.

    Then, Dec. 24, 2012, happened. Amazon Web Services (AWS) suffered a rolling outage of elastic load balancers (ELBs). These components connect requests and route traffic to the compute instances where services are deployed. As the ELBs went down, additional requests couldn’t be served. Since Netflix’s control plane ran on AWS, customers were not able to choose videos and start streaming them.

    To ensure that all of these teams had their services up to the task, an activity was created to take a region offline. Well, AWS wouldn’t allow Netflix to take a region offline (something about having other customers in the region) so, instead, Netflix simulated the activity; labeled “Chaos Kong.”

    Fast-forward to about 2015. Netflix had Chaos Kong working on large-scale vanishing regions and had introduced Chaos Monkey, which worked on small-scale vanishing instances.

    In these early days of chaos engineering at Netflix, it was not obvious what the discipline actually was. There were some catch phrases tossed around about “pulling out wires,” or “breaking things,” or “testing in production,” paired with many misconceptions about how to make services reliable, and very few examples of actual tools to help support that work.

    To formalize the discipline, I was given the task of developing a charter and roadmap for a chaos engineering team at Netflix in early 2015. I built and managed that team for three years. My co-author of “Chaos Engineering: System Resiliency in Practice,” Nora Jones, joined the chaos engineering team early on as an engineer and technical leader. She was responsible for significant architectural decisions about the tools we built as well as overseeing implementation.

    To start, I sat down with my teams to formally define chaos engineering. We specifically wanted clarity on:

    • What is the definition of Chaos Engineering?
    • What is the point of it?
    • How do I know when I’m doing it?
    • How can I improve my practice of it?

    We researched resilience engineering and other disciplines in order to come up with a definition and a blueprint for how others could also participate in chaos engineering. After about a month of working on a manifesto of sorts, we produced the Principles of Chaos Engineering. The discipline was officially formalized:

    “Chaos Engineering is the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.”

    This definition established that it is a form of experimentation, which sits apart from testing. It also mentions “turbulent conditions in production” to highlight that this isn’t about creating chaos. Chaos engineering is about making the chaos inherent in the system visible. The point of doing chaos engineering is to build confidence.

    The principles go on to describe a basic template for experimentation, which borrows heavily from Karl Popper’s principle of falsifiability. In this regard, chaos engineering is modeled very much as a science rather than a technique.

    What Chaos Engineering Is

    The principles define the discipline so that we know when we are doing chaos engineering, how to do it and how to do it well. The more common definition for chaos engineering today is, “The facilitation of experiments to uncover systemic weaknesses.”

    The steps of Chaos Engineering experimentation are as follows:

    1. Start by defining “steady state” as some measurable output of a system that indicates normal behavior.
    2. Hypothesize that this steady state will continue in both the control group and the experimental group.
    3. Introduce variables that reflect real-world events like servers that crash, hard drives that malfunction, network connections that are severed, etc.
    4. Try to disprove the hypothesis by looking for a difference in steady state between the control group and the experimental group.

    By design, there is great latitude in how to implement these experiments.

    Experimentation Versus Testing

    Testing, strictly speaking, does not create new knowledge. Testing requires that the engineer writing the test knows specific properties about the system that they are looking for in advance. Complex systems are opaque to that type of analysis—humans are simply not capable of understanding all of the potential side effects from all of the potential interactions of parts in a complex system. This leads us to one of the key properties of a test.

    Tests make an assertion, based on existing knowledge, and then running the test collapses the valence of that assertion, usually into either true or false. Tests are statements about known properties of the system.

    Experimentation, on the other hand, creates new knowledge. Experiments propose a hypothesis, and as long as the hypothesis is not disproven, confidence grows in that hypothesis. If it is disproven, then we learn something new. This kicks off an inquiry to figure out why our hypothesis is wrong. In a complex system, the reason why something happens is often not obvious. Experimentation either builds confidence, or it teaches us new properties about our own system. It is an exploration of the unknown.

    No amount of testing in practice can equal the insight gained from experimentation, because testing requires a human to come up with the assertions ahead of time. Experimentation formally introduces a way to discover new properties. It is entirely possible to translate newly discovered properties of a system into tests after they are discovered. It also helps to encode new assumptions about a system into new hypotheses, which creates something like a “regression experiment” that explores system changes over time. Because chaos engineering was born from complex system problems, it is essential that the discipline favors experimentation over testing.

    Verification Versus Validation

    Using definitions of verification and validation inspired by operations management and logistical planning, we can say that chaos engineering is strongly biased toward the former.

    Verification

    Verification of a complex system is a process of analyzing output at a system boundary. A homeowner can verify the quality of the water (output) coming from a sink’s faucet (system boundary) by testing it for contaminants without knowing anything about how plumbing or municipal water service (system parts) functions.

    Validation

    Validation of a complex system is a process of analyzing the parts of the system and building mental models that reflect the interaction of those parts. A homeowner can validate the quality of water by inspecting all of the pipes and infrastructure (system parts) involved in capturing, cleaning, and delivering water (mental model of functional parts) to a residential area and, eventually, to the house in question.

    Both of these practices are potentially useful, and both build confidence in the output of the system. As software engineers, we often feel a compulsion to dive into code and validate that it reflects our mental model of how it should be working. Contrary to this predilection, chaos engineering strongly prefers verification over validation. chaos engineering cares whether something works, not how.

    Note that in the plumbing metaphor we could validate all of the components that go into supplying clean drinking water, and yet still end up with contaminated water for some reason we did not expect. In a complex system, there are always unpredictable interactions. But if we verify that the water is clean at the tap, then we do not necessarily have to care about how it got that way. In most business cases, the output of the system is much more important than whether or not the implementation matches our mental model. Chaos engineering cares more about the business case and output than about the implementation or mental model of interacting parts.

    What Chaos Engineering Is Not

    There are two concepts that are often confused with chaos engineering, namely breaking stuff in production and anti-fragility.

    Breaking Stuff

    Occasionally, in blog posts or conference presentations, we hear chaos engineering described as “breaking stuff in production.” While this might sound cool, it doesn’t appeal to enterprises running at scale and other complex system operators who can most benefit from the practice.

    A better characterization would be “fixing stuff in production.” Breaking stuff is easy; the difficulty lies in mitigating blast radius, thinking critically about safety, determining if something is worth fixing, deciding whether you should invest in experimenting on it—the list goes on.

    Breaking stuff could be done in countless ways, with little time invested. The larger question here is, how do we reason about things that are already broken when we don’t even know they are broken?

    “Fixing stuff in production” does a much better job of capturing the value of chaos engineering, since the point of the whole practice is to proactively improve availability and security of a complex system. Chaos engineering is the only major discipline in software that focuses solely on proactively improving safety in complex systems.

    Anti-Fragility

    People familiar with the concept of anti-fragility, introduced by Nassim Taleb, often assume that chaos engineering is essentially the software version of the same thing. Taleb argues that words like “hormesis” are insufficient to capture the ability of complex systems to adapt, and so he invented the word “anti-fragile” as a way to refer to systems that get stronger when exposed to random stress.

    An important, critical distinction between chaos engineering and anti-fragility is that chaos engineering educates human operators about the chaos already inherent in the system so that they can be a more resilient team. Anti-fragility, by contrast, adds chaos to a system in hopes that it will grow stronger in response rather than succumbing to it.

    As a framework, anti-fragility puts forth guidance at odds with the scholarship of resilience engineering, human factors and safety systems research. For example, anti-fragility proposes that the first step in improving a system’s robustness is to hunt for weaknesses and remove them. This proposal seems intuitive, but resilience engineering tells us that hunting for what goes right in safety is much more informative than investigating what goes wrong.

    The next step in anti-fragility is to add redundancy. This also seems intuitive, but adding redundancy can cause failure just as easily as it can mitigate against it, and the literature in resilience engineering is rife with examples where redundancy actually contributes to safety failures. Perhaps the most famous example is the 1986 Challenger disaster. The redundancy of O-rings was one of three reasons that NASA approved the continuation of launches, even though damage to the primary O-ring was well known internally for over fifty of the prior launch missions over the span of five years.

    There are numerous other examples of divergence between these two schools of thought. Resilience engineering is an ongoing area of research with decades of support, whereas anti-fragility is a theory that exists largely outside of academia and peer review. It is easy to imagine how the two concepts become conflated, since both deal with chaos and complex systems, but the spirit of anti-fragility does not share the empiricism and fundamental grounding of chaos engineering. For these reasons, we should consider them to be different pursuits.

    A combination of real-world experience applying the original four principled steps to experimenting on systems at scale, paired with additional thoughtful introspection, led the chaos engineering team at Netflix to push the practice further than experimentation. These insights became the “Advanced Principles,” which guide teams through the maturity of their chaos engineering programs and help set a gold standard toward which we can aspire—we’ll cover these in a follow-up post.

  • Continuous Testing Practices (Part 1)

    Continuous Testing Practices (Part 1)

    In my prior blog, Continuous Testing – The Quest for Quality at Speed, I described five tenets and some of the practices for continuous testing (CT) to help those understand what continuous testing is. In my consulting work, I find it necessary to use 15 categories of practices to assess an organizations’ continuous testing capabilities. Given the large number of practices, I am publishing the complete list in three parts.

    This article is part one, which covers the following categories of continuous testing practices:

    • Continuous testing tenets
    • Leadership
    • Culture
    • Test strategy and plans
    • Test management

    Continuous Testing Tenets

    • The continuous testing principle “shift left” is practiced, in which tests are conducted as early in the pipeline as possible.
    • The continuous testing principle “fail early” is practiced, in which tests are arranged so that the most likely problems are found in the early stages of the pipeline.
    • The continuous testing principle “fail often” is practiced, in which tests are run frequently and with many different conditions such as different variations of operating systems, production topologies, browsers and production versions of services.
    • The continuous testing principle “test fast” is practiced, in which tests are run in quick cycles to get to a verdict as quickly as possible.
    • The continuous testing principle “be relevant” is practiced in which tests are focused on the most important tests and results for each purpose and stakeholder.

    Leadership

    • Leaders establish a definition and vision for software quality.
    • Leaders establish a definition for test coverage.
    • Leaders clarify how everyone’s roles fit with the CT strategy.
    • Leaders establish incentives for CT performance.
    • Leaders proactively sponsor quality assurance testing as a strategic element of their budgets and operations rather than viewing testing as a cost to avoid. This includes budgeting time and money for DevOps test training, test resources, test frameworks, test tools, test management and establishing policies for assessments.
    • Operational and capital expense budgets are planned together with all stakeholders to ensure all stakeholders agree there is enough opex and capex budget for the organization to accomplish quality goals for the budget period being planned.
    • Cost accounting practices ensure the capex and opex expenditures are tracked and made visible to stakeholders.
    • Schedules for CT improvement projects are identified and tracked.
    • Skills development programs provide individuals and teams opportunities to learn CT practices.
    • The performance of continuous quality assurance programs are monitored and made visible to stakeholders.
    • Leaders participate in creating and maintaining guidelines for codifying and tagging business risk priorities.

    Culture

    • Quality is everyone’s concern. Responsibility for attaining high quality software products is shared by everyone on a cross-functional team.
    • Cross-functional team members collaborate for testing. This reduces confrontation and friction between people who find failures and those who fix the code.
    • Dev teams embrace the creation of good tests and analysis of test results.
    • Ops team members contribute to test planning, test creation, test execution and test results analysis.
    • Policies for test creation and test coverage are agreed upon by the cross-functional team including Dev, QA, security and Ops team members.
    • A test architect is a role within the organization. The role has expertise in CT practices and helps cross-functional teams understand and apply them.

    Test Strategy and Plans

    • Test strategies and plans are created for each product or service and reviewed by cross-functional teams including development, QA, Ops and security teams.
    • Test strategies and plans identify the type of tests, and the extent of testing activities, for each stage in the value stream.
    • Test requirements and tasks are included in agile planning procedures for each theme, epic and user story.
    • Test strategies and plans define a test coverage standard, test case standards, and input/output criterion for each pipeline gate.
    • Test plans identify the test cases and all the supporting requirements to implement and execute the test cases as required satisfying the test strategy and test management goals.
    • Test strategies and plans identify the approach for system and acceptance testing.
    • Test strategies and plans identify any tests that are to be conducted in production.
    • Test strategies and plans identify any tests that are to be conducted for security testing.
    • Test strategies and plans identify a target for test automation and manual testing for each type of testing. For example, 100% of all tests in the integration stage and 85% of test in the pre-production stage is a typical goal.
    • The test strategy requires that changes to a database or changes to an application that uses a database have tests defined for the database and the application use of the database.

    Test Management

    • All aspects of test management, such as test planning, test guidelines, test code version management and version management of test results are handled by tools that are well-integrated into the continuous delivery pipeline and any test framework being used for test automation.
    • Schedules for all test tasks are identified and tracked.
    • New features have a scheduled task to ensure test goals are met for each new feature.
    • Each test case has associated attributes that can be used to find tests according to the attributes. For example, an attribute could be type of test and the feature purpose of the test.
    • Test environments and resources needed to satisfy requirements to run each test are identified and available programmatically for scheduling purposes.

    What This Means

    Continuous testing practices are critical to success with DevOps. Many organizations underestimate both the importance and the practices needed to engineer continuous testing. This article is the first in a three-part series that provides a comprehensive list of practices that can be used to understand and assess an organization’s continuous testing capabilities. To learn more about my blueprint for continuous testing, refer to my book Engineering DevOps.

  • Hasura Extends Scope of GraphQL API Platform

    Hasura Extends Scope of GraphQL API Platform

    During its online an HasuraCon ’21 conference, Hasura today announced that it has made it possible for developers to employ GraphQL application programming interface (APIs) to launch queries against data residing in multiple databases.

    At the same time, Hasura announced a preview of schema sharing capabilities along with support for workflows based on GitOps processes that it will make generally available later this year. The Hasura Cloud platform can now deploy local migrations from staging to production with a simple git push in addition to accessing previews of applications on every pull request to the git repository.

    The schema sharing capability added to the Hasura GraphQL Engine is intended to bridge the gap between beginner and power users by allowing them to share and install samples of permissions, relationships and metadata. In addition to improving collaboration, the goal is to make it easier to onboard software engineers as projects evolve and become more complex.

    Originally developed by Facebook, GraphQL has emerged as a popular alternative to REST APIs because it enables developers to retrieve data at a more granular level, which results in improved performance.

    Hasura is a provider of an instance of a GraphQL engine the company has developed around a project that is now hosted by the GraphQL Foundation, an arm of the Linux Foundation. The GraphQL Foundation counts among its members Airbnb, Amazon Web Services (AWS), Apollo, Coursera, Elementl, Facebook, GitHub, IBM, Intuit Hasura, Paypal, Prisma, Shopify and Twitter.

    GraphQL provides a complete description of the data made available via an API and is organized in terms of types and fields rather than traditional endpoints. In addition to clarifying what data is available via that API, developers can employ types to avoid writing manual parsing code.

    Since last summer, Hasura has been making available a pilot of a GraphQL implementation that is capable of supporting remote joins across disparate databases. That capability is critical because data sets that are related to a specific business process may wind up being stored in multiple databases.

    Hasura

    Tanmai Gopal, Hasura CEO, said the open source Hasura GraphQL Engine has already been downloaded more than 250 million times this year. Downloads in 2020 reached 100 million. The challenge is that while GraphQL offers a superior alternative to REST APIs, most of the API tooling that exists within organizations is still designed around REST APIs, noted Gopal.

    To accommodate that reality, Hasura has made a tool available that will convert a query designed using GraphQL into a set of calls that are compatible with REST APIs. Developers still gain the benefits of being able to launch more granular queries, noted Gopal.

    REST APIs, in general, are becoming more problematic as organizations embrace microservices-based applications. It often takes minutes to refresh a simple dashboard because of the amount of data that REST APIs are trying to pull from various data sources. As organizations embrace digital business transformation, they are also encountering application latency issues that can be traced back to the amount of data being called from multiple microservices.

    Hasura says in addition to providing a more efficient method to access data, Hasura is also working on adding caching capabilities to its core engine to further improve performance.

    It’s not clear at what rate GraphQL might replace REST APIs, but DevOps teams should assume organizations will wind up employing a mix of both for years to come. The challenge is going to be determining when to make the transition as organizations become more dependent on APIs.

  • Salesforce Pilots Series of DevOps Initiatives

    Salesforce Pilots Series of DevOps Initiatives

    At its online TrailheaDX 2021 conference, Salesforce today unveiled a DevOps Center pilot that provides a portal through which organizations can track and manage changes to Salesforce applications along with a unified command line interface (CLI) for all its applications.

    In addition, Salesforce plans to add Salesforce Functions to enable developers to deploy code in a serverless environment.

    The company has also promised to extend its low-code App Builder tool to include support for Dynamic Interactions, a capability that enables both professional and citizen developers to add reusable components to any application. The company has also added a Customer Identity Plus tool that enables developers to consolidate and manage customer identities.

    Finally, Salesforce is adding an Einstein Automate tool that employs machine learning algorithms to both integrate data and automate workflows that can invoke, for example, an existing MuleSoft Composer tool for integrating applications using a Flow Orchestrator that will be made available as a pilot project this summer. MuleSoft is a subsidiary of Salesforce. There is also now an Einstein Data Detect tool that ensures organizations adhere to data privacy laws by employing machine learning algorithms to discover when developers might have inadvertently exposed sensitive data.

    Dynamic Interactions, Salesforce Functions and CLI unification are expected to be generally available in the winter of next year, while the Einstein capabilities are expected to arrive this summer.

    Ryan Ellis, senior vice president of product management for platform at Salesforce, said the company is making a concerted effort to make it easier for both professional and citizen developers to build applications within the same environment by fostering collaboration via a DevOps Center portal that is expected to be generally available in the spring of 2022. Today the skills that any citizen developer can bring to bear are generally limited, largely because they don’t typically know how to build a well-designed, secure application that will truly scale in an enterprise IT environment.

    Salesforce

    However, as more artificial intelligence (AI) capabilities are added to low-code tools the amount of developer guidance being provided will increase over time. That additional guidance will enable citizen developers to build robust applications more reliably, noted Ellis.

    In the meantime, the primary beneficiaries of low-code tools continue to be professional developers that employ them to build applications faster than they could otherwise using procedural code.

    In general, Ellis said the relative DevOps maturity of organizations building and deploying custom applications on top of the software-as-a-service (SaaS) platform managed by Salesforce continues to increase steadily. In the wake of the COVID-19 pandemic, many organizations opted to extend existing SaaS applications provided by Salesforce to accelerate digital business transformation initiative using low-code tools rather than developing every application from scratch using procedural code.

    Of course, as the rate at which those applications are being built continues to accelerate, the next big challenge is managing the application life cycles after they are deployed.

    It’s not clear what the ultimate mix of applications built using low-code tools versus procedural code will be. Millions of lines of code are now being regularly deployed on Salesforce platforms, noted Ellis. It may be a little while before Salesforce is able to bring DevOps best practices to all that code, but, starting today, the journey has begun.

  • Plutora Dives Deeper into Value Stream Analytics Data

    Plutora Dives Deeper into Value Stream Analytics Data

    Plutora announced today it has extended the capabilities of its value stream management (VSM) platform to include a common object data model that more efficiently normalizes data collected from multiple DevOps tools and platforms.

    Jeff Keyes, vice president of product marketing and strategy for Plutora, said the goal is to expand the level of scale and depth at which the Plutora analytics engine embedded within the Plutora Platform can analyze flow metrics.

    VSM has emerged as a distinct DevOps discipline that provides a framework for enabling organizations to optimize workflow across DevOps environments in a way that allows them to accelerate application development. The core concept traces its lineage to lean manufacturing methods that continuously measure each step of a process to improve production.

    As organizations develop a deeper appreciation for their dependency on software, the level of interest in understanding how software is constructed has increased. Organizations, for example, want to better understand the impact of missed software development deadlines on revenue, which a business might not be able to generate as quickly as anticipated.

    Plutora is making a case for a dedicated analytics platform that pulls data from multiple platforms. Alternatively, more value stream analytics capabilities are now being embedded in everything from project management software to continuous integration. Plutora contends, however, that navigating multiple value stream dashboards surfaced by different tools that make up a DevOps workflow is not as simple or effective as relying on a single platform that collects data from every tool employed.

    Metrics that track lead and cycle times for projects as well as overall throughput become much easier to surface and comprehend, noted Keyes.

    One way or another, more transparency is coming to DevOps workflows. Historically, collecting this data has been challenging because it required DevOps teams to manually collect data from each tool being used. In effect, DevOps teams were wasting time copying and pasting data from one tool into an analytics application.

    As it becomes simpler to aggregate all that data, the data model provided by Plutora makes it simpler for multiple analytics tools to analyze the same data. A business user, for example, may prefer to analyze data using the Tableau analytics platform that is now owned by Salesforce.

    In general, analytics is being applied more widely across software development processes because organizations need to increase productivity while simultaneously reducing costs. It will be up to each organization to decide to what degree they may want to use that data to better train DevOps teams versus simply using it to remind those teams to stay focused on a specific set of key performance indicators (KPIs).

    In the meantime, the number of requests for reports being made to DevOps teams should decline as the process of collecting metrics becomes more automated. The challenge now will be making sure everyone reading those reports has the same firm understanding of what’s actually possible to achieve using not only the resources at hand, but also how any additional DevOps expertise might positively impact that equation.

  • GitLab Updates Approach to Integrating DevOps Workflows

    GitLab Updates Approach to Integrating DevOps Workflows

    GitLab today took the wraps off a major update to its continuous integration/continuous delivery (CI/CD) platform that embeds analytics tools in workflows to identify the root cause of inefficiencies, among other additional capabilities.

    David DeSanto, senior director for product management at GitLab, said that capability is part of an ongoing effort to include value stream analytics within the core GitLab 14 platform as an alternative to acquiring and deploying a separate platform. There are also additional charts and dashboards that enable DevOps teams to uncover bottlenecks over time.

    In addition, GitLab announced it is extending the security capabilities of the platform by adding integrations with open source Semgrep and Trivy vulnerability scanning tools as part of an effort to enable organizations to embrace DevSecOps best practices within workflows. GitLab has also added support for the fuzz testing tool it gained via the acquisitions of Fuzzit and Peach Tech to enable DevOps teams to test application security more thoroughly along with making available additional dashboards to track security metrics.

    As part of an effort to enable organizations to better secure their software supply chains, GitLab has also added support for immutable pipeline definitions that can’t be altered without specific levels of permissions being granted. That capability also enables DevOps teams to more easily achieve compliance with any number of regulatory frameworks, noted DeSanto.

    The company is also adding a pipeline editor that provides access to a visual tool for creating, managing and visualizing pipelines. That tool is designed to both make DevOps more accessible to novices and provide experienced DevOps teams with an alternative tool for managing pipelines in a way that doesn’t require them to write additional code.

    Finally, as part of its comprehensive approach to DevOps, GitLab is also making available agent software for Kubernetes clusters to eliminate the need for organizations to acquire a separate DevOps platform to build and deploy cloud-native applications based on microservices constructed using containers. The company now provides IT organizations with the option of crafting a GitOps workflow for Kubernetes using either an agentless or agent-based approach.

    Gitlab is betting that, as DevOps continues to evolve and mature, more organizations will prefer to replace the various tools they need to integrate themselves with a platform that lowers the total cost of DevOps. Gartner is forecasting that, by 2023, 40% of organizations will have switched from multiple point solutions to a DevOps value stream delivery platform to streamline application delivery, versus less than 10% using such a platform in 2020. Today, only 10% of organizations are employing such a platform, according to Gartner.

    There should also be many more organizations employing DevOps workflows to build applications. Organizations of all sizes now realize how dependent they are on software to drive a wide range of digital business transformation initiatives. The rate at which that software can now be securely developed and updated, in many cases, goes to the heart of their ability to compete. The issue is finding a way to achieve that goal without constructing a platform that ultimately takes time and resources away from building secure software.

  • Report: The State of Cloud-Native Application Security

    Report: The State of Cloud-Native Application Security

    The cloud brings tremendous capabilities in terms of increased deployment fluidity and automation. Along with cloud adoption has come the use of cloud-native tools built specifically for developing applications for this domain. However, cloud-native tools carry some nuanced security concerns, such as misconfigurations, known vulnerabilities and leaked secrets. As such, 83% of organizations recognize security as very important to their cloud-native strategy, according to a recent report from Snyk.

    Snyk’s State of Cloud Native Application Security report surveyed hundreds of IT professionals on their cloud-native security concerns. Below, we’ll pick out the top takeaways from the report, identify the most common trouble areas for cloud-native application security and see if there’s a correlation between deployment automation and higher degrees of application security.

    The Move to Cloud-Native

    Teams are moving to cloud-native tech, empowering DevOps with software-driven architecture and infrastructure-as-code (IaC). In this new cloud-native paradigm, 58% of production workloads are deployed as containers, 21% now are serverless and 50% use some form of IaC within the deployment process.

    Organizations are moving to cloud-native technology for a variety of reasons. For one, containerized software affords increased speed — 68% of organizations moved to containers for increased deployment velocity, the study found. This is followed by ease of management (at 67%) and reduced cost (43%). There is also the security imperative for adopting cloud-native tools; 36% of respondents cited security as a primary reason for moving production applications to containers.

    Top Cloud-Native Security Concerns

    While cloud-native technologies such as containers, Kubernetes, serverless and IaC enable more rapid development, they also bring unique security issues. The report found that misconfigurations and known unpatched vulnerabilities are by far the most common type of incidents among cloud-native environments. In fact, 45% of organizations have experienced an incident arising from a misconfiguration, followed by 38% due to known unpatched vulnerabilities.

    Other common cloud-native security incidents include secrets leaks, failed audits and malware. Interestingly, the study also found that data leaks by insiders are more common within organizations with higher degrees of cloud-native adoption. According to the report, 38% of organizations with high cloud adoption have suffered data leaks by an insider, whereas the figure is cut in half, at 17%, within organizations with low cloud adoption. Secrets like API keys must be managed carefully, especially as cloud-native tools force more dependencies. “The need for effective management of these kinds of artifacts is a key differentiator from the more centralized pre-cloud era,” the report noted.

    Automating Security Testing

    End-to-end deployment automation shows promise, yet is still at an early stage of maturity for most development teams. Though 95% of organizations use some kind of deployment automation, only about one in three organizations have an entirely automated deployment pipeline.

    Organizations with increased deployment automation tend to embrace a higher degree of security testing, as well. Highly automated pipelines are twice as likely to use security testing throughout their development lifecycle, found the report. Since incidents most commonly arise from misconfigurations and known vulnerabilities, automated scanning could help identify many cloud-native issues, comparing production code against databases of known exploits.

    So, when does this security testing occur? More often than not, it happens within the CI/CD pipeline. Over 60% of organizations perform security testing within the CI system. This is opposed to testing source code repositories or local IDEs and CLI tools early on in the development process. In terms of testing frequency, there is a wide range of executions. For those groups with high deployment automation, 70% test security daily or more frequently. Automated security testing seems to be functioning well, as 72% of fully automated teams find and fix critical vulnerabilities in under one week.

    The report also exposed an interesting disconnect with regard to security testing ownership. Only a small percentage of security engineers (less than 10%) feel developers are responsible for the security of the cloud-native environment and applications, whereas 36% of developers stated they are responsible for security. These figures may demonstrate the security onus shifting left toward the development side. Or it could underline changing attitudes as full-cycle development becomes more realistic. Regardless, teams will have to find clarity on roles to avoid some difficult conversations!

    Cloud-Native Security

    With the shift toward cloud-native strategies, security standards are rising to respond to new issues like misconfigurations. To address mounting application threats, increased automation could help shift security left—this and other reports clearly demonstrate a correlation between fully automated deployment pipelines and an increased security testing regimen.

    As a result of cloud-native adoption, 58% of organizations have increased their security concerns since adopting cloud-native tools. Part of this involves moving to a zero-trust approach across all infrastructure, whether internal or external facing. The report revealed that 58% of organizations are increasingly concerned with misconfiguration, 52% are increasingly concerned with insecure APIs and 43% are increasingly concerned about known unpatched vulnerabilities, while 41% are concerned about secrets leaks. Hopefully, these benchmarks will help you see how your organization stacks up in comparison to others.

    Snyk’s State of Cloud Native Application Security report questioned 600 experts in development, security and operations on cloud-native adoption and security practices. For more insights, view the study in its entirety here.

  • Linux Foundation Makes Available Software Manifest Tool

    Linux Foundation Makes Available Software Manifest Tool

    The Linux Foundation is making available a set of free tools for building software bills of material (SBOMs) based on the software data package exchange (SPDX) file format it curates.

    Backed by more than 20 organizations, SPDX is an effort to standardize the way metadata describing the contents of a software package is described.

    The SPDX specification was submitted last fall to the International Organization for Standardization (ISO) by the Joint Development Foundation, an arm of the Linux Foundation.

    The Linux Foundation, as part of its efforts, is also making available a free, online training course to provide IT teams with the foundational knowledge required to create and maintain SBOMs.

    Kate Stewart, vice president of dependable embedded systems for the Linux Foundation, said SBOMs play a critical, often overlooked role in maintaining the security of a software supply chain. One of the major challenges organizations have today is that an SBOM isn’t often created, which makes it harder to discover what software components have been included in a package. That makes it difficult for organizations to determine if they are employing a version of a software component that now has a critical vulnerability that was just recently reported, noted Stewart.

    Even when there is a SBOM, its construction can vary from one team or organization to the next. The tools provided by the Linux Foundation are intended to standardize how the fields used to describe a SBOM are defined, added Stewart.

    The SBOM tools being advanced by the Linux Foundation are, in part, a response to President Biden’s executive order pertaining to the security of software supply chains, added Stewart. The National Telecommunications and Information Administration (NTIA), as a result, is now asking for wide-ranging feedback to define the minimum requirements for a SBOM.

    It may take time before there is an agreed upon SBOM standard, but as microservices based on containers become more prevalent, it’s also apparent the issue will become more pressing. More, smaller packages of software being deployed in the enterprise without manifests will only further exacerbate an application security crisis that has deepened in the wake of recent high-profile software supply chain breaches.

    Ultimately, a SBOM should automatically be self-generated as part of any DevOps workflow. The challenge is there is still a massive amount of software being created outside of any formal DevOps workflow. On top of that, the massive amount of poorly documented legacy application software already deployed in an enterprise IT environment makes it clear how big an issue the lack of a simple SBOM really is.

    It’s not precisely clear who inside an organization should be responsible for standardizing SBOMs. The challenge is that within many organizations it’s not the kind of issue for which developers are rewarded, much less motivated, to solve. Nevertheless, everything from application security issues to the massive amount of technical debt that stems from duplication of software development projects can all be traced back to the lack of a simple manifest.

    Of course, now that many organizations are starting to appreciate just how dependent they are on software, more questions are being asked about how it’s constructed. The thing that many of the business executives asking those questions are about to discover (with some incredulity) is how much of the software the organizations depend on is still being built using methods that have more to do with a medieval craft than they do a manufacturing line.

  • Google Proposes SLSA Framework to Secure Software Supply Chains

    Google Proposes SLSA Framework to Secure Software Supply Chains

    Google is proposing organizations adopt a framework for securing the integrity of software artifacts across a software supply chain.

    Kim Lewandowski, a product manager for open source software security at Google, said the Supply Chain Levels for Software Artifacts (SLSA) is based on an internal framework, known as binary authorization for Borg, that the company has been employing now for more than eight years to secure its software. Borg is the name Google attached to the platform for building and deploying applications upon which Kubernetes is based.

    In its current form, SLSA is a set of guidelines that identifies how to attain four different levels of application security. The goal, however, is to create a framework that will automatically generate auditable metadata that can be fed into policy engines to create a SLSA certification.

    The lowest level of SLSA requires that the build process be fully scripted or automated and generates metadata that enables provenance to be created. At the highest level, SLSA requires a two-person review of all changes made during a hermetic, reproducible build process. At the highest SLSA security level, organizations should be able to assure others that their software supply chain has not been compromised, said Lewandowski.

    Google

    In the wake of a recent series of high-profile breaches of software supply chains, the level of scrutiny being applied to how software is constructed has increased significantly. In theory, at least, developers have assumed more responsibility for application security as part of an overall shift left that provided them with more programmatic control over IT environments. As a result, cybercriminals are now focused on compromising either the credentials of developers as part of an effort to insert malware into application as it is developed, or they are trying to exploit vulnerabilities that developers have inadvertently created by programmatically misconfiguring infrastructure.

    DevSecOps best practices are, of course, being adopted to teach developers how to better secure their application environments. The issue is that not enough developers are learning how to secure applications fast enough. The volume of attacks against the software supply chain continues to grow at a rate that is now much faster than organizations can thwart simply by educating developers. In some cases, organizations are once again empowering cybersecurity teams to make sure software supply chains are secure regardless of the potential impact on the rate at which applications are being developed.

    Google is trying to promote adoption of SLSA as a framework for securing both open source and proprietary application code, said Lewandowski. There’s no silver bullet for securing software, but a framework that ultimately automates as many of the security processes as possible within the context of a DevOps workflow would go a long way toward hardening the software supply chain, noted Lewandowski.

    SLSA is being put forward as a response to a cybersecurity executive order issued by President Biden that, among other things, tasks the National Institute for Standards and Technology (NIST), an arm of the U.S. Department of Commerce, to consult with federal agencies, the private sector, academia and other stakeholders in identifying standards, tools, best practices and other guidelines to enhance software supply chain security. Given the level of dependencies that exist across those software supply chains, that executive order, to varying degrees, now applies to almost any entity that builds software for an organization that, on some level, interacts with the Federal government.