Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • The Scariest Things About SCA

    The Scariest Things About SCA

    It is a time of ghouls, mischievous spirits and David S. Pumpkins. In the spirit of Halloween, here are the top five scariest limitations of software composition analysis (SCA) tools that are enough to send shivers down your spine. Read on … if you dare!

    1. SCA Scans Only Your Application Code

    SCA’s scope is frightfully narrow. When scanning for vulnerable dependencies, SCA only examines your source code even though vulnerable dependencies used to build your software can exist throughout your entire development pipeline. Vulnerable dependencies can be found in dev tools like Jenkins, dev tool plugins like Jenkins Plugins, build modules like GitHub Actions, build module dependencies like GitHub Actions libraries, and infrastructure-as-code (IaC) dependencies. If you’re only scanning application code, you should be shaking in your boots because it is not enough to protect against modern software supply chain attacks like SolarWinds or Codecov.

    The good news is that next-gen SCA, pipeline composition analysis (PCA), scans your application code and your entire pipeline for vulnerable dependencies. PCA improves upon the foundation of SCA by including a deep understanding of the tools, configurations, processes, activities, components and dependencies used at every stage of the delivery pipeline. When new dependencies like Text4Shell are disclosed, you can be sure every instance of the vulnerability is found and remediated. This way you know you don’t have any proverbial skeletons lurking in your software’s closet.

    2. SCA Can’t Tell You Where Vulnerabilities Are Deployed

    Vulnerability remediation can be ghoulish. When new high-severity exploits are disclosed, you need to remediate every instance of the vulnerability from all production environments where it could be exploited–and you need to do it fast. Identifying a vulnerability like Log4j in your code base is a start, but you’re not truly secure until you have removed every instance of the vulnerability in production systems where they pose a real risk to your organization. Though traditional SCA can tell you where a vulnerability is in your application code, it has no visibility once that code has been deployed.

    PCA gives you visibility into your entire development pipeline, including production environments. This enables you to easily trace the path of a vulnerability from code to deployment locations, such as identifying which Kubernetes clusters contain a specific vulnerability. This enables organizations to be more agile in their response to newly disclosed threats and ensures that every instance of a vulnerability has been remediated.

    3. SCA Doesn’t Correlate Risk Across AppSec Engines

    Some traditional SCA solutions may be able to collect data from various scanning engines, but they are terrible at making meaningful connections between various data points. Because of this, “traditional” SCA is unable to provide the context required to deliver an accurate and nuanced assessment of the impact of one event and how risk might be compounded by other related events. SCA without context generates too many false positives—this results in your team wasting time chasing ghosts.

    PCA correlates risks and threats across all phases of the SDLC as well as from all AppSec scanning tools. By connecting the dots from numerous data sources, PCA helps your security and DevSecOps teams understand holistically how risks relate to each other so they can best tackle the most important security initiatives first.

    4. SCA Ignores Pipeline Security and Governance

    Modern development pipelines are complex and deeply interconnected–almost as if they are stitched and bolted together like Frankenstein‘s creation. Though managing security policies across the entire SDLC is an arduous and painful task, it is essential if you want to prevent software supply chain attacks. Unfortunately, traditional SCA solutions don’t even come close to helping you manage pipeline security and governance policies like enforcing least privilege or hardening authentication.

    Pipeline composition analysis identifies pipeline tooling misconfigurations and helps you implement best practices for security controls across your entire pipeline. Even better, to prevent your tooling from being bypassed, PCA alerts you when pipeline security controls like SCA scanning are removed or disabled.

    5. SCA Provides No Visibility into Your SDLC

    Traditional SCA solutions focus only on application code. They provide no visibility across your SDLC. They were not designed to scan development and deployment environments, so they cannot secure the later parts of your SDLC. SolarWinds was infiltrated via its TeamCity build server, which traditional SCA tools did not scan. So how can you protect your software supply chain if you can’t see it? Don’t be haunted by SCA tools’ inability to scan the entire SDLC.

    Vulnerable dependencies that impact your application can be found across the SDLC. You need to understand all development and deployment tooling–activity, settings, configuration, users and more–to protect your application. Pipeline composition analysis was designed from the ground up to understand your SDLC and bring context to your SCA findings to improve accuracy. It can give you complete visibility into the big picture to cut down on all the noise that traditional solutions generate.

    Pipeline Composition Analysis: Put Your Nightmares to Rest

    While it’s fun to be scared at Halloween, the last thing you want is to be terrified by your AppSec program. When the limitations of traditional SCA solutions threaten to keep you up at night, just remember that AppSec innovation requires the ability to do pipeline composition analysis across the entire SDLC, hence the shift from SCA to PCA. PCA plugs all the holes mentioned here, which hopefully can put even the worst nightmares to rest.

  • Mezmo Adds Observability Pipeline to Analyze DevOps Data

    Mezmo Adds Observability Pipeline to Analyze DevOps Data

    At the KubeCon + CloudNativeCon North America conference this week, Mezmo launched an Observability Pipeline platform that promises to make it simpler to manage, enrich and correlate machine data.

    Previously known as LogDNA, Mezmo is expanding the scope of its reach to add the ability to augment and analyze data on top of its core log management platform. Over time, that capability will be extended to add support for metrics and traces using the open source OpenTelemetry agent software.

    Mezmo CEO Tucker Callaway said the goal is to provide a framework for centralizing the flow of observability data that is now being generated by a wide range of sources across a DevOps workflow. It’s clear that DevOps teams require a means to enrich and correlate high volumes of data in motion that goes beyond simply pre-processing so it can be stored more cost-effectively, he added.

    The Mezmo Observability Pipeline achieves that goal by ensuring the right data is flowing into the right systems in the right format for analysis, noted Callaway.

    As the volume of data being collected by DevOps teams continues to dramatically increase the need to embrace a set of DataOps best practices is becoming more apparent. DataOps has emerged recently to describe a set of data engineering practices that applies many DevOps principles to the way data should be managed. Mezmo’s Observability Pipeline aims to make it simpler to apply DataOps best practices specifically to the data collected by DevOps teams.

    Of course, each DevOps team will need to decide for themselves how much data they want or need to store long term. However, increasingly DevOps teams are holding onto data much longer as IT environments become more complex, noted Callaway. That creates a need to more efficiently analyze as much data as possible and store it more cost-effectively, he said. Those twin goals, however, often create a physics problem in the sense that they can be diametrically opposed.

    In the longer term, it’s not quite clear how much DevOps teams are embracing observability. Most DevOps teams make use of monitoring tools that enable them to track a set of pre-defined metrics. Observability platforms promise to make it easier for DevOps teams to query a wide range of data types to discover potential issues before they disrupt an IT environment. The challenge is that many DevOps teams may not know what queries to form to discover those issues. It’s likely that, over time, machine learning algorithms will be increasingly applied to observability data to make it simpler to surface potential issues.

    In the meantime, however, the need for DevOps teams to find ways to manage all the data being generated by multiple types of applications is becoming a higher priority. In fact, in the absence of any meaningful way to manage all that data, the benefits of an investment in an observability platform is likely to be limited, at best.

  • Building New Value in IT Systems Management

    Building New Value in IT Systems Management

    For decades, companies treated the IT system as a cost center not much different from facilities maintenance. The job of IT was to keep the lights on, ensuring that the technology a company used worked and that the costs of keeping it working were kept under control. Those days are well over. Not only is technology an operational necessity, but everyone now recognizes that the way companies strategically use technology—whether they buy it, rent it or build it—has a direct impact on company performance. This goes beyond efficiency. Technology is key for everything from customer acquisition and engagement to competitive differentiation and advantage.

    As it turns out, however, not every company has been able to rethink and reconfigure its IT system to meet these new demands. That’s not surprising. Not only does this require a mindset shift from the C-suite on down, but it also requires restructuring the IT function to ensure it supports business objectives in a consistent and measurable way.

    While transforming IT from a cost center to a profit center can hardly happen overnight, there are three things that any company can do to turn IT into both a business enabler and a business driver.

    1. Establish Full Transparency Over Your IT System

    A mission-critical IT system needs to have a thorough understanding of what the organization’s IT landscape looks like. Spending money on duplicate applications and unused licenses is a waste. Unknown SaaS applications pose a persistent security risk. Optimizing spend and improving security are table stakes for business-enabling IT.

    The fact that anyone in a company with a credit card can subscribe to a SaaS service is the main challenge. Effectively tracking down all SaaS applications and licenses, along with monitoring SaaS usage, can’t be handled manually. What is needed, as with other tedious tasks, is automation. For example, by integrating a SaaS management platform (SMP) with SSO and expense management solutions, organizations can automate the discovery of SaaS. When the SMP also integrates with the primary SaaS vendors companies rely on (Microsoft, Google, Salesforce, etc.), it allows organizations to keep track of usage.

    The goal here isn’t for a centralized IT function to exercise more control over the software used by organizations. Instead, the goal is leveraging automation strategically to produce the kind of data-driven insights that allow organizations to make better, more informed decisions about the technology it uses.

    2. Promote Engineering Efficiency

    All companies today are software companies. This means that the engineering function needs to be thought of us as part of the overall IT estate. Not only does the IT system need to support engineering as it would any other business function, but the technology created by engineering itself inevitably becomes part of the broader technology landscape managed by IT.

    Mission-critical IT supports engineering efficiency in at least two ways. First, IT should provide the tools and capabilities needed to drive engineering efficiency. For a long time, measuring the efficiency of engineering was either not done at all or was based on meaningless vanity metrics such as lines of code. Recently, DORA metrics have emerged as the scientifically validated method, widely trusted by engineering teams themselves, for tracking efficiency. IT needs to enable easy tracking of these metrics.

    IT should also facilitate the creation and maintenance of a service catalog for engineering teams. Maintenance of this catalog not only promotes the reuse of services, itself a hallmark of engineering efficiency but also makes it possible to quickly mitigate security issues associated with microservices (Log4j being just one example from the last 12 months).

    Beyond efficiency, however, IT can also support in-house software creation by helping engineering teams with value stream management so they can track the business impact of the software they create. While efficiency can have a business impact (realized through time/cost savings), software drives the business forward when it effectively meets customer needs. IT can facilitate these insights by connecting the dots between customer feature requests, product changes and increased adoption (of products and features).

    3. Make Enterprise Architecture Strategic

    An IT function serving as a mission-critical part of the organization cultivates a comprehensive overview of the software landscape and supports engineering efficiency, as suggested, by viewing the development environment as an integral part of the broader technology estate. However, IT system leaders need to do something more to effectively collaborate with the business leaders and drive the business forward.

    Specifically, IT leaders need to provide the business with insight into all the ways technology influences business performance. This means highlighting risks that need to be addressed, uncovering opportunities that should be pursued, and suggesting options for future planning. Thoughtful and strategic enterprise architecture management here is the key.

    A strategic approach to enterprise architecture management means focusing less on governance (who gets to use what) and more on reducing complexity to promote agility and quick decision-making. The first step in this direction is the adoption of a common language that aligns the business and IT. Business capabilities and the creation of business capabilities maps for different departments serve this purpose.

    To make decisions about technology, business leaders need to know what they are working with. Reliable and current data is essential. For this reason, your enterprise architecture management needs to prioritize ongoing data collection and the maintenance of data quality.

    Finally, to make objectives and options transparent, IT needs to be capable of strategically displaying all enterprise architecture data in a way that can be easily understood and acted on.

    Where Do You Start?

    Taming an unruly SaaS landscape, making it possible to build better software and guiding technology strategy are all easier said than done.

    The good news is, ongoing innovation has created solutions that are easier to use, effectively leverage automation and integrate with all relevant systems. In other words, the tools are there. All that’s needed is the commitment to transforming the IT system and developing it into the critical business function it can and should be.

  • Making App Dev More Efficient

    Making App Dev More Efficient

    Applications of all kinds are continually popping up across various industries to help improve consumer customer experiences, provide entertainment, make certain tedious or error-prone tasks easier for workers and to help businesses improve their operations overall. In fact, there are over 5.4 million applications available across the Apple App and Google Play stores as of this year.

    As more and more individuals and organizations turn to applications for various experiences and to drive enterprise efficiencies, we need to take stock of the process we deploy in application development.

    The Inefficiencies of Application Development

    Today, it is estimated that there are more than 26 million software developers worldwide. Each of these developers brings a unique combination of creativity and technology acumen to the organizations they work for and the applications they create. However, the current application development process in many enterprises is riddled with inefficiencies that often require these talented developers to focus on tedious, mundane, non-innovation-related tasks. 

    Technical Debt: Renowned programmer Ward Cunningham first introduced the concept of technical debt in 1992, stating, “Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.” What many didn’t realize at the time was that technical debt would grow to be a major issue in software and application development years later. Recently, a new Scandinavian study found that developers dedicate 23% of their time, on average, to addressing technical debt. While this statistic alone should be alarming to enterprises—given their development teams are spending a quarter of their time on an issue that could have been avoided had a different approach been taken—the study also found that developers often have to introduce new technical debt in the process. 

    Maintenance: As with all technological equipment, applications require routine maintenance to ensure they remain working for the end user, provide ongoing value and are secure. However, unlike a car or kitchen appliance, applications see far more frequent updates from operating systems (OSes), cloud or hardware providers that need to be addressed by development teams. According to research from Stripe, developers spend more than 17 hours per week on maintenance issues like debugging or refactoring. For instance, take a look at Apple. From the time iOS 15 was released in June 2021 as a beta, there were more than a dozen updates made to the OS until early September 2022. That means that every single time the updates were released, developers had to spend time ensuring their applications were working correctly, were bug-free or complied with new security measures. When you add this to the fact that the average enterprise app has 110 third-party integrations, that is an extreme amount of maintenance.  

    The Reinvention of the Wheel: Of all the applications available today, many of them consist of the same foundational features and functionality—login, chat, social sharing, payments and more. What’s astonishing about this is that developers are frequently recreating each of these common features and functionalities from scratch every time they develop a new application. 

    It is clear that these tasks are highly inefficient, and it begs the question, “What can be done to address these inefficiencies?” 

    Prioritizing Innovation 

    It is not enough to just introduce some sort of technology in the hopes that it makes application development more efficient. It also requires an overall mindset change by enterprises. Yes, it is still of the utmost importance to get applications to market quickly and keep up with the pace of competition; however, innovation doesn’t have to be sacrificed to do so. 

    Innovation must be the key driver for enterprises and their development teams. Providing unique value and experiences to end users will truly set enterprises and their applications apart from the competition. But, the reality is that nearly 80% of technology leaders from around the world believe basic application maintenance and support are the biggest barriers to innovation. 

    Enterprises must be able to balance innovation, time-to-market and development processes that are free of technical debt and outrageous maintenance support. 

    Making Application Development More Efficient

    One way in which enterprises can balance innovation, commercialization, and development is through an emerging approach to application development known as features-as-a-service (FaaS). 

    A FaaS approach to application development enables enterprises to implement an agile and accelerated engineering process—similar to that of low-code/no-code without the constraints often seen with those vendors and their functionality today—without sacrificing quality, code or time. More specifically, it combines pre-coded “commodity” features with continuous and automatic updates that empower development teams to spend their time and creative minds on innovation. In most cases, this approach can help enterprises bring applications to market within weeks instead of months and limits the headaches on the backend for development teams down the line. 

    As enterprises work through the fourth quarter of this year, I implore them to reconsider their development process at a fundamental level and bring innovation to the forefront before the beginning of 2023. Time and resources are of the essence, and something must be done now to address the inefficiencies of application development. 

  • Fermyon Adds Managed PaaS for Building Wasm Applications

    Fermyon Adds Managed PaaS for Building Wasm Applications

    Fermyon Technologies this week launched a hosted platform-as-a-service (PaaS) environment, dubbed Fermyon Cloud, that promises to make it simpler to build WebAssembly (Wasm) applications.

    Announced during the Cloud Native Wasm Day event during the KubeCon + CloudNativeCon North America conference, Fermyon Cloud is now available in open beta.

    Fresh from raising $20 million in funding, Fermyon CEO Matt Butcher said IT teams can quickly build and deploy Wasm applications using a Fermyon Spin development tool and a Fermyon Platform the company launched earlier this year.

    Fermyon Cloud adds an opinionated PaaS environment managed by Fermyon that can start up Spin applications in less than a millisecond. It uses a serverless application programming interface (API) that makes it possible to cache and optimize applications as they are loaded, noted Butcher.

    Originally developed more than five years ago by the World Wide Web Consortium (W3C) to create a common format for browsers executing JavaScript code, Wasm is now used to rapidly build lighter-weight applications that can be deployed on any server platform. In effect, the promise of being able to write an application once and deploy it anywhere is finally being realized some 25 years after the initial introduction of the Java programming language.

    Wasm isn’t necessarily going to replace existing approaches for building applications, at least not any time soon, but it does provide an alternative to serverless computing frameworks for developing lighter-weight applications that run faster.

    In addition, at a time when organizations are becoming more concerned about application security, the appeal of Wasm is clear. Existing approaches to building applications rely on the aggregation of software components that tend to lack distinct boundaries between them. As a result, it becomes relatively simple for malware to infect all the components of an application. In contrast, Wasm code runs in a sandboxed environment that isolates execution environments to eliminate the ability of malware to laterally move across an application environment.

    Wasm is especially well-suited for building lightweight applications that previously might have been deployed on a serverless computing framework, noted Butcher. The issue developers are encountering when using a serverless computing framework is that those frameworks are based on containers that have comparatively slow startup times, he added.

    It remains to be seen how much traction Wasm gains in the months ahead, but Butcher said he expects most backend services, such as databases, will continue to be built using containers. Fermyon is betting, however, that many future cloud-native applications built by the average developer will be constructed using Wasm. In the longer term, Wasm will also soon be used to build applications infused with machine learning algorithms, said Butcher.

    In the meantime, DevOps teams should get ready for more Wasm code to move through continuous integration/continuous delivery (CI/CD) pipelines. The challenge, of course, is that code will need to be managed alongside all the other software artifacts moving through those pipelines, as well.

  • Sigstore Code Signing Service Becomes Generally Available

    Sigstore Code Signing Service Becomes Generally Available

    A free digital signing service for software created by the Sigstore open source community has become generally available this week via the cloud.

    Announced at the SigstoreCon event that occurred during the KubeCon + CloudNativeCon North America conference, the cloud service makes it possible for developers to both cryptographically sign artifacts and verify that the components used to build applications are secure.

    The Sigstore community operates under the auspices of the Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation. Sigstore project sponsors, including Google, Red Hat, GitHub and Chainguard, have committed to operating a service with a 99.5% uptime and round-the-clock pager support. More than 70 organizations, including Shopify, Autodesk, Trail of Bits and Rancher Government Solutions are actively involved in maintaining and scaling Sigstore.

    More the four million Sigstore signatures have been logged using Sigstore thanks mainly to Sigstore support that is already included by maintainers of open source Kubernetes and Python projects. The npm community also announced it is working to integrate Sigstore in all packages used to build JavaScript applications.

    Bob Callaway, co-founder of Sigstore and technical lead and manager for the Open Source Security Team at Google, said this service will make it much simpler for developers to add digital signatures to code in a way that can be more easily verified within a software bill of materials (SBOM).

    The goal now is to expand the number of open source projects that will commit to employing Sigstore to better secure software supply chains, he added.

    The security of open source software became a major concern in the wake of the open source Log4j zero-day vulnerability. Sigstore was under development prior to that disclosure, but the Biden administration’s executive order requiring Federal agencies to verify the security of open source software has enabled the availability of more resources for the project.

    In response to the executive order, the OpenSSF outlined a 10-point plan to secure open source software that would require more than $150 million in funding. One of those goals is to have 50 of the top 200 projects and 1,000 of the top 10,000 projects adopt an interoperable approach to software signing. That project was estimated to cost $13 million for the first year and $4 million per year beyond, with a one-time additional $10 million required after the first year.

    It’s not clear how broadly developers will employ digital signatures, but given the current level of concern, it’s probable most organizations will soon require any code they use to be digitally signed so it can be verified as immutable. The next big challenge, of course, is finding a way to operationalize the collection of all the SBOMs that will include those signatures. After all, SBOMs are only the first step in a larger DevSecOps workflow that will enable organizations to accept or reject software components based on all the data being collected.

  • Software Quality is the Heartbeat of the Best Organizations

    Software Quality is the Heartbeat of the Best Organizations

    JPMorgan recently announced it was hiring 2,000 engineers, despite the gloom in global economic markets. Is this not an odd risk for an organization to take, given the demand for (and cost of) software developers today? What’s happened?
    JPMorgan’s hiring drive is no mystery. Virtually every company in the world is leveraging software, with many transforming themselves into software organizations to drive new efficiencies, widen profit margins and create new markets. Software has been disruptive, so much so that the current era has been dubbed the “third industrial revolution.” Consumers are benefiting from switching providers at the click of a few buttons and most consume services through software. Bad software experiences today mean rapid customer churn and competitive disadvantage.

    Quality Should be the Heartbeat of any Organization

    In spite of quality’s importance, testing is still perceived as a secondary discipline and a check-the-box exercise. This reflects practices developed for assembly lines during the original industrial revolution. A product gets assembled and testers verify its quality as it leaves the production line. Testing’s goal in this context was to identify problems before they reach end-users. This same approach has been applied widely to software development and is still used by “waterfall” organizations.

    Yet, software development has been revolutionized by new methodologies and practices. Software today is constructed at incredible speed. Notice I’m using the word “constructed” and not “built”—the best developers leverage libraries or APIs to assemble software using existing components. They then add a few pieces of custom business logic to define the intellectual property of an application. Many organizations don’t even code software, instead customizing off-the-shelf software with vast configurations to modify functionality.

    The assembly-line approach to software development is no longer appropriate. When testing is left late, defects are found before a release or even in production. Release cycles are longer and products poorer quality. Yet, many organizations still stick religiously to this approach, with testing teams operating in distinct silos.

    The Role of the Tester is Evolving

    The best organizations in the world are changing how software is built. Testing is no longer a check-the-box exercise at the end of the assembly line. It is transforming into a discipline that revolves around evangelizing and driving quality throughout an organization.

    Quality should be baked in as early as possible, touching every artifact produced. Ideally, this should occur from the start, as a requirement is formulated, all the way through to the software being shipped. The idea of siloed teams working on different pieces is outdated. For quality to thrive, teams must collaborate.

    Modeling offers one approach to driving quality and collaboration throughout an organization. Its journey starts with a requirement. These are typically formulated by business analysts or product owners. Having been a product owner, I found that this usually involves lots of scribbles on paper before communicating to stakeholders on a whiteboard. The preliminary requirement then gets reviewed and refined.

    At some point, the requirement will be converted into walls of text in a project management tool. Modeling intervenes at this stage, adding visualizations that make requirements more consumable, less ambiguous and easier to understand.

    The word “modeling” may seem a scary concept, but it need be no more than a flowchart that encapsulates a requirement. The benefit of modeling is not only that it provides a digital collaboration asset; models can also generate coverage-focused tests before a line of code is written. Modeling creates requirements that are quality-driven and quality-driving.

    The Journey of Quality-Driven Requirements

    • Models Provide Data Upfront

    Developers today require rich data to develop, while testing is only as good as the available test data. Yet, many organizations still feed slices of production data into developers and test cases.
    This approach is problematic for several reasons. First, it often risks legislative non-compliance. Second, production data features large volumes of low-variant data. Data may not then be available when developing unusual-but-important scenarios. Imagine an overdrawn account with several beneficiaries: What should happen when a transfer is attempted?

    Modeling defines data explicitly in requirements. It allows engineers to understand a requirement explicitly, including the relevant data. From the modeled definitions, correct data can then be masked and subset, or generated synthetically. Data can further be delivered on-demand and at scale using database virtualization.

    • Models Provide Understanding

    Once the requirement and necessary data are available, code can be developed. Using text-based requirements, misinterpretations often lead to incorrect functionality. Visual models instead align closely with business logic, and we often see developers enriching models with additional context so that models align with the actual software implementation.

    • Models Generate In-Sprint Tests

    Once the code is developed, the same tests which were created earlier from the requirements can be executed against new functionality. This facilitates reuse and means the test cases are ready to be executed immediately.

    Some tests might further be selected for automation. In this instance, automation can be overlaid onto the requirements model, reusing code to generate scripts. These scripts then become part of the standard regression pack as new features are added to the product.

    A Change Request Comes In

    With iterative development, change requests are common. This typically creates havoc for quality, since it is difficult to understand the impact of a change, and time-consuming to update artifacts like tests and data.

    Traceability is key to overcoming this challenge. Models can connect to external DevOps tools to provide end-to-end traceability, from a requirement, all the way to code and tests. The impact of a change can then be identified directly. Models can be updated, automatically synchronizing test data, test cases, and automation to bring everything up to date.

    Testing for the Fourth Industrial Revolution

    Software is playing a vital role in virtually every organization, as already recognized by organizations thriving today. Baking quality in early is the only way to ensure quality software. The role of testing must change on a large scale. Models offer a way to achieve a “shift left” and collaborative approach to quality while testing earlier and more rigorously by automating and optimizing test and data creation.

  • BMC Extends Scope of ServiceOps Initiative

    BMC Extends Scope of ServiceOps Initiative

    At the BMC Exchange conference this week, BMC revealed it has updated its Helix IT service management (ITSM) platform to include mapping and discovery capabilities. Those updates make it simpler for cross-functional teams to correlate incidents, alerts and data and pinpoint root causes faster.

    In addition, BMC is now applying pre-trained artificial intelligence (AI) models to observability data and the service models it tracks to assess service health, predict service impacts and proactively manage IT environments as the volume of metrics, events, and alerts continues to increase.

    Finally, BMC also announced that the BMC Helix Control-M orchestration engine now supports open source Apache Airflow, a workflow management platform for defining data pipelines using a Python programming tool.

    Margaret Lee, vice president and general manager for digital service and operations management at BMC, said these updates extend a unified ServiceOps initiative the company launched last year to unify ITSM, DevOps and customer service desk workflows. While each of these disciplines may have distinct objectives—such as speed and reliability—that are often in conflict with one another, Lee said it’s apparent IT organizations need to reduce the current level of friction that is often encountered when managing IT.

    As a software-as-a-service (SaaS) platform, Lee said the BMC Helix platform achieves that goal via a consumer-like user interface that can be easily personalized. The goal is to make it simpler to create virtual agents, knowledge bases, live chat and tickets that make it easy for customers and employees to request IT help, she added.

    Achieving that goal, however, will require IT organizations to rely more heavily on AI models to automate routine tasks as IT environments become more complex. There are simply not enough IT professionals available to manage IT environments using legacy processes, she noted.

    In addition, end users and customers are expecting service requests to be automated in way that allow them to resolve issues via a mobile device, added Lee. In many cases, how cumbersome IT processes now has a direct impact on both customers and employee retention. The primary goal is to reduce the number of interactions required to complete a task such as changing a password or provisioning a laptop whenever a new employee needs to be onboarded.

    While the pace at which IT management is evolving will vary greatly from one organization to the next, it’s apparent that many DevOps principles are being absorbed into traditional ITSM platforms. The challenge is that cultural divide between DevOps teams and centralized IT teams is still very wide. Most organizations are still trying to determine how far left they want to shift responsibilities toward application development teams versus continuing to rely on IT administrators using graphical tools to centrally manage IT workflows. It’s rarely going to be an all-or-nothing proposition. Rather, the issue is finding a way to strike a balance that enables organizations to be as agile as possible without introducing unacceptable levels of risk to resiliency.

  • Three Ways to Speed up SAST

    Three Ways to Speed up SAST

    In modern, continuous software development life cycle (SDLC) processes, when code is written and before it’s committed to the repository, it’s run through testing, which may include unit testing, regression testing or static application security testing (SAST). The benefit of SAST for DevSecOps is the real-time feedback it provides developers as they create and make changes to the source code before they submit it. The amount of time that’s acceptable in this kind of workflow is 15 minutes to 30 minutes maximum, with some allowance for how big the change is. Fast feedback means quicker turnaround and more progress on features. The business value lies in those features. Testing makes sure they work.

    Depth Versus Breadth

    Checkers used by SAST are not all created equal, at least in terms of the computation power needed. The time for analysis is directly related to the complexity of the checkers used to detect certain types of vulnerabilities. Typically, coding standard enforcement rules are easier to compute versus complex tainted data analysis used to detect command or SQL injections, for example. Therefore, there is a trade-off to be made in the types of bugs and vulnerabilities being detected. When developers are coding, it makes sense to optimize the compute time for SAST to reduce any delays. During software builds, more time and compute power is available so depth and breadth can be increased to catch complex vulnerabilities.

    Compute Power

    SAST analysis time scales well with compute power. The availability of more computing resources such as CPU and memory, can impact the time of analysis. Bigger is better in this case and investing in additional hardware for SAST can pay off in terms of productivity.

    Codebase Size

    SAST analysis time is also dependent on the amount of source code analyzed. Depending on the application, the code base is a determining factor in terms of what depth and breadth of analysis is optimal.

    There is some correlation between small code base size and strict requirements for SAST such as in safety-critical software. It’s expensive to build and test safety-critical code, but code size is typically smaller than in other types of applications. However, these products need to conform to industrial coding standards and thus require rigorous levels of testing and analysis to detect a wide variety of defects.

    Larger and extremely large code bases require a trade-off in terms of depth and breadth of analysis. Prioritizing the types of defects and vulnerabilities to concentrate on is critical.

    Speeding up SAST means reducing the amount of work. The most intensive operation is a full analysis of the entire source code base. Just as full compilation from scratch takes a long time, the same is true of SAST analysis.

    To accelerate SAST processing times, follow the same three techniques used to avoid compilation in large C/C++ projects: incremental analysis, component analysis and developer analysis.

    1. Incremental Analysis

    Incremental builds are a major factor in reducing developer build times for C and C++ projects. Small changes don’t require the entire code base to be recompiled. It’s impractical and unnecessary. When a developer modifies something, only what’s needed to be recompiled gets recompiled automatically, and the build infrastructure handles it.

    SAST works in a similar manner. As source code is being compiled, SAST parses the same source code and creates a program model. The second phase is analyzing that program model against the configured checkers and generating warnings. The warnings typically go into a database for later analysis.

    During the build of the software, SAST is run in parallel. Although similar to a compiler in operation, SAST must do more work than the compiler and takes longer to finish and provide results. The amount of code needed to recompile for each change directly impacts analysis time.

    Meanwhile, SAST of an incremental build requires a lot more work than just analyzing the changes. SAST uses abstract execution of code flow through multiple different compilation units. So even though a single file is modified, the calls it makes to other files need to be analyzed as well. The incremental analysis doesn’t increase the build and parse workload, but the analysis scope will be a lot bigger than expected. So, whenever you do an incremental build, SAST will need to reanalyze the dependencies, changes and related bits and pieces around those changed units.

    Nevertheless, it remains a good option for reducing SAST analysis times.

    2. Component Analysis

    If the software being developed has a loosely coupled, component-based architecture with well-defined interfaces, it’s possible to isolate both the testing and the SAST analysis to the component where changes were made. Just as this type of architecture simplifies many aspects of development, it pays off with SAST as well. It’s possible to analyze the component in isolation and get good results.

    3. Developer Analysis

    To further reduce SAST workload, start the first build without SAST, which removes the burden of the extra computation needed. Since subsequent builds are a lot smaller and build speed is much faster.

    However, there is an increase in false negatives due to the smaller scope of analysis since only files that changed are analyzed. This smaller scope means that all possible dependencies are outside the program model and are not considered in the analysis. However, this rapid analysis works well with coding standard enforcement, for example.

    Developer SAST

    Where to Start?

    Begin with a full build and analysis. This sets a baseline for detected bugs and vulnerabilities and establishes a performance target for reducing build time. This step will determine the exact number of lines of code, how long the analysis takes and its impact on build server load. These metrics should drive the analysis optimization effort.

    Given the options, the first step should be a component-based analysis. If this works well, it’s still possible to further reduce analysis with incremental and developer analysis within components. This approach can be integrated into typical developer workflows.

    When creating a developer branch for implementing changes, quick developer local analysis will catch local errors. These tend to be the largest set since they represent the largest part of the code changes. When enforcing coding standards, this is the point where violations are identified. A full analysis is recommended on a merge request to detect remaining defects related to cross-component-boundary behavior.

    Speed up SAST

    To speed up SAST consider using these techniques at the appropriate development stage to reduce time to complete, while still producing excellent results. Keep in mind that:
    ● Complete analysis yields the best and most precise results,
    ● Incremental analysis only analyzes recompiled code that has been modified,
    ● Component analysis works well in software that follows a well-defined component architecture and
    ● Developer (local) analysis provides a good compromise of precision and analysis time.

  • When Self-Service DevOps Forgets Who it Serves

    When Self-Service DevOps Forgets Who it Serves

    For years, self-service has been touted as the salvation of DevOps. Why? Well, every department in your organization has its own reasons for putting self-service DevOps at the top of its wish list:

    ● Ops teams need to ensure high availability, performance and security
    ● Dev teams need speed, self-sufficiency and transparency
    ● Security teams need guardrails, compliance and accountability
    ● And, of course, the C-suite needs ROI, strategy and insight on top of everything else.

    These goals often cause tension—especially between Dev and Ops. Verifying security takes time that developers usually don’t have. And once at scale, even the fastest manual processes break down. Like at the U.S. Department of Defense, for example, with a workforce of close to 200,000 internal and external developers.

    Self-service DevOps platforms promise virtual self-service kiosks for software delivery, uniting teams and creating a frictionless path to independent developer access and provisioning. According to the State of DevOps report from Puppet, self-service DevOps is a hallmark of “highly evolved” DevOps organizations.

    Many self-service solutions are available, but has anyone actually figured it out yet? After all, if everyone is happy with their self-service approach, why are they still searching for answers?

    Approaches to Self-Service DevOps

    True self-service DevOps promises a wide range of benefits: It lets Ops stop rushing to put out fires and shift to a big-picture strategy; it makes compliance simpler with automation, safety guardrails and clear accountability. And, there is the promise of tremendous capital efficiency.

    It’s no surprise that so many vendors are rushing into the space, offering so many flavors of approaches that Baskin-Robbins would be jealous:

    ● Internal developer portals
    ● Shared service platforms
    ● Workflow automation
    ● ChatOps/Chatbots
    ● Service catalogs
    ● API marketplaces
    ● Cloud competency centers

    But let’s face it: In reality, most of these approaches fail. And when self-service isn’t working as it should and companies fall behind on their service request queue, many DevOps teams default to risky practices that aim to expedite operations but actually expose the company to attack:

    ● Over-privileged and over-permissioned accounts that give broad access to anyone who gets their foot in the door, as happened with the SolarWinds breach of 2020
    ● Inactive accounts that leave a door open to cybercriminals, as we saw with the Colonial Pipeline breach of 2021.

    What you might not realize is that these two risks are so prevalent that IBM Security X-Force penetration testers managed to use them to hack into 99% of client cloud environments. So when we say self-service DevOps is failing, it’s failing big-time.

    So what’s the solution? Why is true self-service DevOps so hard to achieve?

    Why They Fail

    There are actually a number of reasons why today’s self-service DevOps platforms fail:

    ● They break end-user workflows, forcing users to change their behavior and learn new processes.
    ● They force devs to switch context which kills productivity.
    ● They need domain expertise to operate, reducing the target audience that can use them.
    ● They’re heavily reliant on human-in-the-loop involvement, often creating more work.
    ● They require significant effort to update and maintain workflows and meet or exceed SLAs.

    Most importantly, with current self-service DevOps solutions, users are often flying blind. They don’t know what they don’t know; for instance, how to locate, search, identify, create and use resources and workflows.

    Imagine, for example, choosing a self-service checkout to avoid a long line at the grocery store cashier. But then …

    a) You get stuck behind someone frantically trying to scan a wrinkled, expired coupon.
    b) You get stuck behind someone who doesn’t understand barcodes and scans all their items twice.
    c) You forget that alcohol requires age verification, causing an alarm bell and flashing lights to start going off above the register (spoken from experience; embarrassing).
    d) All of the above.

    In all these scenarios, a human attendant must come running to fix things, defeating the whole point of self-serve. Self-service is only self-service if it’s easy to use even when things go wrong.

    With most so-called self-service DevOps solutions, if something goes wrong, you’re back to waiting—drawing the operator back into the loop—just like with the old system.

    What True Self-Service Devops Looks Like

    To achieve true self-service DevOps, you need a few elements in place.

    Headless UI
    Most developers are (gasp!) human beings. So the best interface is no interface at all. Why not let developers express their needs freely in natural language using existing tools such as Slack, Microsoft Teams or a command line interface?
    Workflows
    Developers are frustrated when processes don’t make sense or take more steps than necessary. Let’s adopt workflows that actually handle all the nitty-gritty—so that all developers need to do is ask (in their own words) to be granted access to resources, information and workflows.
    Intelligence
    For developers, there’s nothing more frustrating than not knowing where to turn for the information you need. True self-service DevOps, powered by AI, will use your existing data to provide a single source of truth with all the answers and insight devs need.
    End-User Feedback Loop
    An ideal self-service DevOps platform will ask the developer for feedback that is then used, in turn, to improve the platform itself. First, because everybody enjoys having their voices heard and second because this reinforces the role of the platform, which is to serve DevOps by improving the end-user experience. Similar feedback loops have been used in apps like Waze, which uses social navigation to continuously improve the quality of its end-user experience. Self-service DevOps is next in line.

    Once you’ve accomplished all of this, you can work more efficiently, providing immediate on demand assistance so your Dev and Ops teams can work without interruption.

    How will you know when you’ve achieved true DevOps self-service? When you forget you ever did things any other way—and wouldn’t go back for a million dollars. When you wake up one day and realize that your platform not only lets you build trust and accountability but also gets all your teams working together…and—(gasp) dare I say it?—having fun.

    That will be the last DevOps platform you ever buy.