Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: The EU wants to put servers in orbit, a GDPR penalty for Discord, and Amazon has the hatchet out. (more…)

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: The EU wants to put servers in orbit, a GDPR penalty for Discord, and Amazon has the hatchet out. (more…)

Foundations and frameworks, concrete and steel—not exciting. But that’s the foundation and framing of pretty much every modern building. Everything else that is part of a building–flooring, wiring, lighting, room placement and so on—is made possible by the foundation and the framework. If you want DevOps to succeed in your organization, you have to start by building a solid foundation and a framework, including security. What’s below is a suggested approach to the concrete and steel of DevOps.
1. Leadership and Governance
Someone has to make the decisions, be responsible for leading the team(s), give direction and provide operational governance. Securing DevOps is not just about compliance with a set of security concepts; it’s also about setting the right tone for the workplace. Leadership and governance are key enablers of development success.
The pressure to deploy software faster and more often is real—but so are the security, regulatory and many other concerns around automation that may uncover bugs or defects in software before it gets pushed to production. In an environment where there’s pressure to release code faster, how does one balance these competing priorities? What’s more important: Speed or quality? This tension between speed and quality remains one of the main challenges faced by organizations adopting DevOps practices today.
Organizational leadership and governance regarding people and data must be in place to keep people working on the right things, keep up morale, provide direction about project prioritization and budget the right tools for getting the job done.
An important subcategory of governance is data classification. Where does the data reside? Over which channels does it flow? What kind of data is it? Classifying the data has long-term implications for resource procurement such as what storage to use and if it needs to be encrypted, how stringent access control for the environments needs to be and interoperability of the various technologies.
2. Regulatory Compliance
It wasn’t too long ago that if a business remained only national, then there weren’t necessarily many regulatory requirements. Presently, if a business has a website (and who doesn’t?), then international and cross-border data transfer is a given unless one takes the course of geo-restriction, which in itself can be a major undertaking with repercussions.
Determine regulatory needs. From where do people or other services access the data? What cross-border transfer is involved? To paraphrase the late 1960s-1980s public service announcement, “It’s 10:00 p.m. – do you know where your data is?” Design and implement accordingly.
3. Risk Management
One cornerstone of technology management is that it’s all about risk. If there’s a serious incident such as data intrusion or theft, what will the cost be to the organization and was the company part of the problem (e.g., negligence)?
What are the technical vulnerabilities, threats and risks presented by the product or service? What kinds of testing (e.g., regression, penetration, vulnerability, CTI) would it take to assess and address the risks? What does the patch cadence look like?
On the personnel side, what are the risks of key players leaving? What’s the morale of the team regarding aspects such as culture and pay? What’s being done to address those risks? Included are third-party, contractual, and privacy risks (remember data classification above?).
NPM packages are a growing trend as an avenue of exploitation; uptime is a common requirement for meeting obligations, and privacy is considered by many to be more than a privilege but a human right, and not something to take lightly.
The foundation activities undergird the entire structure of DevOps, but also consider what shapes the structure.
1. Software Development Life Cycle (SDLC)
The SDLC is vital. Without it, where do all personnel (team, contractors, consultants) know how to proceed with designing and implementing the product? What model is used, and who’s responsible for what? It’s all here in the SDLC. Of course, things like references for API architecture specifics and corollary application documentation would make one feel like wandering in the Library of Congress (and make developers want to burn it down like the ancient Egyptian Library of Alexandria!).
Speaking of APIs: Due to the enormous rise of API use in all industries, including health care and retail, any use of APIs must be part of the SDLC.
Not only are APIs on the rise for public use and interaction, but microservices use within an org has increased drastically. APIs are necessary internally and externally for businesses to reach, maintain and accelerate the speed of innovation, so including them as critical resources is crucial. The average API usage per company increased by 221%, so attending to that ecosystem by ensuring proper activities, especially security, are included is needed for developers.
The SDLC is there to make workflow and production consistent (and concomitantly easier) by codifying what needs to be done for both company and customer, not to make work more difficult by presenting nauseating tedium. Like any corporate policy, it needs to remain useful, while not watered down.
Here are a few specific aspects to include:
● Threat modeling in some form needs to be included. It doesn’t have to be a magnanimous effort, but the realities of threats will play a major part in development.
● Testing. Regression, stress, pentest, security – these and others will help guide what kind of testing environment one will need and lead to any need to increase items such as capacity and scalability.
● Secure coding. If it’s being done, take steps to improve. If it’s not being done, just get started. It’s always hard to go back and fix insecure code, but it’s a necessity due to the many risks and regulations involved in providing software to the public.
2. Training
Training is required because businesses have to adapt as technological changes create new opportunities that lead to continuously changing customer demands.
Training is everywhere! There’s no one right place or right approach – just get training and change as needed (specifics should be handed down by leadership).
On a strict and lowball budget? One option is OWASP membership (which costs $50 per person per year) and includes several types of AppSec and DevSecOps training. That small investment can help raise the level of clean and secure coding, threat and security consciousness and professional reputation.
An issue with setting any course in business is that it’s semi-permanent. A DevOps model put in place becomes part of the foundation, which can make one reticent to commit. The only way to prove the chosen methods is to set them in stone and do them consistently for a long time. Aspects of the model may change over time, but once that foundation is put in place, it takes an enormous amount of work, time and money to change it. The change wouldn’t be a remodel, but a reconstruction, so the model takes proper consideration at the outset to chart the right course.
While DevOps needs a solid foundation, it’s an always-in-motion model. There are new technologies, new people, business changes, customer demands and societal and cultural shifts. These concepts can be included in calculations, but what those changes will be is incalculable. Hire and train the right people and the necessary steps and adjustments will be possible.

Oxeye today disclosed that it has discovered a critical vulnerability in the open source Backstage software used to build developer portals. Backstage was originally created by Spotify.
A 1.5.1 update to the Backstage platform remediated a sandbox escape vulnerability that can occur via a third-party Scaffolder plug-in that could be used to conduct unauthenticated remote code execution (RCE).
Oxeye CTO Ron Vider said Oxeye worked with Spotify to responsibly disclose the issue and enabled a patch to be created prior to the disclosure of the vulnerability, which Spotify gave a 9.8 out of 10 severity rating.

Backstage is now being advanced under the auspices of the Cloud Native Computing Foundation (CNCF), but is already widely used by organizations such as American Airlines, Netflix, Splunk, Fidelity Investments, Epic Games and Palo Alto Networks, as well as by Spotify.
The vulnerability is another example of a potential template-based attack through which run shell commands can be manipulated to inject malware into an application development environment.
In general, separating the logic from the presentation layer as much as possible can greatly reduce exposure to the most dangerous template-based attacks.
Vider said Oxyeye is focusing its vulnerability research on cloud-native platforms such as Backstage that are generally being used to build modern applications based on a microservices architecture. It’s not clear to what degree organizations have transitioned away from more monolithic approaches to building applications, but it’s clear that a much larger percentage of applications are now being constructed using microservices.
Cybercriminals, of course, are targeting these platforms as part of a sustained effort to compromise software supply chains in the hope that the malware they inject will find its way into multiple downstream applications. Those attacks, in turn, are convincing more organizations to embrace DevSecOps best practices to better ensure the integrity of their software supply chains. The challenge is many of those attacks are being aimed at open source software projects that don’t always have enough resources to quickly develop a patch in the event a zero-day vulnerability is disclosed.
While it’s impossible to know the extent to which cybercriminals are already exploiting vulnerabilities in DevOps platforms, the level of cybersecurity scrutiny being applied has dramatically increased in the wake of a series of high-profile breaches. The Biden administration has even gone so far as to issue an executive order requiring federal agencies to better secure their software supply chains. As a result, DevOps teams should expect there to be a lot more DevOps platform vulnerability disclosures in the months ahead, and that those vulnerabilities will require immediate patching.
Like it or not, software supply chains are going to become more secure. The only thing left to be determined is the level of pain that will be experienced as more vulnerabilities are discovered in the DevOps platforms that were used to construct them.

Codenotary today launched a tool that enables an application to automatically generate a software bill of materials (SBOM) by adding a single line to its source code.
Codenotary CEO Moshe Bar said TrueSBOM makes it possible to self-report the components used to construct applications to any organization that uses them for the first time. In contrast, existing SBOMs only provide a snapshot of the components of an application at the time it was initially created, he added.
IT organizations are being asked to trust application providers that all the modules specified in the SBOM are the only components being used. Codenotary is making a case for an approach that enables the organization consuming that software to spin up an SBOM in real-time on demand.
From a DevSecOps perspective, adding a single line of code to the application to enable TrueSBOM should also eliminate the need to create and maintain separate text files to generate an SBOM.

Awareness of the need for SBOMs has skyrocketed since the Biden administration’s executive order made it clear that federal agencies would require them from any software provider starting next year. Many enterprise IT organizations are likely to follow suit as part of a larger effort to better secure software supply chains in the wake of a series of high-profile cybersecurity breaches.
That approach also makes it a lot simpler for organizations to accurately pinpoint where components are actually running any time a new zero-day vulnerability is discovered, added Bar.
Priced at $1,450 per application stack per year, there are also lower-cost instances of TrueSBOM available for applications based on either a serverless framework or using the portable WebAssembly (Wasm) format.
Most internal DevOps teams already have a good handle on what software components are being employed with the applications they deploy, said Bar. The issue is that the organizations that use that software can’t easily verify what components are being employed, he added. That’s problematic because an organization may have decided to prohibit deployment of a specific software component because of a known vulnerability. Rather than trust a text file created by the provider of an application, TrueSBOM allows the user of an application to maintain control over their software environment, noted Bar.
It’s not yet clear how most organizations will operationalize SBOMs now that more of them are being created. Ideally, organizations should be able to approve only software that has components that have been verified to be secure. Armed with those insights, over time, it should become simpler to start reducing security technical debt with more secure applications, added Bar.
The challenge, of course, is that the Codenotary solution requires the addition of one line of code to an application. However, as SBOM mandates become more stringent, the number of application providers that are anxious to comply with rules for securing software supply chains should increase. The issue now is finding a way for both the developer and consumer of that software to streamline a verification process that, in its current form, is too cumbersome to effectively manage.

Aqua Security this week claimed it is the first software supply chain security platform provider to meet the attestation requirements as defined by an executive order issued to federal agencies last year by the Biden administration.
A supplementary memo issued by the Biden administration required federal agencies to collect compliance attestation letters for all software they use by September 14, 2023. Those attestation letters need to confirm that the software development environments they employ are secure, that sources of code are trusted and that code vulnerabilities have been remediated, provenance for data and code is being tracked using a software bill of materials (SBOM) and that these processes are being consistently maintained.
Eilon Elhadad, senior director of supply chain security for Aqua Security, said Aqua Security was able to meet those requirements within 100 days of the issuance of that memo. The Aqua Security platform for securing software supply chains is based on a platform it gained with the acquisition late last year of Argon, a provider of a set of tools that provides an agentless approach to integrating with continuous integration/continuous delivery (CI/CD) platforms that maps the development environment. Armed with those insights, it then becomes possible to prioritize and automate the remediation of vulnerabilities based on security best practices and compliance rules, noted Elhadad.
The Argon platform also includes code-tampering detection technology that performs validity checks to make sure the integrity of the build process has not been compromised.

Naturally, there’s plenty of time for other security platform providers to comply with the requirements set forth in Biden’s executive order. However, software development teams are also in a race against time. In the wake of a series of high-profile breaches, it’s become apparent how vulnerable software supply chains really are. Cybercriminals have increased their efforts to compromise developer credentials so they can embed malware in software components and activate them downstream if future application incorporate that component, noted Elhadad.
It’s not clear how closely enterprise IT organizations are tracking the efforts of the U.S. government to secure its software supply chain, but any entity that builds software used by any of those agencies will be impacted. Advocates of DevSecOps best practices within enterprise IT organizations will undoubtedly use the Biden administration’s requirements to define a minimum baseline for securing software supply chains.
In the meantime, a new front has clearly been opened in the cybersecurity war. The more applications are targeted, the greater the impetus for shifting responsibility further left toward application development teams becomes. The challenge is not just scanning the code being written by developers, but also making sure the software build created by development teams has not been compromised by bad actors that have inserted themselves into the process, noted Elhadad.
Given the magnitude of the effort required, it’s not likely that every software supply chain will be secured any time soon. However, the responsibility for securing those software supply chains is weighing more heavily on DevOps teams.

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: People won’t shut up about Rustlang, and Musk mandates Twitter teams return to the office. (more…)

DevSecOps is having a moment in the federal government. With President Biden’s Executive Order on Improving the Nation’s Cybersecurity and federal agencies’ issuance of DevSecOps best practices based on the Enduring Security Framework, government organizations are shoring up and standardizing their DevSecOps processes to meet today’s cybersecurity challenges head-on.
Integrating security into the development process is particularly important for federal agencies that have invested significant time, effort and money in creating and running software development factories. These agencies are actively trying to balance security and innovation–and many are succeeding. The United States Air Force’s Kessel Run program is an example of a tightly run and innovative software factory.
The question is: How can they continue this success as the calls for standardized cybersecurity grow increasingly louder?
It’s tough to balance security and innovation and wrap them both in a standardized process. It’s certainly not impossible, as Kessel Run proves. But it does require a high level of rigor to ensure that applications are being developed at speed while taking the necessary time to mitigate and minimize security risks and vulnerabilities.
Projects like Kessel Run and others work well in part because they bake security into the development process from the outset. That core tenet of DevSecOps allows teams to integrate and standardize security throughout their development cycles without losing momentum.
But, there are challenges to these standardization efforts that are inhibiting DevSecOps practices.
First, many government agencies are still working with legacy systems and toolsets rather than the modern technologies that many commercial companies that employ DevSecOps have access to. These technologies can include everything from collaboration platforms, to messaging applications, to underlying platforms that tie all the work being done together.
Second, despite the government’s commitment to innovation, old ways of thinking still persist in many pockets of the public sector. Some important people still think, “Why change when email has worked just fine for all these years?” Those people fail to immediately see the value of modernizing their toolsets and practices.
But outdated technologies and processes won’t help government agencies balance security and innovation–and they won’t work with new cybersecurity policies and mandates. For instance, president Biden’s EO is filled with allusions to ensuring visibility into software development practices and the software supply chain. Doing both requires developers, security managers and operations managers to be on the same page and in constant communication at all levels of the development cycle.
Yet even in the most mature DevSecOps environments, this isn’t always possible. Because while DevSecOps is driven by collaboration, that doesn’t mean that developers, security managers, and operations managers always know who’s working on what, or what the status of a particular project is at all times. This is particularly true in a government setting where old processes, technologies, and siloed teams are still present.
But maintaining the balance between security and innovation is highly dependent on being able to have unfettered situational awareness throughout the development lifecycle. For example, a developer needs to know if a security operations teammate is working on fixing a vulnerability the developer discovered during testing. Likewise, an operations manager needs to know the status of the patch, as that might impact other factors important to the development process, including time to deployment.
High-level dashboards help all team members quickly ascertain the status of projects in development and are useful in helping individuals understand where things exist in the pipeline. They can provide a quick, at-a-glance take that leaders can use to plan their projects, discover backlogs or potential roadblocks to deployment, and more.
But even dashboards are not necessarily enough, because stakeholders need to be able to quickly and securely communicate status updates to each other.
The problem is that not all of these individuals will necessarily be using the same tools. Some might be using one messaging service. Others might be holding impromptu video meetings. Others might be posting updates to project management boards. Software development is a living and agile process, and communication is happening all the time through different channels.
That’s why it’s critical for agencies to pull all of these disparate threads together, especially if they are to properly manage security and maintain compliance with the EO. A single missed communication about a software vulnerability could end up becoming tomorrow’s SUNBURST attack. There must be a way for all DevSecOps stakeholders to interact with each other, regardless of their choice of tools or the channels they’re using.
This is where the choice of modern toolsets and platforms comes into play. Pulling disparate threads together to facilitate better communication and collaboration requires technologies with the ability to connect various services, automate task management and alerts, and break down any lingering communication silos. It requires an underlying architecture that connects all key stakeholders so they can collaborate and ensure the security of the applications they’re working on.
Older technologies and practices aren’t up to the moment government agencies are facing. They are unsuitable for today’s software development processes, and will not aid in the quest to stay ahead of today’s cyber threats.
To meet the moment, government agencies must embrace new ways of communicating and collaborating. They must use the tools that are available to keep innovating even while standardizing according to cybersecurity regulations. They must provide their DevSecOps teams with the ability to work freely yet securely and set them up for success.

A survey of 500 DevSecOps professionals in the U.S. found nearly three-quarters (73%) of organizations plan to increase investment in application security in 2023.
The survey, conducted by Wakefield Research on behalf of Invicti, a provider of dynamic application security testing (DAST) tools, also found 97% of DevSecOps teams said they ignored a real vulnerability at least once a month because they assumed it was a false positive.
Invicti CTO and head of security research Frank Catucci said that while it is apparent organizations are under pressure to improve the security of their software supply chains, the tools and processes used to discover vulnerabilities need to be revisited at a time when the security technical debt accrued in production environments is already massive.
Many DevOps teams have routinely shipped code with known vulnerabilities for years, noted Catucci. They are not only under pressure to improve the security of new applications but also remediate applications with known vulnerabilities already running in production environments, he added.
Cybersecurity teams have, of course, been identifying those vulnerabilities for years. The challenge is that the long list of vulnerabilities cybersecurity teams provide to DevSecOps teams tends to lack any context. As such, it becomes impossible for developers to prioritize their remediation efforts based on the actual severity of the vulnerabilities found. DAST tools help address that issue because they provide insights into vulnerabilities in applications already running in a production environment.
Application security, of course, has historically been problematic because many developers assume cybersecurity teams are securing the environments where applications are deployed. Most cybersecurity teams, however, allocate their budgets to tools and platforms they control and manage. Most of those tools and platforms are focused on security operations tasks that tend to have more to do with securing infrastructure rather than applications. The assumption is application development teams are responsible for securing the software that runs on those platforms. That disconnect is now finally being addressed as responsibility for application security shifts left to become an integral part of DevOps workflows.
The Invicti survey suggested that application security budget allocations are now catching up with that transition. In fact, the Invicti survey found that 100% of DevSecOps professionals are tracking the return on investment (ROI) on the application security tools and platforms they employ. Of course, measuring the effectiveness of any cybersecurity tool or platform has always been a challenge. It’s never quite clear whether a breach didn’t occur because a tool or platform was present or if it didn’t happen because of sheer luck and happenstance.
Regardless of the ROI achieved, however, it’s clear that cybercriminals are increasingly targeting software supply chains. The recent executive order issued by the Biden administration requiring federal agencies to review the security of their software supply chains got the attention of traditional enterprise IT organizations, as well. As such, the overall state of application security should steadily improve in the months and years ahead, as more funding is allocated to prevent vulnerabilities from finding their way into production environments in the first place.

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: The OpenSSL project has egg on its face, and half of Twitter’s staff are for the chop tomorrow. (more…)

Tanium this week added the ability to detect libraries and software packages with known vulnerabilities within a software bill of materials (SBOM) manifest that can then be used to automate remediation of endpoints running vulnerable code.
Pete Constantine, senior vice president of product management for Tanium, said the Tanium Software Bill of Materials (SBOM) module provides IT teams with both the ability to discover where vulnerable code is running in real-time and automatically apply any required software patch.
The Tanium SBOM tool examines the contents of individual files wherever they reside in IT environments to make it simpler to immediately discover where, for example, vulnerable instances of the Log4j log management software or OpenSSL software for securing communications are running. Today many IT teams are spending weeks trying to track down every potential instance of code each time a new vulnerability is discovered.
Once those vulnerabilities are discovered, IT teams can then opt to use a Tanium Patch tool to remediate vulnerabilities or use other Tanium management tools to kill specific processes or uninstall applications, noted Constantine.

Awareness of the need for SBOMs to secure software supply chains has increased sharply in the wake of a series of high-profile cybersecurity breaches. However, few organizations have the tools in place to operationalize the data collected in SBOMs that are essentially little more than a list of components employed to build an application. The goal is to enable IT teams to decide whether to go ahead and deploy an application based on the level of threat the vulnerabilities included represent.
It’s unlikely that vulnerabilities will ever be completely eliminated from applications running in production environments, but it’s clear there is a need to be able to quickly identify issues based on the level of severity any vulnerability represents. The trouble is that an SBOM is only the first step toward achieving the level of remediation automation that is required to address vulnerabilities discovered in applications after they have been deployed.
Hopefully, all the focus on securing software supply chains will lead to a narrowing of the divide that has historically existed between cybersecurity and application development teams. Developers still need to create patches to fix issues, but overall the number of vulnerabilities that find their way into production environments should decline as more organizations embrace DevSecOps best practices before applications are deployed.
In the meantime, however, given how much developers today reuse code, the odds are good multiple vulnerabilities have already been widely propagated across an application environment. In fact, the amount of technical security debt that organizations face today is simply staggering when considering the total number of potential vulnerabilities that need to be remediated. In the absence of IT automation capabilities, it might take the better part of a decade to resolve all those vulnerabilities.
On the plus side, however, the tools and platforms required to automate remediations are now integrated with SBOMs that, at the very least, make pinpointing where vulnerabilities are located easier. The challenge now is defining a set of end-to-end processes that make fixing all those vulnerabilities a much less arduous task for all concerned.