devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

The DevOps Journey: Continuous Mindset Starts With Cultural Change
DevOps is not just about technology. In many ways, the technology aspect of DevOps is finite but the people, organizational and cultural aspects are infinite. We see many clients get so focused on the tools and technology that they forget about the impact beyond the technology. An important part of building a world-class DevOps process is understanding the cultural and organizational changes that are required to be successful and making those a priority, as well.

Many organizations have still either not started their DevOps journey or are in the first generation of DevOps, which is creating a greater gap between elite and low performers. The business changes and delivery cycles are coming faster than ever, making it critical to use world-class DevOps. Pushing toward continuous flow requires technical, process, people and cultural evolution to keep pace. To achieve continuous flow, we need a continuous mindset. A continuous mindset requires new skills, new ways of thinking, new cultural norms and new responsibilities.

Continuous Mindset, Continuous Flow

Most companies don’t realize the cultural impact a fully automated pipeline and continuous flow will have on development and operations (and beyond). Change can be difficult and organizations are often met with FUD: Fear (automation brings fear that jobs will be automated away), uncertainty (the unknown creates anxiety) and doubt (needing to employ new skills and new ways of thinking can create doubt about whether resources can adjust).

The rapid pace of business change requires more nimble, flexible and lean approaches to DevOps to meet tomorrow’s demands. This requires changes to structures, interactions, control, processes and expectations. All of these impact the culture and connections within the organization.

Let’s start by defining a few terms:

Continuous integration: The practice of merging all developers’ working copies to a shared mainline several times a day. Grady Booch first proposed the term in 1991, although he did not advocate integrating several times a day. Extreme programming (XP) adopted the concept of CI and did advocate integrating more than once per day—perhaps as many as 10 or more times per day. Common continuous integration practices include maintaining a code repository, automating the build, making the build self-testing, testing in a clone of the production environment and more.

Continuous Delivery: The official definition is, “A software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time and, when releasing the software, doing so manually. It aims at building, testing and releasing software with greater speed and frequency. The approach helps reduce the cost, time and risk of delivering changes by allowing for more incremental updates to applications in production. A straightforward and repeatable deployment process is important for continuous delivery.”

Continuous Deployment: A strategy for software releases wherein any code commit that passes the automated testing phase is automatically released into the production environment, making changes that are visible to the software’s users. The logical next step after continuous delivery: Automatically deploy the product into production whenever it passes required validations.

Continuous mindset is intertwined throughout all three of the above approaches. To reach the maximum value in each area, a continuous mindset needs to be enabled, matured and applied to disciplines in other DevOps areas. The changes are across people, processes, technology and culture. Like DevOps, gaining a continuous mindset is a journey that will take time, many steps and constant learning.
Continuous mindset changes the way people achieve outcomes for your business. This requires an emphasis on changing the way people think at every level of the organization, the way people are connected and the way they work. Achieving true continuous mindset ROI requires significant cultural adoption and collaboration. A key component is leveraging strong organizational change management (OCM). Six OCM steps to achieving the cultural shift to a continuous mindset include:

Change management and readiness: Share with internal stakeholders where we are going and how we are going to get there. Set expectations.

Sponsorship: Create clear alignment and engagement at the executive level.

Stakeholder engagement: Get everyone on board. Identify who is impacted as well as who is responsible and accountable. Understand reactions and resistance and provide support.

Communication: Get the right information to the right people at the right time. Move stakeholders from awareness to buy-in, commitment and readiness. Communicate messages to multiple levels of the organization.

Capability development: Provide the education and training needed to be successful in the new culture.

Organizational alignment: Align roles, structure, processes, policies and systems with changes and desired business outcomes.
Shifting Responsibilities

Ultimately, a continuous mindset changes the organizational engagement model and requires organizations to align on how and when they will work together. This requires shifting responsibilities and empowering teams/individuals to contribute to the improvement of DevOps capabilities. Learning and knowledge transition is an important element to enabling the team long-term.

The organization will need coaching as they begin to understand the concepts and adjust the way they work. Different approaches can be used depending on the overall needs. This purposeful DevOps learning approach will provide DevOps coaches to engage with teams along with DevOps subject matter experts to provide input in the preparation activities. Another option is to provide less formal DevOps coaching to engage with targeted teams. This approach is less structured and primarily relies on direct engagement with less focus on the training and scaling project approach.

Transparency is Fundamental to a Continuous Mindset

Part of creating a continuous mindset involves an environment where transparency is fundamental; where everyone feels committed to the results takes ownership and responsibility for improving them. Successful DevOps transformation requires a holistic approach that includes people, processes, technology and culture. Working in cross-functional agile teams; working with small batches; working in sprints; communicating and collaborating all provide the strong foundation necessary to create a solid continuous mindset. If done correctly, breaking cultural changes down into smaller changes can ensure a certain level of quality in the long term.
February 14, 2022
Unreliable Server Scare | Information Batteries | ARM IPO PDQ

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: We worry about chips failing randomly, we ponder a new way of thinking about workload shifting, and we grok Arm’s IPO.

1. Smaller Features Lead to Random Failures

First up this week: Could cosmic rays be causing increasing numbers of errors in cloud servers? As chips get faster and more densely packed, the worry is that undetected bit-flips could cause serious problems.

Analysis: How can you mitigate the problem?

As workloads grow in number and complexity, this is becoming a real issue—not merely theoretical. (See also: DevSecOps worries such as Rowhammer.) Think about where the critical data are and how to detect random errors. Even consider running important workflows in lockstep pairs.

John Markoff: Tiny Chips, Big Headaches

Imagine for a moment that the millions of computer chips inside … the largest data centers in the world had rare, almost undetectable flaws. … As the tiny switches in computer chips have shrunk to the width of a few atoms, the reliability of chips has become another worry for the people who run the biggest networks. … There is growing evidence that the problem is worsening with each new generation of chips.
…
Both Facebook and Google have published studies describing computer hardware failures whose causes have not been easy to identify. … Facebook researchers noted that some processors would pass manufacturers’ tests but then began exhibiting failures when they were in the field. … Smaller transistors, three-dimensional chips and new designs that create errors only in certain cases all contributed to the problem … according to Google.
…
Intel executives said they were familiar with the Google and Facebook research papers and were working with both companies to develop new methods for detecting and correcting hardware errors. [But] computer engineers are divided over how to respond to the challenge.

Have we hit a wall? So asks Thutmose V:

For more than 50 years, semiconductor manufacturers have been trying to keep up with Moore’s Law, which … has effectively served as the target for semiconductor engineers to push them forward. … The size of some parts of the transistors is now 10 nanometers or less. 10 nanometers is about equal to the width of 100 silicon atoms—100 atoms is not very many.
…
When things are that small, does quantum randomness start to be noticeable? Normally if the odds of something failing is a trillion to one, you don’t need to worry. But if you have 10 billion transistors operating at a clock speed of 3 billion times a second, a one in a trillion chance of failure is nowhere near good enough. … It is a miracle these things work as well as they do.

We’ve known about the problem for years, but only now is it becoming reality, says dsgrntlxmply:

We observed certain network failures to be caused by corruption of an on-chip routing table memory in a SoC that lacked parity protection for these tables. The failure rate was low, but was higher in units that were deployed at higher altitudes. … The problem was eventually mitigated by software table sweeps, and in newer SoC generations, by parity in critical tables.

These Single Event Upsets are said to be caused by muons generated by high energy cosmic ray interactions in the upper atmosphere. Third semester undergrad physics.

2. Mind-Bending Speculative-Compute Research

Many of us are familiar with delaying a workload until a time when servers are idle—or when energy is cheap. But some fascinating research asks: What if we could predict a future workload and perform it in advance?

Analysis: “Information batteries” can be cheaper than Li-ion

The researchers argue that pre-caching even parts of a workload’s output when energy costs are low—or even negative—can save far more money than traditional grid-scale storage such as batteries or pumped hydro. For example, speculatively transcoding video versus generating it on the fly, or predicting future machine learning training workloads.

Caitlin Dawson: A New Way to Store Sustainable Energy

Renewable energy has an intermittency problem. … What if surplus renewable energy could be stored as computation? … That’s the thinking behind “information batteries,” a new system proposed by … Barath Raghavan, an assistant professor in computer science at the USC Viterbi School of Engineering … and Jennifer Switzer, a Ph.D. student from UC San Diego.
…
When renewable energy is available in excess, it is used to speculatively perform computations. … The stored computed results can then be used later when green energy is less plentiful. … For instance, every day, YouTube data centers transcode more than 700,000 hours of videos to different resolutions. Many of these computations are predictable and can be performed at a time when there is excess green energy.
…
[But] computations that are completed in advance do not need to match exactly with the computations completed at a later time. … The challenge, said the researchers, is determining what computation to perform, where and when, and how … to efficiently retrieve the results later. … It is only possible in some workloads and in some contexts. But Raghavan believes with improved prediction and integration into large systems, the technology points towards a promising future alternative.

This isn’t just batch computing with a fancy name. cognizant_ape explains:

On AWS we do a lot of batch processing when the data centers are low on traffic. This is just the standard SPOT instances which you can trigger once the cost goes below a certain amount and then end when it goes about a certain amount.

The big issue is being effective with I/O and setup time. The article seems to be getting ambitious to the point of a distributed compiler with a flexible optimization metric.

Cut to the chase. It hinges on prediction, says bradley13:

Predicting loads is easy, but predicting the computations that will make up those loads? Not so much. … Imagine: Most employees start work in an hour — let’s predict which database queries they will generate. Your success rate better be high.

3. Confirmed: Arm will IPO, Not be Sold to Nvidia

SoftBank and Nvidia have bowed to the inevitable. Arm, designers of ubiquitous chip cores, will IPO sometime in the next 12 months.

Analysis: The only sane choice, but ecosystem risk remains

As I’ve said before, it seemed everyone was against the idea of Nvidia owning Arm. It put the kibosh on Arm’s open licensing stance, dooming the ecosystem to scraps from the table as Nvidia grabbed first dibs on everything. Now regulators must turn their attention to the very real risk that Nvidia will still acquire Arm anyway—via the stock market.

Richard Waters, Arash Massoudi, James Fontanella-Khan and Antoni Slodkowski: $66bn sale of chip group Arm to Nvidia collapses

The deal, which would have been the largest ever in the chip sector, was set to give … Nvidia control of a company that makes technology at the heart of most of the world’s mobile devices. A handful of Big Tech companies that rely on Arm’s chip designs, including Qualcomm and Microsoft, had objected to the purchase [arguing] that Nvidia would get an unfair advantage by having first rights to Arm’s technology.
…
Jensen Huang, Nvidia’s chief executive, hoped to use Arm’s processor designs to cement his company’s growing role in data centres. … Nvidia abandoned its pursuit of Arm at a board meeting on Monday, said a person familiar with the discussion.
…
The deal’s failure brought an immediate management shake-up at Arm, with chief executive Simon Segars replaced by Rene Haas, head of the company’s intellectual property unit. … In the UK, where politicians view Arm as a strategic national asset, attention is set to shift to whether the company will be listed in London [but] Haas said … no decisions had been made on where Arm would be listed.

Reminding us why this matters to DevOps, here’s Timothy Prickett Morgan:

Now there is no question that Arm Holdings will be going public. … The plan is to do so before the end of [its] fiscal year in March 2023. … Arm Holdings will be returning to its IP licensing roots, having spun off some uncore business back into SoftBank, and seeking out new markets as it has been able to do with hyperscalers [and] cloud builders.
…
When Intel was worried about AMD Epyc chips, what it really needed to worry about was boosting price/performance to keep Arm CPUs out of the hyperscaler and cloud builder datacenters. [But AWS] has Graviton Arm server chips on a pretty regular schedule, and you can bet Masayoshi’s last yen that Graviton4 will debut at re:Invent 2022 this year.

But what happens next? Pun it up, HereAndGone:

If Softbank is going to take ARM public, Nvidia will find it much cheaper simply to acquire a controlling interest. … Looks like a duck, walks like a duck, but cheaper than a duck—while ducking regulatory issues. This plan is not quacked up to solve what it pretends to solve and will likely prove most fowl.

The Moral of the Story: Fall seven times and stand up the eighth

You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

Image: Nicole Geri (via Unsplash; leveled and cropped)

February 10, 2022
The Evolution of Incident Management
Have you ever thought about the history of incident management?

If you’re an SRE, you might be so caught up in the day-to-day work of managing reliability and responding to incidents that you never take time to step back and reflect on the evolution of your role and your responsibilities. And that’s a shame because SREs didn’t invent incident management concepts and strategies on their own.

On the contrary, the way SREs think about incident response, structure incident management teams and rank the priority of incidents owes much to incident management strategies developed in the offline world decades ago. To understand fully what it means to be an SRE today, you have to appreciate this deep history of incident management and response.

So, let’s take a look at that history and examine where modern incident response concepts originated.

Historical Problems in Incident Management

Societies have always had incidents, of course. Fires, floods, infrastructure breakdowns and similar crises have been happening for millennia.

For most of history, however, humans lacked an efficient, purposeful way to manage these sorts of incidents. Response efforts were ad hoc, and their effectiveness owed more than a little of their success to sheer luck.

Particular challenges included:
- Lack of effective, consistent communication between stakeholders.
- Varying organizational structures which made it hard to identify leaders, coordinate response efforts and delegate tasks.
- Inconsistent response strategies.
- Differing approaches to assessing the priority of incidents.
Historically, organizations may have been able to handle incidents well enough if the incidents required response from only one, small group. But the more stakeholders involved, the harder it was to respond quickly and effectively.

Putting Out Fires: The Birth of the ICS

Things began to change for the better when stakeholders started thinking about better ways to put out fires—literally.

By the 1960s, fire chiefs in California realized that they were struggling to respond effectively to the wildfires that broke out every summer. Each year brought worse blazes than the last, with more land burned and more buildings lost. The Laguna fire of 1970 brought matters to a head and was the catalyst for a new approach to incident response for fire agencies.

After assessing what was going wrong, the fire chiefs determined that it wasn’t a lack of equipment or personnel. It was poor coordination among the various firefighting agencies that responded to blazes. Lacking a clear chain of command and a systematic approach to firefighting, the agencies struggled to deploy their resources rapidly and, thus, effectively.

To fix the problem, California fire chiefs developed what became known as the Incident Command System, or ICS. The ICS defined a hierarchy for incident response with an incident commander at the top. It also defined several categories of incident response processes including operations, planning, logistics and finance. And it established a consistent set of terms that stakeholders can use to describe their actions during incident response, which makes it easier to communicate clearly.

Although the ICS was initially conceived to fight fires, it became the de facto standard for organizing incident response strategies of all types.

From ICS to NIMS

The history of incident response doesn’t end with the ICS. A new chapter began in the early 2000s when the U.S. federal government developed an even more comprehensive approach to incident management called the National Incident Management System, or NIMS.

NIMS was born in the wake of the September 11, 2001, terrorist attacks, which underlined the importance of efficient communication not just between different agencies of the same type (like fire departments), but of entirely separate organizations. To achieve this, NIMS expanded upon the principles of the ICS.

In addition to adopting most of the incident command principles and practices included in the ICS, NIMS included standards for coordinating the distribution of resources. It also embraced the concept of the emergency operations center, which is in some ways similar to a network operations center in the digital world.

In some respects, NIMS resembled a compliance framework (although to be clear, that’s not what it is). It includes fourteen management principles, similar to compliance controls, that organizations must implement to manage incidents using a NIMS approach.

Incident Management Today

Obviously, putting out forest fires and responding to terrorist attacks is pretty different from dealing with data center failures or a buggy application deployment. ICS and NIMS weren’t designed for site reliability engineering or IT teams specifically.

Still, the influence of ICS and NIMS on the way SREs think is clear enough. Terminology like “incident commander” comes from these frameworks. So do concepts like shared accountability for incident response processes and the importance of involving all stakeholders—not just technical teams—in incident response.

ICS and NIMS may not be acronyms familiar to most SREs. But they should be, because they are the historical sources of the incident management philosophies that form the foundation for SRE work today and offer valuable lessons for any SRE on the job today.
February 8, 2022
Codenotary Launches Cloud Service to Generate SBOMs

Codenotary has launched a Codenotary Cloud platform that can automatically generate a software bill of materials (SBOM) and make it easier to discover what components have been included in an application.

Moshe Bar, Codenotary CEO, said that capability can also play a key role in identifying which components in an application might contain, for example, the Log4jShell vulnerabilities that were recently discovered in Java applications.

That capability can reduce the time required to remove insecure artifacts by as much as 80% simply because organizations are not dependent on scanning tools to search for vulnerable components, added Bar.

Codenotary Cloud is based on an immutable open source immudb database that cryptographically attaches an identity to each software artifact. In the wake of a series of high-profile breaches, there’s now a lot more focus on the integrity of software supply chains. In fact, the Biden administration recently issued an executive order requiring all federal agencies to, among other things, make sure an SBOM is available for all the software they use.

Codenotary claimed Codenotary Cloud can scale to millions of integrity verifications per second across source code, builds, repositories, Docker container images and Kubernetes deployments in a way that doesn’t require DevOps teams to upload data to a centralized platform, noted Bar. That makes it possible to notarize and verify the provenance of all the code being employed, he added.

Codenotary Cloud can also be fully integrated with most vulnerability scanners and popular cloud-native continuous integration/continuous delivery (CI/CD) systems running on any cloud or on-premises host via a managed service. Pricing starts at $5,500 for a workgroup of 10 developers.

It’s not clear to what degree the Biden administration’s executive order will improve overall software security. There have been multiple vulnerability issues over the years that have not resulted in any material change in how organizations manage their application development processes. However, the executive order issued by the Biden administration, coupled with a recent summit on the state of open source software security, means more senior business and IT executives are at least aware of growing concerns.

The one thing that is certain is there will be more zero-day vulnerabilities discovered. Developers today routinely reuse open source software and free components maintained by a small number of volunteer programmers. Like any other developer, the amount of security expertise those individuals have is limited. Many organizations reuse that code without contributing anything meaningful back to the project—either in terms of financing or vulnerability identification and remediation support. Contributors to these projects freely donate their time and expertise to build these components. Many of those contributors feel the onus for securing the software they created is on the organizations that decide to deploy that software.

Regardless of who is responsible for security, it’s apparent cybersecurity teams are going to be reviewing application development processes. Theoretically, that should benefit all concerned and lead to DevSecOps best practices being more widely adopted.

February 7, 2022
Trends in the API Industry

The application programming interface (API) economy is chugging along full steam ahead. The API management market alone is expected to expand 35% by 2025, supported by the sheer number of web APIs coming to market. APIs have become ubiquitous across microservices architectures, public product initiatives, SaaS platform offerings, IoT and partner-partner integrations.

The industry has proliferated and continues to change, introducing new API use cases across all industries. This ubiquity creates a heightened need for an API program to maintain a quality developer experience and remain competitive. There are also different styles and specification options on the market, meaning designing the right API strategy is a moving target. And, amid peaking cyberattacks, developers now have a strong imperative to retain high security and reliability for these integration points.

I recently met with the CEO at Stoplight, Steve Rodda, to discuss the above trends and other forces pushing the API industry forward in 2022. Below, we’ll dive into these trends to see how they affect API strategies and what developers need to know to stay on top of their applications.

Non-Software Companies Embrace APIs

As software eats the world, more companies are becoming software companies. And, an API strategy is part and parcel of this digital transformation. A big trend that Rodda recognizes is increased API strategies among traditionally non-software companies.

For example, Rodda explained how a large beverage manufacturer evolved to adopt APIs company-wide to better use and scale their data. Standardizing the API development and design process helped avoid the “rat’s nest” of custom code, said Rodda. “APIs are no longer a byproduct; they’re a design artifact,” he said.

APIs are no longer reserved for startup cloud-native unicorns like Stripe or Twilio, either. We now see many industries and sectors like supply chain management, health care, shipping, built environments and mainframe modernization adopting API approaches. The financial services sector is especially keen to open up APIs as open banking is being driven by regional regulation or market pressures.

A Rise in Internal API-First Adoption

ProgrammableWeb, the most comprehensive API directory, lists over 24,000 APIs at the time of writing. There’s no doubt that public-facing APIs are very well documented. But this is only a sliver of total API adoption.

The 2021 Postman State of API report found that only 15% of APIs are publicly available. Most APIs are either partner-facing (27%) or private (58%), according to the report. In fact, the most popular use case for APIs is integration between internal applications, programs or systems. Along with the growth of internal API use, API-first thinking is gaining mind share. Postman defined API-first as “defining and designing APIs and underlying schema before developing dependent APIs, applications or integrations.”

Internal use cases are less frequently discussed but are pervasive for large enterprises, said Rodda. Whereas in the past, development teams paid less attention to style guides and documentation for internal services, he now notices that these practices are shifting. The zero-trust approach is influencing teams to take more care in defining and standardizing internal APIs as if they’re external-facing. This also eases consumption. “You need to take care of the folks next door to you and treat them with just as much respect,” said Rodda.

The Growing Need for Robust API Security

APIs have a security problem. Many don’t handle authorization correctly and expose far too much information. A hacker can often escalate their privileges or simply swap a different user ID into an API call to return loads of sensitive data or even make changes to the data. Many HTTP API endpoints that are intended to be private really aren’t. As a result, OWASP lists broken object-level authorization as a top API vulnerability.

In recent years, we’ve seen what can happen when APIs are improperly configured. For example, a Facebook API lacked an authorization check for unlisted posts, allowing a client to post on any user’s behalf. A similar authorization hiccup occurred with Peleton that allowed hackers to alter any user’s back-end account. APIs constantly work with sensitive data and must carefully navigate the various data privacy regulations around the globe.

According to Rodda, a design-first mindset is vital to ward off security woes early in the API life cycle. This means deciding on a structure with a common design and choosing appropriate style guides well before coding. “Good documentation, good design guides and good discipline around building something with a cross-functional nature will increase security and scalability,” said Rodda. When you know what’s going on, you have more control and things are inherently more secure, he added.

OpenAPI Carries The Day

The OpenAPI specification, formerly known as Swagger, is an industry-standard specification for describing REST web APIs. OpenAPI can help direct your API’s design and development. OpenAPI tooling can also help generate useful documentation, sandboxes and SDKs, making it a valuable specification for creating interoperability between partners.

“OpenAPI is going to carry the day for a while,” said Rodda. The OpenAPI Initiative has gotten the support of big names in the tech industry. Yet, Rodda estimated only 40% of companies have adopted OpenAPI in production. Rodda said he believes tooling will make it easier for more groups to maintain OpenAPI adoption across their organization, reducing the redundancy of writing “yet another YAML file.”

REST is still by far the most-adopted API design style—59.7% of developers use REST for production APIs, according to RapidAPI’s State of APIs Developer Survey 2021 report. Yet, asynchronous event-driven communication protocols are gaining traction, and those protocols tend to prefer alternative description formats like AsyncAPI. Websockets, gRPC and GraphQL are other options consistently used by a significant percentage of API developers.

Developer Experience Matches User Experience

Another major trend is the rising importance of developer experience (DX). DX is akin to user experience but is all about increasing usability for developer consumers and improving their ongoing relationship with software-as-a-service. “DX is as big a deal as UI/UX these days, if not bigger,” explained Rodda. “Developer experience directly translates into efficiency and reusability.”

In the context of APIs, increased consideration of developer experience means reducing the onboarding effort and maintaining more reliable connections. For example, users will likely look to other solutions if a third-party API has poor uptime and routinely introduces a breaking change. Better DX likely will also equate to increased abstraction layers and more code generation. Rodda pointed to GitHub’s CoPilot as an example of the use of AI and automation in the developer-of-tomorrow’s workflow.

“We’ve evolved as a developer society,” said Rodda. Just as consumers expect high-quality real-time applications, developers expect highly performant APIs. To help get there, one increasingly popular philosophy is the API-as-a-product perspective. “If you treat it all as a product, you’re building it with the same standards and quality that you would share externally,” he explains. “Then, there’s no attention to detail that goes missing.”

Final Thoughts: The Design-First Mentality

More industries are utilizing APIs than ever before, and APIs are seeping into traditionally non-technical companies to aid their internal integration efforts. But data privacy is a pressing concern when opening up data, requiring careful measures. Taking a design-first approach with standards like OpenAPI could help organizations stay alert and ensure consistency across their portfolio of services.

There are other trends at work in the API economy, too. For example, one interesting Web3 initiative is an attempt to combine the world of blockchain and APIs by enabling on-chain smart contracts to interact with traditional HTTP APIs. There is also a lot of development activity around GraphQL, which could act as a command line for consuming multiple APIs, thus making integration easier.

Rodda said the biggest takeaway is that APIs should no longer be treated as byproducts. He compared it to a construction project: When building a house, you must start with a blueprint before installing the foundation, wall, roof or electrical system. Similarly, APIs require solid forethought to ensure stability and scalability.

February 3, 2022
App Store Antitrust Bill | GDPR vs. Google Fonts | Wordle Worth $1M+

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: The Open App Markets Act polls well among devs, Germany fines a website for using Google Fonts, and the New York Times buys Wordle for an unfeasible sum.

1. Antitrust Bill Would Open Up App Stores

First up this week: The Open App Markets Act seems like a big hit with developers. In a survey, more then 80% wanted lawmakers to pass this bipartisan Senate bill.

Analysis: But 30% of nothing is nothing

One of devs’ biggest complaints is Apple’s 30% cut of all revenue—including in-app purchases. Many devs have also experienced random rejections of submissions to the stores. And these seem to be the main driving forces behind devs’ calls to tear down any walls around the gardens. But they should be careful what they wish for—users might not see it the same way.

Karl Evers-Hillstrom: 8 in 10 app developers back measure to rein in Google, Apple

The Coalition for App Fairness, [an] industry group for app developers, is pushing Congress to pass the Open App Markets Act. [The] bipartisan Senate bill … would block app stores from favoring their own in-house apps in searches, requiring developers to use their payment systems and preventing users from downloading apps from third-party stores.
…
Developers surveyed … complained about exorbitant fees … and expressed how they’d experienced difficulty getting their apps featured or accepted by app stores. Just 13 percent of app developers surveyed oppose the bill. … The margin of error is plus or minus 7.11 percentage points.
…
The proposal, co-sponsored by Sens. Amy Klobuchar (D-Minn.), Richard Blumenthal (D-Conn.) and Marsha Blackburn (R-Tenn.), is scheduled for a committee markup this week. The CEOs of Apple and Google have personally lobbied lawmakers to abandon [such] antitrust legislation.

Wouldn’t that just kill the proverbial golden goose? No, says this Anonymous Coward:

The only reason Apple has any success with the iPhone is because developers write apps for it, it’s no different to Windows in that regard. … If Apple’s app store is so good then everybody from developers to end users will continue to use it and nobody will use third party app stores.
…
Obviously there is a clear profit motive to exclude any competition in the app store space. But I don’t understand how people can claim that Apple makes great products and simultaneously argue that they couldn’t possibly survive competition on the platform.

But Dr. Edith Müdyk—@BabyHueby—doesn’t sound hopeful this will go anywhere:

Too bad policy isn’t dictated by public opinion.

2. German Court Fines Site for Using Google Fonts Without Consent

Did you know you need to get permission from a European user before you remote-load a Google Font? You do now.

Analysis: Schrems II comes home to roost

More GDPR popups ahoy! A German court has “clarified” that websites and apps should obtain informed consent before loading content—if loading it would reveal a user’s IP address to a third party. Luckily, the fine was only $110—this time, but if the perp was to do it again, the court threatened 2500× that amount.

Thomas Claburn: Website fined by German court for leaking visitor’s IP address via Google Fonts

A German court fined an unidentified website €100 ($110) … for violating EU privacy law by importing a Google-hosted web font. [It] passed the unidentified plaintiff’s IP address to Google without authorization and without a legitimate reason for doing so. And that violates … GDPR.
…
The website could have avoided this drama by self-hosting the font. … The ruling explicitly cites this possibility to assert that relying on Google-hosted Google Fonts is not defensible under the law.
…
The ruling directs the website to stop providing IP addresses to Google and threatens … a fine of €250,000 for each violation, or up to six months in prison, for continued improper use of Google Fonts. … These data privacy judgments … reflect the consequences of the EU Court of Justice’s 2020 decision to strike down the Privacy Shield data protection arrangements that had previously allowed US companies to exchange data with European partners under “Standard Contract Clauses.”

Isn’t that ridiculous? And surely not the intention of GDPR? Aristos Mazer very much disagrees:

This is very much the intended consequence, with a lot of analysis going into this regulation. … Even if that means redesigning foundations of the network.
…
Forcing sites to stop transferring data in any way shape or form to Google, Apple, Microsoft, Facebook, etc. … is exactly the goal. [It’s been] repeatedly validated by polls of EU citizens.

And indeed, the point is to get “informed consent.” Here’s Hans Neeson-Bumpsadese:

“To proceed beyond this point … your IP address needs to be shared with a 3rd party.” Would they click on that? … I know at least one person who would read that as, “If I click that link then you will tell people where I live.” (I know because they got a pop-up like that once, and phoned me in a state of mild agitation.)

3. Wordle Sold to NYT for Over $1M

Josh Wardle, a Welsh developer living in New York, has sold this winter’s social gaming hit to his state’s namesake newspaper of record. The New York Times thinks it’ll be a useful addition to its Games section. But the price tag is incredible.

Analysis: Nice payday for Josh

How much? Hard to see how the Times is going to make money out of this. Tomorrow’s word must be: NOROI.

Alexis Benveniste: The Sudden Rise of Wordle

In less than six months, Wordle has gone from a gift to a viral sensation, to … part of The New York Times. … Josh Wardle, a software engineer in Brooklyn, created the game as a gift for his partner … and it exploded in popularity.
…
Nearly two months later, 300,000 people played it. … The buzz around the game can be attributed to the spoiler-free scoring grid that allows players to share their Wordle wins across social media.
…
Wordle was acquired for an undisclosed price in the low seven figures.

In other words, north of $1 million. Not bad for a quick, one-dev hack by Josh Wardle:

The game has gotten bigger than I ever imagined (which I suppose isn’t that much of a feat, given I made the game for an audience of one). … I feel so grateful for the personal stories some of you have shared with me.
…
[But] I’d be lying if I said this hasn’t been a little overwhelming. [So] I am incredibly pleased to announce that I’ve reached an agreement with The New York Times. … I’m thrilled they will be stewards of the game.

But “RIP Wordle,” mourns _xeno_, seemingly speaking for many:

It’s about to be bogged down with trackers, and then later, ads. … A paywall is almost certainly on its way—The New York Times very carefully said it would “initially remain free.” … Not surprisingly, there are already plenty of clones. … There’s no way for the New York Times to make money off this.

The Moral of the Story: No disguise will hide one’s true character.

You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

Image: Masahiro Miyagi (via Unsplash; leveled and cropped)

February 1, 2022
Linux Foundation Survey Sees Rise in SBOM Use

A global survey of 412 organizations published today by the Linux Foundation found nearly half of respondents (47%) producing or consuming software bills of material (SBOMs), and more than three-quarters said they expected to produce or consume them in 2022.

SBOMs have become a bigger area of focus in the wake of a series of high-profile software supply chain-related breaches. The breaches involved application components compromised by malware that had been surreptitiously inserted during the application development life cycle. The Biden administration has gone so far as to issue an executive order requiring all agencies to provide SBOMs for all the commercial and custom software they use, but as of yet a deadline for compliance with that order hasn’t been set.

In general, the Linux Foundation survey found that a large percentage of respondents (82%) are already familiar with the term SBOM. Respondents cited various benefits of having an SBOM, including making it easier for developers to understand dependencies across components in an application (51%), making it easier to monitor components for vulnerabilities (49%) and making it easier to manage license compliance (44%).

However, the survey also makes it clear there is much SBOM work to be done. A full 62% of respondents, for example, are looking for better industry consensus on how to integrate the production/consumption of SBOMs into their DevOps practices. Respondents also cited a need for consensus on the integration of SBOMs into their risk and compliance processes (58%).

More than half (53%) also said there is a need for better industry consensus concerning how SBOMs will evolve and improve.

The survey also makes it apparent that organizations are more aware of the role SBOMs play in securing supply chains. A full 80% of organizations are aware of the White House’s executive order on improving cybersecurity, with more than three quarters (76%) considering making changes as a direct consequence of the order. Overall, the survey found the top consideration organizations focus on when evaluating open source software is security.

Stephen Hendrick, vice president of research at the Linux Foundation, said 2022 is likely to be remembered as the year of the SBOM. There are several standards for building SBOMs, including an open source software package data exchange (SPDX) specification backed by The Linux Foundation and the Joint Development Foundation that is now recognized as the ISO/IEC 5962:2021 international standard. There is also CycloneDX, a lightweight SBOM standard based on an open source project that originated via the Open Web Application Security Project (OWASP) community and a software identification (SWID) standard for tags that some organizations use to create SBOMs.

SBOMs are, essentially, a tool for listing the components that make up an application and are used inconsistently. However, when a critical vulnerability is discovered, they can sharply reduce the amount of time required to discover all the software components that need to be remediated. The challenge is, with the rise of containers, it has become much simpler to rip and replace software components in a way that may not always be reflected in the SBOM. Hendrick said in time, advances in technologies such as machine learning algorithms should make it easier to instantly reflect those changes in an SBOM.

Regardless of approach, it’s apparent DevOps teams, like it or not, are about to be held more accountable for the provenance of every component employed across their entire application portfolio.

February 1, 2022
Continuous Security: The Next Evolution of CI/CD
Following a wave of high-profile cyberattacks and the White House’s release of the executive order on improving the nation’s cybersecurity, how to build a successful cybersecurity program has never been so hotly debated as it is now. From SolarWinds to the Colonial Pipeline attacks, these events demonstrate that defensive measures alone cannot mitigate cybersecurity risk, and any cybersecurity program must begin where nine out of 10 breaches begin—the code.

The adoption of CI/CD in recent years as an extension of Agile and DevOps has enabled organizations to embrace digital transformation and ensure code is always being delivered that matches the most current business needs. Unfortunately, this has further siloed security as processes failed to keep pace. While there are a number of point solutions to address the ever-expanding domain that encompasses application security—from traditional pentesting and vulnerability scanners to monitoring of the individual components of the DevOps toolchain like containers—too many tools results in a DevSecOps “hairball” that leaves organizations inundated with data, yet without actionable information.

Moving Beyond Secure DevOps

As organizations continue to evolve their delivery processes, security has found its role in secure DevOps (SecDevOps) programs to ensure the execution of security processes at every step of the software development life cycle (SLDC). By continually managing security practices, policies and debt in existing pipelines, this approach ensures that all stakeholders—from the C-suite to the developers and AppSec managers—have the information they need at every step of development to share responsibility in delivering secure software.

However, these programs, while a good starting point, have left security requirements and checks in PDFs, Excel documents, and emails failing to capture the same benefits of security integration into the SDLC that Dev and Ops have commanded. Without SecDevOps orchestration, the information often lags the pipeline, creating new bottlenecks and uncertainty. SecDevOps orchestration gives security and development teams the confidence to know they’re following the right policies and eliminates the manual work that creates drag on the SDLC. With the automation of the application security process, enterprises get consistency and insight into their program that allows their focus to stay on shipping code while knowing it meets their organizational security standards.

Adding Continuous Security

With CI/CD, organizations are able to rapidly meet the most current needs of the business as they rapidly evolve, but without a continuous approach to security as well, once again, security is siloed off. Organizations must be able to adapt their security requirements for changes both internal (changing databases, different software versions, etc.) and external (compliance requirements, changing threat landscape, etc). We need to take CI/CD one step further and also have security integrated continuously.

By extension, continuous security (CI/CD/CS) is the philosophy of continuously shipping software that meets the most current security standards for the business, accounting for internal and external change throughout the SDLC.

To effectively adopt a CI/CD/CS, organizations must focus on three key principles:
- There is no perfect code, therefore there is no perfect security—Good application security manages cyber risk in line with overall business strategy. When you have the ability to accept risk within the risk tolerance of the businesses, you know the right times to stop, and the right times to carry on because other controls are in place. Don’t let perfection be the enemy of the shipped.
- Bite-sized application security—DevOps isn’t about moving fast and breaking things, it’s about moving efficiently and fixing things at the right time. Deliver the right application security information at the right time to the stakeholder responsible for executing it. This ensures better collaboration so stakeholders know where, when and with whom to direct their attention.
- Micro-feedback loops—Leverage orchestration to get real-time feedback on policy compliance before it creates bottlenecks down the line. This enables you to respond to changes both internal and external as quickly as possible, creating an iterative cycle of information for future decision-making.
By continually managing security practices, policies, and debt in existing CI/CD pipelines, this approach allows a scalable application security program to be deployed in any SDLC so it can continually evolve to always be in lockstep with today’s business.

Different organizations have different risks they need to account for, which means security must be aligned to business strategies and priorities in order to dynamically respond. With end-to-end integration in the SDLC, continuous security supports CI/CD to improve productivity and speed time-to-market, while reducing the risks that might impact a particular business. Software is inherently impermanent and organizations need to be able to continuously balance security, technical and business priorities to ensure they are maintaining their focus on what matters most: Delivering value to customers and shareholders.
February 1, 2022
A Pandemic Side Effect: Rampant Technical Debt

On average, about a quarter of all IT budgets are allocated to handling technical debt each year. And this figure is set to rise, as 68% of organizations say their technical debt will increase in 2022. As digital acceleration continues to drive more IT investment across divisions, the technical burden almost seems like an inevitable outcome of delivering more software and amplifying end-user experiences.

Unsurprisingly, IT budgets are set to increase this year. This has been a stable trend throughout the pandemic as even companies that are not traditionally technical increased their reliance on software. Most teams quickly moved to remote working arrangements and acquired additional software tools to keep internal operations afloat. Not to mention the new complexities and array of tools emerging to support the microservices design paradigm.

Although new tech adoption is vital to enable key initiatives across a company, unbridled technical debt can also inhibit digital transformation initiatives. Software AG recently released its Situation Report 2022, a survey that analyzed the state of technical debt among IT professionals in 2021 and technology adoption forecasts for 2022. Below, I’ll review the key points from the report and consider how organizations can respond.

Increasing Reliance on Tech

The growth in this area is astounding—78% of companies say their technical debt rose in the last year. A full 38% of companies reported taking on a great deal more; at the same time, IT budgets are set to increase for most organizations in 2022. On average, this increase is expected to be 37.1% higher than 2021 spending. It’s easy to correlate increasing technical debt with these IT budget increases.

A large portion of this increased spending is related to digital transformation. This checks out; modernizing existing technology is a top priority for 19% of tech leaders. Other top priorities include exploring new technologies (17%) and decreasing technical debt (15%). Though managing technical debt is not ranked as the highest priority for IT leaders, it still ranks as a pressing concern due to its ability to hurt innovation. The report found that 69% of companies admitted it has caused their digital transformation initiatives to slow.

Perceptions of Technical Debt

So, what are the leading causes of technical debt? It turns out that most stems from recent modernization and digital transformation efforts. In fact, 39% said new digital products were a major cause. This is followed by infrastructure modernization (34%), data analysis and integration (32%) and new customer-facing services (27%).

Interestingly, not all technical debt is bad. Often, IT professionals say the benefits justify the expense. For example, 86% say that increasing product or service launches justifies taking on tech debt.

Technical debt is also a result of the remote nature of today’s hybrid work environment. During the pandemic, businesses made many concessions to enable remote work and adopted new collaboration and development tools to enable productivity. As a result, 88% reported the pandemic increased their awareness of technical debt. The majority, 83%, also said the pandemic has increased acceptance of technical debt, a sign that many of these changes are here to stay in the new normal of working life.

Responding to Technical Debt

Not all investments lead to the same degree of technical debt; some platforms are bulkier than others. Or, specific software vendor lock-in schemes can prevent lift-and-shift which can swell a technology ecosystem. So, what areas are companies investing in and which are the leading causes of technical debt?

In 2021, companies invested heavily in cloud computing (60%), integration (43%), IoT (39%) and 5G (39%). Investment strategies will change slightly in 2022, with 5G becoming the top most-invested area at 45%. Cloud computing drops to 42% and edge computing is rising to 35%. Mapping where new investments will occur can help organizations anticipate the repercussions. Of these investments, 60% of IT professionals ranked 5G as a leading cause of technical debt. Other high producers include cloud computing, IoT and blockchain.

In 2021, companies spent 24.8% of their IT budget handling technical debt. Managing it presents some significant challenges—32% say siloed data and processes are a key challenge when addressing technical debt. Limited budgets, regulations and a lack of internal alignment also get in the way. Clearly, resolving the issue requires both inter-department collaboration and a hefty budget.

Final Thoughts

Technical debt is, arguably, unavoidable as companies adopt cutting-edge technologies to remain relevant. And, taking on more technical debt is often intentional; for example, new DevOps tools are helping increase deployment fluidity and manage complex cloud-native architectures. Or, new SRE platforms bring observability and monitoring capabilities to improve application reliability. A newer programming language may be more performant, while a low-code abstraction layer complements an engineer’s workflow. In all these scenarios, the benefits often outweigh the added debt.

Yet, much of it is unintentional and is slowing innovation down. This could be shadow APIs or forgotten rotting legacy infrastructure. Sluggish code and un-optimized routing between applications can also be a drag on resources. The reason these situations sound murky is because technical debt, itself, is difficult to quantify. And, only those addressing it with adequate visibility into their stack will have a clear indication of how much debt there really is.

Though the current pandemic has indeed amplified technical debt, things will always evolve and new trends will replace the old. In the digital era, some degree of tech debt is inevitable within a large organization. The key will be effectively managing it over time to maintain a healthy, lean codebase and progressively release features that meet the demands of new customer experiences.

The Software AG Situation Report 2022 surveyed 700 IT professionals on their software usage and annual predictions at the end of 2021. For more insights, you can read the full report without an email gate or paywall here.

January 28, 2022
Google FLoC is Dead | Meta AI Supercomputer Lives | ARM Deal is Dead

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: Google’s FLoC proposal is dead, Meta/Facebook is buying RSC—a huge AI supercomputer, and Arm “will IPO” instead of selling to Nvidia.

1. Forget ‘Federated Learning of Cohorts’—Long Live ‘Topics’

First up this week: Google bows to the inevitable. Third-party cookies will soon go away, because people are fed up with being tracked. But Google’s FLoC proposal wasn’t the answer.

Analysis: FLoC is a flop

Google doesn’t want to invade your privacy. But Google does want to sell well-targeted ads. The key difference between FLoC and Topics is that FLoC buckets were automatically generated—and hence opaque, risking overlap with “protected categories”—e.g., race or gender. But the topic classifications in the Topics API are curated and have user-visible, descriptive names.

Frederic Lardinois: Google kills off FLoC, replaces it with Topics

Google’s controversial project for replacing cookies for interest-based advertising … is dead. In its place, Google today announced a new proposal. … Your browser will learn about your interests as you move around the web.
…
When you hit upon a site that supports the Topics API for ad purposes, the browser will share three topics you are interested in. … The site can then share this with its advertising partners to decide which ads to show you. … Users will be able to review and remove topics from their lists [or] turn off the entire Topics API.
…
It’ll remain to be seen if other browser vendors will be interested in adding the Topics API. [But] they all quickly turned a cold shoulder to FLoC. … The plan is to start trialing the Topics API at the end of this quarter.

A minor tweak and rename? “Yes and no,” equivocates mananaysiempre:

Yes, it still amounts to building user-tracking functionality into user agents. No, it’s not the same approach.
…
FLoC was doing unsupervised clustering of users, whereas Topics allocates users into predetermined clusters. This makes it potentially more transparent and also seems designed to work around the “protected categories” objection.

But Google has a long way to go to convince garett_spencley:

I’m glad FLoC is being ditched, but as an end-user I’m still not asking for this Topics nonsense. It doesn’t solve any problem that I have. It solves problems that the advertising industry has.
…
I’ll choose to use browsers that don’t support this.

2. Machine Learning at Metaverse Scale is Expensive

Zuckerberg’s new toy is RSC: A $400 million Nvidia-based AI supercomputer. And that’s just Phase One.

Analysis: 3.8×10¹⁸ integer operations per second is … a lot

If you rely on Open Compute Project systems for your DevOps, know that it’s no longer the bee’s knees over at Hacker Way. Meta/Facebook is investing eye-watering sums into a custom configuration of off-the-shelf hardware.

Timothy Prickett Morgan: Meta Buys, Rather Than Builds And Opens, Its Massive AI Supercomputer

If you thought it took a lot of compute and storage to build Facebook’s social network, you ain’t seen nothing yet. … To build the Metaverse … is going to take an absolutely enormous amount of supercomputing power.
…
But what is a surprise, given Facebook’s more than decade long focus on … opening up its hardware designs through the Open Compute Project, is that the … Research Super Computer [RSC] system is built from … commercially available … servers, storage, and switches. [It] probably comes down to … working a deal with some vendors who want the Facebook publicity and are willing to work the price and the parts availability.
…
[So Meta bought] 760 DGX A100 systems from Nvidia—which each … have a pair of AMD “Rome” 64-core Epyc 7742 processors … 1 TB of main memory … eight A100 GPU accelerators [and] eight 200 Gb/sec HDR InfiniBand controllers—all-flash storage from Pure Storage, and … caching servers from Penguin Computing. … At list price this machine would cost around $400 million … Phase one of the RSC machine is rated at 59 petaFLOPS … (FP64), and 118.6 petaFLOPS FP64 using the Tensor Core matrix engines. … With INT8 processing on the Tensor Cores, which is needed for AI inference, the phase one machine is rated at 3.79 exaOPS.

How much??? It doesn’t seem like news to kleiba:

I used to work at a university where my professor had been in automatic speech recognition for a long time, but basically gave up on that line of research about 10 years ago because he figured that universities simply cannot compete budget wise with the big industry players. I suppose the same will soon be true for most ML-related areas of research sooner or later.
…
Already, a substantial amount of research innovation in [Natural Language Processing] and [Computer Vision] has been coming from big companies in recent years. Of course there is a discussion to be had about what that means for society at large.

But 3.79 exoOPS? That’s the way the wind is blowing, according to sungazer:

Give it 15 or so years—this level of computation will be on your desktop. If desktops still exist.
…
Or desks.

3. Arm Bid: No Legs

Speaking of Nvidia, as I’ve said before, the Arm merger really shouldn’t go through. And it’s looking less and less likely. ARM chips are an increasing fixture in the data center—especially those that value “performance per Watt.”

Analysis: ARM M&A DOA—IPO PDQ

A veritable hornets’ nest of Bloomberg journalists have put their names to an anonymously-sourced report. The story argues that the proposed Nvidia buyout of Arm from SoftBank Group is dead. Next stop: IPO.

Ian King, Giles Turner, Peter Elstrom, Dina Bass, David McLaughlin, Ruth David and Dinesh Nair: Nvidia Quietly Prepares to Abandon … Arm Bid

According to people familiar with the matter … Nvidia … doesn’t expect the transaction to close. … SoftBank, meanwhile, is stepping up preparations for an Arm initial public offering.
…
If Nvidia manages to get the deal over the line, it would be a massive coup. … But it will be an uphill fight. … The world’s biggest tech companies rely on Arm technology, and they fear they could lose unfettered access under Nvidia.
…
Within SoftBank, there are factions. [And] the ordeal has created divisions within Nvidia. … “We remain hopeful that the transaction will be approved,” a SoftBank spokesperson said in an emailed statement.

Post-Brexit Britain would no doubt like to keep Arm independent. Here’s simonh:

Arm has I think ⅔ of its employees in the UK—mainly Cambridge—and they’re expanding the campus there. The Semiconductor Physics Group at the Cavendish Laboratory at the University is at the cutting edge globally with 18 PhD projects currently.
…
Guess where scads of Arm’s top scientists are recruited from? Arm and the University are joined at the hip. That company’s not going anywhere.

More broadly, it’s good news, thinks hatchet:

Hopefully this means NVIDIA will focus on RISC-V instead. This will be good for everyone.

The Moral of the Story: Zeal should not outrun discretion.

You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

Image: Reed Geiger (via Unsplash)

January 27, 2022