devopsy.kinsta.cloud

Category: News

  • Vercel Acquires Turborepo to Gain Build System

    Vercel Acquires Turborepo to Gain Build System

    Vercel today announced it has acquired Turborepo, a provider of a build system for JavaScript and TypeScript applications that provides developers with access to an easy-to-use monorepository for their code.

    In the wake of that acquisition, the command line interface (CLI) for accessing Turborepo will now be available under an open source license.

    Jared Palmer, creator of Turborepo and other popular open source projects including Formik and TSDX, will continue working on accelerating Turborepo’s capabilities as the new leader of a build performance team at Vercel.

    The goal is to move existing users of the Turborepo cloud service to a Vercel platform for building applications using the Next.js framework, which makes it possible to build web applications based on the React library for creating user interfaces that enable both static site generation (SSG) and server-side rendering (SSR).

    Palmer said monrepositories such as Turborepo not only make it easier for developers to manage builds but also reduce the amount of time required to process those builds using a continuous integration/continuous delivery (CI/CD) platform.

    Rival build systems are also much more complex to configure and manage, which Palmer said is a primary reason many developers have not yet set up their own mononrepositories.

    As the number of applications that organizations want to build and deploy in the digital business transformation era increases, the focus on developer productivity is intensifying. Rather than having to maintain a complex build system, Turborepo is designed to be simple to configure. As a result, more developers can reduce build times by as much as 50%, noted Palmer.

    In effect, front-end developers are able to more easily become full-stack developers by combining that build system with the Next.js framework, he added.

    DevOps teams will need to adjust to applications capable of providing richer application experiences without relying as much on backend infrastructure. That shift has implications for everything from the amount of network bandwidth consumed to the performance of web applications. The Vercel platform employs caching, routing and a React framework to optimize application performance in a way that reduces the dependency on backend infrastructure to optimize application performance.

    As web application development continues to evolve, the overall rate at which they web apps are built should increase significantly. Naturally, DevOps teams will need to consider the implications of having more applications move through DevOps pipelines and the need to continuously update those after they are deployed.

    It’s not clear whether or how much DevOps teams will nudge developers toward frameworks such as Next.js to develop web applications that might not tax backend infrastructure as much as previous generations of applications. In effect, developers can take advantage of frameworks that enable them to treat the underlying IT infrastructure as if it were serverless whenever possible with tools that have a familiar JavaScript construct. Vercel claimed there are already more than 30,000 sites running Next.js in production at organizations like Airbnb, Hulu, Nike, Ticketmaster and Uber.

    One way or another, it’s clear that the way web applications are being built is fundamentally changing. The only issue now is determining to what degree the rest of the IT organization is prepared to absorb that level of change.

  • AWS Outage Outrage | Rusty Linux | ARM Latest

    AWS Outage Outrage | Rusty Linux | ARM Latest

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Amazon Web Services falls on its face, Linux’s move to Rust takes the next step, and the FTC stabs another fatal wound in the horrible Arm/Nvidia deal.

    AWS us-east-1 Breaks the Internet

    First up this week: Amazon Web Services’ outage on Tuesday sent scads of DevOps teams scrambling. But what was the cause?

    Analysis: DNS DDoS?

    While we still don’t officially know, there is some chatter to speculate on. Could it be that AWS’s DNS infrastructure is vulnerable to DNS reflection attacks?

    Hello. It’s Dave Lee: From angry Adele fans to broken robot vacuums

    It started early on Tuesday morning. Robot vacuums ceased sucking, Wi-Fi cameras stopped watching and eager Tinder daters were left unable to “swipe right.”
    …
    Within Amazon itself … the unthinkable occurred: grounded delivery drivers were unable to load packages and deliver to customers’ doorsteps, just as the peak Christmas season begins to step up. … Drivers at multiple facilities across the country were sent home … some dreading whatever workload may await.
    …
    An outage at Amazon Web Services … had rippled through the online economy, crippling services used by millions of people. … An “impairment of several network devices” in … US-EAST-1 was the “root cause” of the disruption, Amazon said.


    I was wondering what DevOps’ers learned? Enter Max Rozen:

    AWS took approximately an hour to update their status page to indicate anything was happening. … While our infrastructure is hosted across 20 AWS regions, a single point of failure was hosted in us-east-1.
    …
    What we learned: … I learned the saying, “Friends don’t let friends deploy in us-east-1.” As a result, we’ll be moving some things over to us-east-2 to start with. Secondly, we’ll be adding redundancy and the ability to failover to another AWS region.


    Getting inside the business, here are Eugene Kim and Katherine Long, to go over everything:

    Even inside AWS … information on the outage remained sketchy [and] rumors spread among staff. One AWS employee speculated that the outage was caused by an “orchestrated DNS attack,” while another employee downplayed those concerns.
    …
    According to a screenshot of an internal AWS communique … ”firewalls are being overwhelmed.” … A separate internal note said … the network issue “specifically impacted” Amazon’s internal DNS servers.

    Linux Officially Gets Rust Code

    The Rust language has taken the next step toward being part of Linux kernel development. Some “drivers and other non-core kernel programs” written in Rust can now be PR’ed. But there’s still no plan to go further.

    Analysis: Still needs lubrication

    Rust is much better than C for writing memory-safe code, and coding provable implementations. However, bootstrapping the toolchain is still stupidly hard.

    Steven J. Vaughan-Nichols: Linux is getting more Rust in it by the day

    It wasn’t that long ago that the very idea that another language besides C would be used in the Linux kernel would have been laughed at. [But] this has been coming for several years now. … Why? Because it’s much safer than C, especially at handling memory errors.
    …
    The next “patch series [adds] support for Rust as a second language to the Linux kernel.” … While he’s encouraging a slow but steady approach to introducing Rust into Linux, [Linus Torvalds] has also said that using Rust interfaces for drivers and other non-core kernel programs makes sense. [But] no one’s going to rewrite the kernel’s 25 million lines of C in Rust.
    …
    The Rust code proposed for the kernel now relies on the stable Rust compiler rather than the beta compilers. … As Linux kernel and lead Rust on Linux, developer Miguel Ojeda, put it, “By upgrading the compiler, we have been able to take off the list a few unstable features we were using.”


    How is it “safer than C”? ssokolow shares this fascinating insight:

    Rust tries to encourage that through good API design which allows it to be proven at compile time that the invariants will be upheld, thus avoiding the need to choose between safety and performance. My favourite example … is “the typestate pattern” which lets you teach the Rust compiler to prove correct traversal of any state machine.
    …
    A more mundane example would be the iterator APIs: No need for bounds checking if the iterator looks up the length once at the beginning and the APIs prevent you from wandering out of each entry it hands you. … Rust programs tend to run faster than their C and C++ counterparts because the developers feel comfortable introducing and maintaining more aggressive optimizations.


    “And then a miracle occurs,” quips @dave_universetf:

    I’m intrigued [but] I can’t help but feel sorry for the people doing bootstrapping work. C is hard enough to bootstrap, but Rust is a whole new level of pain.
    …
    Rust is literally not source bootstrappable. [The compiler] that works right now is Rust 1.39. So then you have to compile 18 more Rust compilers to get up to the current version.

    Nvidia’s ARM Woes Continue

    As I’ve said before, this deal really shouldn’t go through. Ownership of the ARM ISA and the core designs would be a huge strategic stick to beat Nvidia’s competition with. ARM chips are an increasing fixture in the datacenter—especially those that value “performance per Watt.”

    Analysis: Stick a fork in it

    Nvidia’s attempt to buy Arm from SoftBank is suffering death by a thousand cuts. And the latest wound comes from the FTC’s Higonokami knife.

    Paul Thurrott: FTC Sues to Block Sale of Arm to Nvidia

    The U.S. Federal Trade Commission … sued to block Nvidia’s $40 billion acquisition of … Arm, citing competitive concerns. … Nvidia announced its intention to purchase Arm from Softbank for about $40 billion, over one year ago, in September 2020. But the deal fell under immediate criticism from the major industry players that rely on Arm’s … designs, and the UK government announced … it would “intervene” for national security reasons.
    …
    Now, it may not happen at all. “We will continue to work to demonstrate that this transaction will benefit the industry and promote competition,” Nvidia said.


    Explain like I’m 5? Here’s swillden:

    ARM has a non-trivial and increasing share in the data center. … ARM is much, much bigger than phones … Arm is an important competitor to Intel and AMD. … Arm licensing IP to Qualcom, nVidia, Mediatek, Unisoc, Samsung, Apple, Google, etc., has been great for many areas of technology.
    …
    Arm acting as a common … designer for a bunch of different chip vendors has been extremely good for … progress in CPU/SoC design. … It would be really unfortunate if a lot of the industry were to shift back to x86, eliminating all of the competition on hardware features that we get from the many ARM chipmakers.


    But what of SoftBank? Ian King runs the numbers:

    Wall Street analysts are all but calling the deal dead. … According to Citibank analysts, the chances of the Nvidia purchase closing are now just 5%.
    …
    The company faces the prospect of losing a jackpot of roughly $74 billion. … Other suitors are unlikely to match such an offer. … So SoftBank’s best alternative may well be an initial public offering for the Arm business—but that is likely to value the company at … $19.8 billion—only about 27% the value of the … Nvidia deal.

    The Moral of the Story: It’s no secret that the both of us are running out of time


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk. (It’s so typical of me to talk about myself—I’m sorry.)

    Image: Ləman Şirinzadə (via Unsplash)

  • Iterative Adds Experiment Versioning to MLOps Platform

    Iterative Adds Experiment Versioning to MLOps Platform

    Iterative today added an experiment versioning capability to an open source platform for managing machine learning operations (MLOps) using GitOps workflows.

    Dmitry Petrov, Iterative CEO, said the latest version of the Data Versioning Control (DVC) platform makes it simpler to save, compare and reproduce machine learning (ML) experiments at scale without requiring organizations to set up an additional repository to track them. Instead, organizations can store those experiments alongside other software artifacts in a Git repository, he said.

    As a result, IT teams will no longer need to try to keep track of experiments using spreadsheets or notebook tools such as Jupyter, he noted. Organizations that are building AI models for applications constructed using DevOps workflows will also find it easier to comply with a wide range of existing and forthcoming compliance mandates, added Petrov.

    The DVC platform makes use of a Git-like interface to enable organizations to track multiple versions of data sets, models and pipelines across an MLOps workflow. That capability is being extended to make it easier to manage the experiments data science teams create during the construction of an AI model.

    Iterative is at the forefront of an emerging debate over how best to infuse AI models into applications. Providers of platforms used by data science teams are making a case for separate repositories for tracking the artifacts used to construct an AI model. Iterative, on the other hand, is arguing for the use of an AI model build framework that uses the same Git repositories where any other type of software artifact is stored and shared. Iterative’s argument is that, in effect, AI models are just another type of software artifact, noted Petrov.

    In addition to reducing the total cost of building AI models by reducing the number of platforms required, a Git-based approach also makes it simpler for DevOps and data science teams to collaborate because the DevOps team will have more visibility into what AI models will eventually need to be incorporated into an application.

    It may be a while before the development of AI models and applications fully converges. Data science teams today typically have defined their own workflow processes using a wide range of graphical tools. However, as it becomes obvious that almost every application is going to be infused with machine learning and deep learning algorithms to some degree, the need to bridge the current divide between DevOps and data science teams will become more acute.

    In the meantime, DevOps teams should assume that many more AI models are not only on the way but that those models will need to be continuously updated. Each AI model is constructed based on a set of assumptions; however, as more data becomes available, AI models are subject to drift that results in less accuracy over time. Organizations may even determine that an entire AI model needs to be replaced because the business conditions on which assumptions were made are no longer valid. One way or another, the updating and tuning of AI models is likely to soon become just another continuous process being managed via a DevOps workflow.

  • API Sprawl a Looming Threat to Digital Economy

    API Sprawl a Looming Threat to Digital Economy

    New estimates say the total number of public and private APIs in use is approaching a whopping 200 million. APIs are becoming increasingly crucial to the global digital economy. They are the backbone of many digital platforms and drive the composable enterprise model. But this ubiquity presents sprawl issues.

    F5 recently released a study that examines the conditions of the API economy at large. Authored by Rajesh Narayanan and Mike Wiley of F5, the paper, Continual API Sprawl: Challenges and Opportunities in an API Driven Economy, articulates the state of API sprawl and the conditions behind its arrival.

    According to Narayanan and Wiley, “If data is the new oil, then APIs will become the new plastic.” This looming reality will require a continuous approach to API management to avoid further polluting the digital ecosystem.

    Below, I’ll review the report’s main takeaways to see what has given rise to API sprawl and consider how IT leaders should respond.

    State of API Growth

    APIs have evolved as a standard mechanism for businesses and services to connect and share value. And, as more companies begin to rely on them, the API economy is becoming big business—83% of organizations today consider API integration a critical part of their business strategy, according to the 2020 Cloud Elements State of API Integration Report.

    There is a proliferation of API styles. Thousands of public productized APIs exist, but more private and partner APIs are in use. Technically speaking, APIs come in many different forms—they may be web-based, browser-based or embedded into devices. The report also identifies single-purpose APIs and those that aggregate multiple data providers.

    API use cases are pervasive, from hotel bookings to weather, stock tickers, transportation, IoT, DevOps workflows and many other areas. “API-powered apps have permeated every aspect of our lives,” said the report. Yet, typical market estimates are usually quite conservative when sizing up this market, said F5. According to F5’s aggressive calculations, we will be approaching 1.7 billion active APIs by 2030.

    APIs present a high value from startup use cases to enterprise applications. However, a downside is that this growth is leading to sprawl, said Narayanan and Wiley.

    API sprawl is the term used to describe both the exponentially large number of APIs being created and the physical spread of the distributed infrastructure locations where the APIs are deployed.

    Factors Driving API Sprawl

    Sheer growth. Sources predict the number of developers will grow to 45 million by 2030. SlashData also estimates that 30% of developers already use APIs. As the number of APIs on the market moves into the millions, managing growth poses a significant challenge, especially without proper governance and best practices.

    Lack of standards. “The lack of a common shared model contributes to API sprawl,” the report said. API design standards do exist, yet guidelines often leave room for nuances between services. Standards have also emerged around specific industries, like financial data exchange (FDX). While helpful for a single sector, this doesn’t advance multiple industries simultaneously. A lack of standardization leads to differing versioning approaches and integration challenges.

    New development approaches. Integration requirements are forcing many new APIs to unite disparate business apps. The microservices architecture trend is also adding to sprawl, as “APIs are both northbound to interfaces via microservices and horizontally between microservices.”

    Continuous software development. Continuous development, paired with the need to connect business units with bespoke requirements, could produce multiple versions of the same API. This quickly leads to maintenance difficulties, out-of-date documentation and broken clients. The report also cites rising data creation and “everything-as-a-service” trends as harbingers of an API sprawl.

    Various computing evolutions. New computing trends are also prompting a sprawl, says the report. Business units may be operating on different on-premises or cloud environments. When this hybrid situation occurs, APIs could get dispersed over many locations and become difficult to track. Specific connections may be created to cater to new environments like edge computing and IoT devices, further increasing sprawl.

    Problems With API Sprawl

    A digital economy reliant on APIs also relies on the underlying stability and availability of these services. Yet, “APIs have a shelf life and become unsupported if ignored by the developers,” the paper acknowledged. It could be hard to maintain service reliability for APIs sprawled across a distributed cloud. In addition to reliability issues, there are many other possible repercussions of an API sprawl.

    For one, management and operation at scale become difficult. Not all APIs are based on a specification, like OpenAPI. This means that API documentation may not be as streamlined and accessible, thus hurting discoverability and onboarding. “A simple means to connect these APIs may not be possible with conventional or legacy networking approaches,” the report added.

    As developers are top consumers of APIs, a sprawl could negatively impact their integration experience. Maintaining a swelling library of API dependencies can be cumbersome. Plus, APIs are prone to evolve and version over time and when endpoints are altered without advance notice, it leads to broken clients. Even small changes in API functionality can have a huge impact since all observable API behaviors will become relied upon, whether documented or not. (This is known as Hyrum’s Law).

    But the security ramifications of API sprawl are perhaps the most troubling. A whopping 91% of enterprises experienced an API security incident in 2020. Malicious API traffic also rose by a staggering 300% in mid-2021, Salt Labs found. Increased API use will undoubtedly cause more frequent attacks.

    “Unmanaged API sprawl is a security breach waiting to happen,” said the report. APIs typically adopt API keys for authentication and authorization, but this method is prone to misuse. Credentials are often exposed or misconfigured. Bad actors can use APIs to steal loads of data and computing power. As a result, sprawl is undermining trust in API connections and responses.

    Mitigating The Sprawl

    With the growth of the API economy slated to reach the billion mark in the next ten years, figuring out secure, stable inter-cluster communication will become more critical. According to F5, existing solutions (gateways, ingress controllers, service mesh and forward and reverse proxies) are not an adequate response.

    “We believe the solution should hence involve an intermediary (proxy) device focused on solving inter-cluster connectivity, security and integration challenges.” The writers call this solution API Gateway 2.0, which they describe as a bi-directional application-level proxy. The technology exists to implement such middleware and perhaps it could help companies address sprawl issues.

    Outside of Narayanan’s and Wiley’s recommendations, there are plenty of actions API owners can take to avoid adding to the sprawl. Here are a few:

    • Treat the API as a product
    • Improve developer experience
    • Use spec-driven development
    • Ensure up-to-date documentation and code libraries
    • Use consistent endpoint naming
    • Set clear guidelines for versioning and deprecation
    • Go beyond API keys with OAuth and OpenID Connect

    For more context, check out Continual API Sprawl: Challenges and Opportunities in an API Driven Economy for free without an email gate here.

  • Sumo Logic Extends Observability Reach to AWS Lambda

    Sumo Logic Extends Observability Reach to AWS Lambda

    At the AWS re:Invent conference this week, Sumo Logic announced that in addition to collecting log data, metrics and traces, it now can collect telemetry data from the Lambda serverless computing service provided by Amazon Web Services (AWS).

    In addition to collecting telemetry data, Sumo Logic also revealed that the Sumo Logic Continuous Intelligence Platform hosted on the AWS cloud can now analyze how functions used on the AWS Lambda service perform during transactions.

    That data can then also be correlated against the performance of other AWS services that Sumo Logic tracks via integrations with the AWS CloudWatch and AWS CloudTrail monitoring services.

    George Gerchow, chief security officer for Sumo Logic, said the company is making extensive use of the OpenTelemetry agent software created under the auspices of the Cloud Native Computing Foundation (CNCF) to make it simpler to instrument IT environments. That data is not only being used to improve application performance but also to surface anomalies indicative of a security breach, he added.

    As part of this ongoing security effort, Sumo Logic this week also added Sumo Logic AWS Quick Start integrations for rapid access to security and compliance insights along with support for Amazon Inspector, a vulnerability management service provided by AWS.

    AWS this week named Sumo Logic as its independent software vendor (ISV) partner of the year. The two companies have a longstanding relationship that began with Sumo Logic’s decision more than 10 years ago to build a monitoring platform hosted on the AWS cloud.

    As IT monitoring tools continue to evolve into observability platforms, Gerchow said it’s becoming much easier to leverage machine learning algorithms and open source agent software to identify the root cause of an issue before it results in a major disruption.

    Most IT teams today are still relying on legacy monitoring tools that only tack a set of predefined metrics to identify when a specific platform or application is performing within expectations. The metrics tracked generally focus on, for example, resource utilization. In contrast, observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshoot issues.

    Observability, of course, in one form or another, has always been a core tenet of DevOps best practices. Initially, DevOps teams focused on continuous monitoring as the most effective way to proactively manage application environments. However, it can still take days (and sometimes weeks) to discover the root cause of an issue. Observability platforms promise to make it easier to manage IT environments even as the overall complexity of those environments continues to increase.

    It’s not clear at what rate IT organizations will be transitioning to observability platforms. However, it generally takes only one major outage before IT organizations start looking for better tools to manage their IT environment. As those IT environments become more complex, the probability that there will be a significant disruption is now higher than it’s ever been. The challenge and the opportunity is to find a way to instrument applications more widely; the expectation being that an observability platform will be able to analyze telemetry data to reduce the number of issues that lead to service disruptions.

  • AWS Unfurls Bevy of Automation Tools to Streamline DevOps

    AWS Unfurls Bevy of Automation Tools to Streamline DevOps

    At the AWS re:Invent conference, Amazon Web Services (AWS) added a bevy of tools to its portfolio intended to accelerate the pace of application development while simultaneously simplifying DevOps processes.

    AWS development tools unveiled this week include AWS Amplify Studio, a visual development environment that allows developers to create web application user interfaces (UIs) with minimal coding.

    Ken Exner, head of product for developer tools at AWS, said as an extension of an existing AWS Amplify Studio tool, this latest addition allows developers to customize application UIs at a higher level of abstraction using a library of components while enabling them to drop down to a lower level of coding to customize their application further whenever required. After the UI is designed, AWS Amplify Studio automatically generates the associated JavaScript or TypeScript code for the developer.

    In general, AWS is committed to improving developer productivity by providing, for example, automated reasoning tools that automate processes without locking devs into a layer of abstraction that, ultimately, may create a wall that blocks them from meeting customization requirements, he said.

    Other developer tools launched by AWS this week include updates to Amazon CodeGuru Reviewer that adds a secrets detector that uses machine learning algorithms to identify hardcoded secrets that can be secured using AWS Secrets Manager; a revamped Amazon Inspector vulnerability management tool that provides automated assessment scans, automated resource discovery, support for container-based workloads, improved risk scoring, integration with Amazon EventBridge and AWS Security Hub and the removal of the stand-alone Amazon Inspector scanning agent.

    AWS also revealed it has eliminated the need to manually pull images from the Amazon Elastic Container Registry (ECR) repositories by adding automated synchronization capabilities that keep private and public registries consistent with each other.

    In terms of IT operations tools, AWS added a managed database service to its Amazon Relational Database Service portfolio that allows IT teams to customize the underlying database and operating system using an RDS Custom option, a new table class for Amazon DynamoDB designed to reduce storage costs for infrequently accessed data and a service for RDS, dubbed Amazon DevOps Guru, that extends existing machine learning capabilities to automatically detect and diagnose performance bottlenecks and operational database issues.

    Jeff Carter, vice president for relational databases at AWS, said that as databases are increasingly managed by DevOps teams rather than traditional database administrators (DBAs), the cloud service provider is moving to reduce friction by automating as many manual tasks as possible. That’s critical at a time when IT teams are deploying a much wider range of purpose-built databases that are optimized for different classes of applications, he added. The overall goal is to rely more on automation to reduce friction across the IT environment, said Carter.

    Overall, there are two classes of AWS customers: The first are development teams that are increasingly relying on managed services provided by AWS to automate a wide range of DevOps processes. The second tend to be enterprise IT organizations that prefer to manage cloud services themselves.

    Mitch Ashley, a principal for Techstrong Research, an arm of the parent company of DevOps.com, noted that, in many cases, enterprise IT teams are managing DevOps workflows that span multiple clouds. As such, they tend to prefer tools and platforms that can be applied to both multiple clouds and on-premises IT environments. The more those organizations rely on managed services provided by a cloud service provider the more difficult it becomes to consistently manage DevOps workflows across multiple IT environments.

    It’s too early to say just how much IT organizations will rely on managed services going forward. However, at a time when more organizations are using more IT platforms than ever, it’s clear the DevOps management challenge increases with each new platform introduced into the IT environment.

  • AWS re:Invent Roundup: Private 5G | Graviton3 ARM Chips | DevOps Guru++

    AWS re:Invent Roundup: Private 5G | Graviton3 ARM Chips | DevOps Guru++

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Three things that caught my eye from Amazon Web Services’ re:Invent 2021 conference: Private 5G, Graviton3 chips, and something called DevOps Guru for RDS (yes, really).

    Private NRaaS vs. Wi-Fi

    First up this week: AWS Private 5G is a new turnkey service, offering local private cellular data infrastructure, for places where Wi-Fi just doesn’t cut it. Despite the name, it offers a 4G/LTE fallback to 5G/NR. Because it’s AWS, the hardware is free—you just pay for usage.

    Analysis: Combine with an Outpost for a sharper edge

    Wi-Fi is unreliable and low powered. 5G over the 3.5 GHz “citizens’ band” seems like a great alternative (just don’t tell the lunatic fringe of the anti-5G brigade).

    And, of course, AWS makes it easy to buy—Amazon ships you all the private radio gubbins for “free” and just bills you based on capacity and usage. It’s automatically configured and remotely maintained. You just take your internet connection and plug it in.

    Aisha Malik: Amazon launches preview of new AWS Private 5G managed service

    “AWS Private 5G” … is a new service that aims to make it easy to deploy and manage your own private network. … AWS CEO Adam Selipsky said that with AWS Private 5G, you can set up and scale a private mobile network in days instead of months.
    …
    There are no upfront fees or per-device costs: … Customers pay only for the network capacity and throughput they request. [It] scales capacity on-demand.
    …
    AWS Private 5G is available in preview in the United States.


    And this is better than Wi-Fi how, exactly? Ballu counts the ways:

    Industrial area coverage: Where you need 50 Wi-Fi radio units, you can provide service with 3–4 p4G/p5G radios.
    …
    Hazardous areas: Where you can’t provide the networking at all the corners or areas, one p5G radio blast area (esp with beam technology) can.
    …
    Packet loss: Less packet loss compared to Wi-Fi (believe me, it’s a big deal in the Industrial world)
    …
    Compatibility: What’s a negative for 5G modems (not many devices are available), is positive to some extent. Once investment is made with 5G modems, the devices can be on the road too.


    Unsurprisingly, Brian Roemmele feels vindicated:

    Precisely as I wrote about in 2017 to great ridicule from an analyst at a famous VC firm: Amazon is becoming a phone company.
    …
    From the Sidewalk Network to the Kuiper satellite system to local 5G. The network is no longer just “dumb pipes.”

    Graviton3: AWS ARM SoC is PDQ

    Graviton3 is the latest version of Amazon’s ARM server chip. It looks like it’s based on a 5nm process implementing the latest ARMv9 ISA. AWS is making the usual claims about it being much faster than the previous generation, but it also has new security features.

    Analysis: Cool chips

    AWS is deadly serious about low-power, high-performance compute. Amazon is just one licensee that would be distressed to have Nvidia owning Arm.

    As I said last week—and the week before that—ARM chips are an increasing fixture in the datacenter, especially ones that value “performance per Watt.” Other IaaS providers might bleat about how they’re reducing the energy used to cool their data centers, but surely it’s a better plan to start with less waste heat.

    Maria Deutscher: AWS debuts new compute-intensive and AI instances powered by custom chips

    AWS’ new Amazon EC2 C7g instances target compute-intensive workloads such as analytics tools and scientific modeling software. The C7g series is based on the AWS Graviton3 chip, the third iteration of the cloud giant’s internally-designed processor.
    …
    Companies that plan to use C7g instances for machine learning workloads can expect an up to threefold performance jump in AI performance. The speed jump is partly the result of the processor’s support for bfloat16, a specialized data format [that] enables applications to process floating point numbers faster and using less memory than FP32, the data format historically used for the task.
    …
    For cryptographic tasks, Graviton3 delivers as much as double the performance of the Graviton2. … The chip introduces a feature called pointer authentication to reduce the risk of cyberattacks.


    Tim Anderson has been sifting through the re:Invent announcements:

    Top of the list perhaps is Graviton3, the next generation of AWS’s homemade [ARM] server processors. These chips will be used in new C7g instances previewed offering DDR5 memory (50 percent more bandwidth than DDR4) and up to 25 per cent better compute performance compared to instances powered by Graviton2.
    …
    Adam Selipsky gave his first Re:Invent keynote as AWS CEO. [He] said Graviton uses “up to 60 per cent less energy.” … Graviton matters because it is among those leading a charge to get more [ARM] microprocessors into data center and cloud infrastructure, and because if it’s more energy and cost efficient to operate, it’s critical to Amazon’s bottom line, sorry, today’s environmental concerns.


    And is that pointer-authentication feature useful? Yes, very, says spijdar:

    Very useful. … If the performance is good, this is a nice extra layer that makes return-oriented programming more difficult. Combined with NX bits, it really raises the difficulty in developing/using many types of exploits (it’s not impossible to bypass … but there’s no perfect security).

    DevOps Guru Adds RDS Support

    DevOps Guru for RDS is the latest upgrade for AWS’s oddly-named smart monitoring features.

    Analysis: Silly name; useful tool

    Laplace and Gauss were right: Your DevOps team aren’t all rockstars. “Average” ops staff are in the majority. So extending AWS’s monitoring automation to your database instances should be a positive step.

    But that branding: Can anyone take it seriously?

    Stephanie Condon: AWS brings more automation to database management

    The new Amazon DevOps Guru for RDS automatically can find and fix database issues … helping developers using Aurora detect, diagnose and resolve database performance issues. [It] can help remediate a range of issues, such as over-utilization of host resources, database bottlenecks, or misbehavior of SQL queries.
    …
    Last year, AWS introduced DevOps Guru, a service that uses machine learning to automatically detect and alert customers of application issues. … Amazon DevOps Guru for RDS builds on that
    …
    Users can view [issues] in the DevOps Guru console or via notifications from Amazon EventBridge or Amazon Simple Notification Service (SNS).


    Sounds interesting, says u/random_dent:

    I thought the DevOps Guru for RDS sounded interesting. You need to enable performance metrics first, which have their own charges, but it still looks not too expensive, plus three months free-tier, so we may try it out.

    We’ve had some odd issues with RDS performance that are too inconsistent to lock down. It would be interesting to see if this could give us some insight.


    But that name! Here’s Thomas LaRock—@SQLRockstar:

    I’m old enough to remember when DevOps Guru for RDS was just called “database performance monitoring.”


    And @shmick is young enough to say it like this:

    such a cringe name

    The Moral of the Story: Misfortune tests the sincerity of friends


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Andrew Svk (via Unsplash)

  • Fugue Marries Compliance-as-Code Tool to AWS Well-Architected Framework

    Fugue Marries Compliance-as-Code Tool to AWS Well-Architected Framework

    Fugue has infused the best practices defined by Amazon Web Services (AWS) within its software-as-a-service (SaaS) offering for provisioning infrastructure using its infrastructure-as-code (IaC) platform.

    Fugue CEO Josh Stella said that IT teams can now evaluate templates for provisioning AWS infrastructure that were created using AWS CloudFormation or Terraform tools to ensure they comply with the AWS Well-Architected Framework.

    The AWS Well-Architected Framework spans five pillars that AWS has defined for designing and operating reliable, secure, efficient and cost-effective systems in the cloud. AWS has defined a set of best practices for each pillar and rewards organizations for following those by making available additional cloud credits. The goal is to improve overall cloud security by providing guidance that ultimately reduces the number of misconfigurations so that cybercriminals can’t discover and exploit them.

    The Fugue platform complements that effort because it is based on Open Policy Agent (OPA), a general-purpose engine for managing compliance-as-code that is being advanced under the auspices of the Cloud Native Computing Foundation (CNCF). Fugue extended OPA by creating Regula, an open source tool that evaluates IaC files for potential security and compliance violations. Fugue then created a SaaS platform that makes extensive use of the AWS Lambda serverless computing platform to make it easier to enforce compliance policies.

    Those policies include turnkey coverage for specifications such as SOC 2, NIST 800-53, GDPR, PCI, HIPAA, ISO 27001, CSA CCM, CIS Controls, CIS Docker and CIS Foundations Benchmarks for AWS, Microsoft Azure, Google Cloud and Kubernetes.

    Stella said Fugue has now incorporated the technical aspects of the AWS Well-Architected Framework. However, it’s still up to each IT team to navigate the cultural aspects of the shared responsibility model of compliance and security that cloud service providers like AWS follow.

    Unfortunately, developers that employ IaC tools to provision cloud infrastructure often make some faulty assumptions about the level of security maintained by the cloud service provider. The result is often a raft of security and compliance issues that arise mainly because most developers don’t have a lot of domain knowledge expertise in either area.

    It’s not clear how many organizations have adopted the AWS Well-Architected Framework, but as various tools make it easier to implement these best practices defined by AWS the more secure and efficient cloud computing environments should become. In fact, in many cases, those tools are being embedded within a larger set of DevSecOps best practices that are being implemented across multiple clouds.

    Stella said he doubted there will ever be any standardization when it comes to security and compliance frameworks that could be applied to multiple clouds; however, it’s clear that cloud service providers are at least making available frameworks that borrow concepts from one another.

    Regardless of the approach, the overall state of cloud security and compliance will continue to improve as more guardrails are automatically implemented. The issue then becomes not only finding a way to ensure that cloud infrastructure is secure when deployed and also determining how much of the infrastructure being relied on today is less secure than it should be.

  • WhiteSource Report Finds NPM Vulnerabilities Fixed Fast

    WhiteSource Report Finds NPM Vulnerabilities Fixed Fast

    WhiteSource today published a report that found most of the vulnerabilities that affect node package managers (NPMs), widely employed to deploy JavaScript applications, are addressed long before they are assigned a Common Vulnerabilities and Exposure (CVE) in the National Vulnerability Database (NVD).

    The report, based on an analysis of the vulnerabilities that WhiteSource tracks in its own vulnerability database and revealed at the AWS re:Invent conference, found more than 90% of vulnerabilities in NPM packages are fixed before the vulnerability is published in the NVD.

    Susan St. Clair, director of product management for WhiteSource, said that fact suggests that, overall, the open source community is staying on top of vulnerabilities as they are disclosed rather than waiting for them to be assigned a CVE number.

    The WhiteSource report comes on the heels of a breach of UA-parser.js, a widely employed NPM based on a JavaScript library that detects browser, engine, OS, CPU and device type/model from User-Agent data. Unknown malicious actors found a way to inject malware into the NPM that modified the library. That vulnerability has since been remediated.

    In fact, the WhiteSource report noted 83% of the CVEs assigned to the most widely used versions of vulnerable NPMs could be remediated before the publication of the CVE.

    In the wake of a series of high-profile breaches of software supply chains, there is a lot more focus on how open source software is actually secured. Commercial software products have a distribution list that they typically notify of a security patch. Open source updates, on the other hand, need to be tracked by developers that use that software. The level of attention individual developers might pay to any number of open source updates can vary widely.

    Many developers are also hesitant to immediately update their applications because they are not sure what impact that additional code may have on their application, said St. Clair. For example, their application may have dependencies that are no longer backward compatible with the updated code, she noted.

    Developers also need to determine the degree to which that vulnerability may affect their applications because they often don’t use every component of an open source library, so the fix being offered may not be required.

    Open source project maintainers often depend on their community to help uncover vulnerabilities and create the fixes as quickly as possible. When a vulnerability is detected in an open source library, the more common practice is to report it to the NVD even after the fix for that issue has been made available.

    In general, St. Clair noted the most important thing for the open source community to remember is to remain proactive when it comes to tracking vulnerabilities and then implementing DevSecOps best practices. The one thing that is certain is that a wide range of malicious actors also are tracking those vulnerabilities closely. It’s not uncommon to see a wave of exploits being created once a vulnerability is disclosed via the NVD.

    The challenge, of course, is that the more open source software that is employed, the more difficult it becomes for individual developers to stay on top of every vulnerability in an application portfolio that only continues to grow. On the plus side, vendors have pledged more than $10 million to help better secure open source software. Hopefully, the return on that investment will become apparent sooner than later.

  • WTH? We Wanna WFH | DoD Dual-Sources JWCC | More Nvidia ARM Woes

    WTH? We Wanna WFH | DoD Dual-Sources JWCC | More Nvidia ARM Woes

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Working from home is de rigueur, JEDI redux, and more about Nvidia/Arm.

    Survey Says: Devs Stay Out of Office

    First up this week: GitHub’s annual survey of developers has some fascinating data on the workplace (ahem) “new normal.” The pandemic has radically changed devs’ expectations of where they can do their jobs.

    Analysis: WFH is SOP

    As with other roles, Dev people want to continue to work from home. But does this pose a threat to your DevOps culture? Make sure you can do Ops with equal flexibility.

    Paul Krill: Dev productivity is back to pre-pandemic levels

    GitHub’s “2021 State of the Octoverse” research … sees signs that work rhythm is returning to pre-pandemic levels. GitHub also found that 46 percent of developers who worked co-located with teammates now expect to work fully remotely or in a hybrid environment.
    …
    Based on data culled from more than four million repositories and surveys of more than 12,000 developers … JavaScript and Python remain the top languages, followed by Java and TypeScript. [And] GitHub found that development team performance can increase as much as 87 percent when reusing code and as much as 43 percent when using … CI/CD.


    Zeroing in on the WFH angle, it’s Richard Speed:

    Respondents who were in the office either full or part-time dropped from 42 percent before the pandemic to a mere 10.7 percent now. … That’s a lot of empty desks.
    …
    Fans of the ctrl-c, ctrl-v combo will be delighted to note that GitHub found that when code reuse was made “frictionless” at work, developer’s performance increased by up to 87 percent. … It’s all splendid stuff.


    But beware of statistical fallacies, thinks Henrik Ingo—@h_ingo:

    #WorkFromHome may have been more common than anyone realized. [But] is the population of GitHub users/respondents somehow skewed toward pioneers/early adopters? … This might be a case of correlation is not causation.

    Defense Dept. Orders New Clouds

    The DoD issued its latest cloud contract opportunity notice, with the snappy title of “Joint Warfighting Cloud Capability” (JWCC). Reading between the lines of the notice, it all sounds very DevOps-cum-Agile.

    The Pentagon has backed away from its previous insistence on a single supplier, and now wants at least two cloud providers.

    Analysis: JEDI redux, because AWS wants a do-over

    JWCC is the follow-on from the aborted JEDI procurement. Amazon Web Services was “disappointed” to have been passed over in favor of Microsoft. Now it appears the DoD wants them to share.

    What can you learn from such a multi-vendor DevOps strategy?

    Jordan Novet: Pentagon asks Amazon, Google, Microsoft and Oracle for bids

    Amazon and Microsoft were the finalists for a single Joint Enterprise Defense Infrastructure. … Microsoft won it in 2019, Amazon filed a protest and the Pentagon canceled the contract in July.
    …
    The new effort [is] known as Joint Warfighting Cloud Capability. … JWCC differs from JEDI because it allows the Pentagon to rely on multiple cloud providers.
    …
    The U.S. General Services Administration … said only two U.S. cloud infrastructure providers, Amazon and Microsoft, appear able to comply with all of the Pentagon’s requirements. … The value of the new contracts is not known, but the Defense Department estimates it could run into the multiple billions of dollars.


    In summary, heed this Anonymous Coward:

    Competing suppliers will try harder than one non-competing supplier. The DoD has had a lot of problems with One-Ring-to-rule-them-all systems like the F-35. … JEDI would have been the same.
    …
    Smaller steps with upgrades while testing/using offers better returns. Done right they can get a better deal by making the companies compete against each other in making sure the products are actually useful.


    But look carefully at the DoD contract opportunity, says bustinbrains:

    They’ll all get contracts. Then DoD will simply use the one that they actually like the best. No one can complain because it’s open-ended with no guarantees.

    Arm/Nvidia Deal on FTC’s Radar

    Last week, we talked about Nvidia’s goal to acquire Arm Ltd from SoftBank, and the regulatory challenges it poses—notably in the UK, home of the chip architect. After we went to press, the other shoe dropped.

    Analysis: Just Give Up Already

    As we said last week, Nvidia can’t be allowed to do this. Not only are ARM chips found in 99% of smartphones, but they’re an increasing fixture in the datacenter—especially places that value “performance per Watt.”

    Richard Waters: What’s next for the tumultuous takeover of Arm?

    Nvidia’s acquisition of UK chip design company Arm from SoftBank has provoked serious opposition on both sides of the Atlantic. … The US Federal Trade Commission had its own worries. China, meanwhile, is waiting in the wings.
    …
    The deal has shone a spotlight on the unique position Arm plays in the chip industry. [Nvidia] offered a guarantee that it won’t block any other companies from licensing Arm’s designs. The offer has fallen on deaf ears, with both the EU and UK ruling it inadequate. … Post-Brexit politics have also come into play.
    …
    At what point might Nvidia and SoftBank call it a day? … A stock market listing [is] the most likely alternative, with the UK a favoured venue.


    How’s it viewed from the inside? Here’s a hint from fivemack:

    I was at ARM when Softbank bought them; it came as a surprise to everybody, because we were absolutely confident that we were unbuyable because anyone interested in us would want to use differential licensing as a weapon against their competitors, and we … were confident that regulators wouldn’t accept that.
    …
    Lots of people went home with a year’s pay check in free money. And not a few people … went home with a decade’s pay check in free money.


    More and more DevOps workloads are running on ARM silicon. Here’s Biophoton:

    The low power expertise of ARM has to be an advantage in the data centre (DC) market. … Plus Linux is available on ARM so architectural lock-in of Intel could be eroded in the DC market.

    The Moral of the Story: You are known by the company you keep


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Lukas Mann (via Unsplash)