devopsy.kinsta.cloud

Tag: hardware

  • What the Evolution of Specialized Hardware for AI and ML Means for DevOps

    What the Evolution of Specialized Hardware for AI and ML Means for DevOps

    The widespread adoption of artificial intelligence (AI) and machine learning (ML) technologies is accelerating the evolution of computational hardware, essential for automating complex processes and enhancing accuracy in decision-making. This acceleration is crucial in advancing computer science and data processing, especially within the various segments of AI/ML data pipelines.

    This growing demand is prompting a diverse array of chip manufacturers, from established players to emerging contenders, to innovate and lead in the development of faster and more efficient processing solutions. The primary goal? To design chips that optimize data transfer, improve memory management and enhance power efficiency. These advancements are far from incremental; they are pivotal in meeting the escalating demands of sophisticated AI and ML applications.

    This evolving hardware landscape for DevOps necessitates greater unification and automation of the various applications to the specialized AI infrastructure. With chip architectures diversifying, building the application stacks focusing on portability, performance and ease of access becomes even more crucial. The ability to seamlessly embrace multiple architectures will be key to striking the right balance for the varying technical abilities of the people who want to interact with technology.

    This article explores the implications of these hardware advancements on DevOps processes within AI and ML. Additionally, we explore strategies DevOps teams may consider to ensure applications remain efficient and portable across various chip architectures.

    The Rise of Specialized Hardware

    Historically, general-purpose hardware, including GPUs and CPUs, was the staple for many workloads. However, a clear pivot to specialized hardware is underway in artificial intelligence and machine learning, where training and inferencing demands have grown exponentially. These applications often hit performance limitations inherent in traditional hardware, partly due to the modern constraints of Moore’s Law. As a result, there is an increasing need for hardware capable of handling specific AI tasks with increased efficiency and speed. For instance, in certain machine learning scenarios, using hardware that supports single-precision floating-point computations can speed up processes without the need for the accuracy provided by double-precision computations.

    Although NVIDIA remains a dominant force in the AI chip market, the competition is heating up, with various companies offering innovative alternatives. And it’s not only from the usual suspects like Intel or AMD; Google has made strides with its Tensor Processing Units (TPUs). Amazon recently announced Trainium2, a new AI chip designed specifically for training AI systems. This chip, set to compete with Microsoft’s Maia and Google’s TPUs, underscores the growing custom AI chip development trend among major tech companies. Beyond these giants, startups such as Cerebras, SambaNova Systems, Graphcore and Tenstorrent bring fresh takes on AI hardware solutions.

    Challenges for DevOps

    As specialized hardware becomes more prevalent, the DevOps community will need to manage various new challenges, including performance portability. Performance portability refers to ensuring that applications run efficiently and perform well across varying computing architectures with minimal to no modifications.

    Cognitive computing (the broader category of AI and ML) varies in complexity, depending on the algorithms, models and creators’ unique demands for hardware-specific features. While an architecture-tailored version will certainly maximize performance on respective platforms, it complicates the process of ensuring a consistent software experience across diverse hardware.

    The challenge of system design lies in optimizing the environment for maximum efficiency, especially when the precise nature of the workload is unknown to those responsible for designing and supporting the systems.

    Of course, there are also significant and related considerations with continuous integration and continuous deployment (CI/CD) pipelines. The intricacies of CI/CD pipelines are magnified when pursuing performance portability. The need to validate software performance across multiple hardware configurations introduces a more elaborate testing matrix and can extend deployment cycles, directly affecting time-to-market requirements. Suddenly, workloads are now crossing the traditional IT infrastructure and high performance/supercomputing boundaries once defined by microservice and batch processing technologies; they are now one in a CI/CD pipeline.

    As organizations adopt specialized hardware, there’s the potential for a parallel rise in specialists focusing solely on one type of hardware or application use case. While such expertise can drive innovation and optimization for a particular platform, it also creates complexity and the potential for knowledge silos and unnecessary complexities for the operations teams and clients that use those systems.

    Leveling up Performance Portability

    Performance portability and closely related concepts, such as hardware-agnostic performance and cross-platform efficiency, are increasingly important for DevOps teams. As the technological landscape shifts, the pressing question becomes: How can the industry and DevOps teams navigate this evolution seamlessly?

    Ongoing research and development will no doubt play a key role. For example, the U.S. Department of Energy (DoE) is exploring new methodologies to support its Exascale Computing Project. This includes refining existing software libraries, pioneering new programming models and developing new tools that could eventually influence wider DevOps practices. Other researchers are developing software abstraction layers, aiming to simplify the adaptation of generic code for specific hardware configurations.

    Beyond new tools and methodologies that may come from current R&D efforts, there are many existing tools and processes that lend themselves to improving performance portability, including:

    • Containerization: Containers encapsulate applications and their dependencies in a way that helps ensure they run consistently across various environments. Open source tools like SingularityCE with Open Container Initiative (OCI) compatibility can help standardize and simplify deployment across different hardware configurations, promoting performance portability for high-performance computing and traditional IT infrastructure management.
    • Benchmarking and profiling: To achieve performance portability, it’s imperative to understand how software behaves across different architectures. Benchmarking tools provide quantitative performance measures, while profiling tools offer insights into software behavior, helping developers identify bottlenecks and areas that need optimization.
    • Code Portability Libraries: In addition to libraries like OpenCL that enable software execution across diverse hardware, recent advancements in container technologies complement this capability. For example, enhancements in container device interfaces, such as those in SingularityCE, streamline the integration of hardware-specific resources. This development aids DevOps teams in optimizing software for various hardware without extensive codebase rewrites, exemplifying the tools supporting hardware diversity and software agility in all aspects of cognitive computing.

    Of course, on top of all these tools and strategies, agile methodologies will remain essential since they prioritize iterative development, continuous feedback/improvement, and adaptability—all of which are important to fast-evolving hardware and software configurations.

    Looking Ahead: Adaptability as the Key

    As we embrace the new frontiers in AI and ML, the role of DevOps teams in navigating a constantly evolving hardware landscape becomes increasingly vital. At the heart of this journey lies the formidable challenge of application portability—a challenge that calls for technical expertise and a strategic shift toward adaptability. Enter containers emerging as indispensable tools that ensure a consistent application experience and performance across diverse platforms and handle the intricacies of portability.

    Likewise, adopting agile methodologies transcends process adherence; it embodies a mindset of flexibility and responsiveness, essential in rapid technological shifts. These approaches, far from temporary fixes, are integral to a strategy that aims to unleash the potential of AI. As DevOps teams continue to navigate these challenges, their success will depend on their adaptability and willingness to explore and integrate established and emerging technologies—particularly those excelling in scalability and efficiency. This proactive exploration and integration will be the key to surviving and thriving in this dynamic technological landscape. And yet, while we have discussed this problem regarding scientific applications, the impact is also felt on the AIOps side of the house, using AI-specific hardware with AIOps practices. While our goal here was to discuss the complexities of creating and administering systems for artificial intelligence and machine learning, we have yet to approach the security conversation, particularly with confidential computing, which is a topic for another time. Happy computing.

  • Of Max and Min: When Performance Engineering Plans Go Awry

    Of Max and Min: When Performance Engineering Plans Go Awry

    Modern software developers have access to powerful tools and services which allow them to quickly develop, demo and deploy fully functional applications. But what happens when the single-user prototype satisfies all required functionality but the user says the system seems slow? Or if the initial implementation turns out fine for one user, but multiple users start to experience delays? Or if users are happy, but the cost of auto-scaling grows prohibitive?

    The Problem with Performance Engineering

    Concern over non-functional aspects of computing systems—response times, resource utilization and costs—falls under the domain of computer systems performance engineering (CSPE). Unfortunately, the practice of CSPE has evolved into something more akin to art than engineering—with few standardized principles and practices. Because software is constantly evolving its languages and abstractions, CSPE practitioners appear obligated to evolve and change their abstractions and languages, too.1 As a result, part of our jobs as performance engineers is to understand what the heck other performance engineers really intend or mean.

    There are many reasons for this state of affairs, but ultimately this lack of a common language and engineering standards for CSPE is a deficiency in the education and training of software engineers. There is very little in the typical university software engineering curriculum that prepares software engineers for doing CSPE. This sometimes leads to practices that are not backed by scientific or engineering principles. As a result, practitioners are forced to “wing it”—creating their own art or hitching their wagons to someone else’s star and learning (or not) through trial and error.

    The PMWG – A POSIX for Performance?

    The Performance Management Working Group (PMWG) was started in the late 1980s during the “UNIX wars,” when computer vendors were feuding over claims to the UNIX leadership mantle and had formed two factions: UNIX International and the Open Software Foundation. The group was unique in that its members included computer systems vendors on both sides of the “war.” Its members put aside those differences to try to come up with performance management standards and practices that would put UNIX on par (at least) with the performance management tools found on mainframes.

    In retrospect, the PMWG was probably the best chance UNIX had for establishing, promoting and enshrining some basic performance engineering principles—a kind of POSIX for performance. Unfortunately, the group failed—in large part, in my opinion, because it put too much priority on the latter stages of the performance management data pipeline and did not spend enough effort toward establishing key primitives. Today, in the absence of established rules and practices for performance engineering, history is repeating itself. Current efforts in observability continue to focus mainly on data presentation and data transport and storage, without much focus on the right metrics and data quality based on basic performance engineering principles. Presentation, transport and storage are important, but the efforts around metrics, logging and tracing coverage and data quality are sporadic at best.

    Inertia, Expediency and Superstition

    Why are more formal, engineering-based practices around CSPE needed? Consider the use of CPU utilization as a key performance indicator (KPI) of system performance. I’m a member of Bloomberg’s Trading Solutions SRE team, which manages hundreds of machines on which the company’s Trading Solutions software runs. We often see or hear statements like “This host’s CPU is too busy,” or “That host is out of CPU resources.” I’ve found that these observations are not actually very helpful and sometimes divert our attention from the real problem(s). When misused as a system KPI, focusing solely on CPU utilization can lead us down the wrong path and draw incorrect conclusions. To understand why, let’s first look at how we assess the performance of real-world, non-computing systems.

    Take fast-food restaurants. One of the most important attributes of fast-food restaurants (besides their health inspection rating) is that they are fast. But we’ve all been to fast food restaurants that are anything but fast. Without actually standing in line and waiting, we can often tell that we will have a slow experience at a fast food joint if there is a long line. The line length gives us a lot of information. Our assessment of the situation is different whether there are 100 customers versus 10 customers versus a single customer waiting in line.

    In the real world, this line length (or queue length) assessment is almost universal. Supermarket checkouts, ATMs, airport security screening, toll booths, elevators and COVID-19 testing are all areas where we apply this semi-conscious assessment. In contrast, we almost never use “cashier utilization” or “self-serve soda fountain utilization” as a way of assessing how slow a fast-food restaurant is. We always say “the lines are long” or, if we actually choose to wait in line, we’ll make the more direct statement that “service is slow.”

    Which brings us back to our computing systems. Why do we reflexively look to CPU utilization when computing systems are slow while, in the physical world, we rely on queue lengths? The question is even more relevant when we consider that the concept of CPU is less well-defined today. Are we measuring the CPU, its cores or hardware threads?2 I believe the contributing factors to this misuse include inertia, expediency and superstition.

    In the early days of computing, CPUs were perhaps the most expensive component of the large monolithic computer systems of the time. Measuring how much these expensive components were being used was an important part of maximizing the financial investment in these large machines. Fortunately, measures of CPU utilization were relatively easy to estimate. Ease of implementation meant that CPU utilization measures were almost universally available.

    The universal availability of this simple metric, combined with some correlation (sometimes weak) between high CPU utilization and system slowness made it a default indicator of systems performance. When a belief becomes ingrained to the point of superstition, even weak correlations are enough to validate and perpetuate it. Better metrics—for example, run queue lengths at the CPU(s)3—can help us shed our superstitions to better understand our systems and lead us in the right direction in identifying and correcting problems.

    Dispelling Superstition and Other Irrational Practices

    Which brings us to the title and purpose of this series. As a title, “Of Max and Min” is meant to evoke John Steinbeck’s classic novella “Of Mice and Men,” which in turn got its title from the famous line in Robert Burns’ poem “To a Mouse”: The best-laid plans of mice and men oft go awry. The two main characters in “Of Mice and Men” fail in their efforts to better their lives during the Great Depression because they are tragically ill-equipped to do so. Similarly, we should not be surprised if plans go awry when software engineers are tasked with doing performance engineering without the proper training in performance engineering principles.

    “Max and Min” also highlights the importance of data analysis and statistics in understanding systems behavior. Effective CSPE requires us to be familiar with concepts in statistics, numerical analysis, and even operations research—in addition to the more standard computer science areas of algorithmic complexity and computer architecture.

    With “Of Max and Min,” in addition to calling out superstitions, I will be pointing out some common mistakes we make and blind spots that we have.4 Through case studies, I’ll illustrate how our blind spots can lead to mistakes that can persist for years. I will also try to cover some basic CSPE principles that I believe are skipped in most software engineering training. Like everyone else, I’ve been winging it for 40+ years. And I’m still learning. If you disagree with any of my points or have further insights into the topics that I am presenting, please let me know.

    1But, as in the application of the principles behind algorithmic complexity, there are some performance engineering principles that can and should be adhered to regardless of the system and language du jour.

    2There are in fact some bizarre implementations of “CPU utilization”. Here’s an example of how AIX “broke” the meaning of CPU utilization on their multiprocessor systems running in hyper-threading mode.

    3UNIX and Linux do in fact have metrics that could be used as CPU queue length indicators.  In a future installment, I will go into the quirks of some of the CPU queue length implementations and further argue for more consistent mathematical bases for queue length metric implementations.

    4For example, have you ever wondered why, after so many outages caused by logging, malloc, DNS and other low-level services, we still don’t have good, out-of-the-box visibility into these and other low-level components? We are accepting key blind spots as a fact of life when we really shouldn’t.

    About Me

    Like many software engineers, I started off studying a different field (Physics). Unlike most software engineers, I’ve always wanted to be a performance engineer. When I started taking computer classes as an undergraduate in Columbia College’s Physics program, I did not wish for a career writing programs in Fortran (yes, it was that long ago). But I did envision myself making systems run better and faster.

    To that end, when I added a Computer Science major to my undergraduate studies, I also added a minor in Industrial Engineering and Operations Research. For those unfamiliar with IEOR, it is a multidisciplinary engineering field that leverages mathematical and analytical methods to address complex system problems like resource allocation, supply chains, wait times, fairness, etc. Tools and methodologies from IEOR are invaluable to CSPE.

    At the start of my career, I joined the Bell Labs Computer Center as a computer systems performance engineer, where I made the somewhat contrarian decision (for 1980) to work in the group supporting their in-house, low-key UNIX systems (as opposed to their commercially popular MVS mainframes). I didn’t realize it at the time, but having access to the source code of the system you support/study can be a boon to understanding. At Bell Labs, I had the opportunity to contribute to the UNIX System V kernel, where I co-developed the first general-purpose kernel and user-land tracing system for UNIX and made significant performance enhancements to the virtual memory subsystem, and participated in the PMWG.

    Since then, I’ve spent the bulk of my professional career working on financial software systems—ranging from market data to messaging to transaction management. I’ve worked on monitoring systems on an enterprise level. I’ve made and witnessed many CSPE mistakes. In my current role as an SRE on the Trading Solutions team at Bloomberg, I’m seeing and helping remedy lots of operating system issues that stem from our runtime scale.

    Thanks

    Thanks to everyone who reviewed this document and provided constructive feedback. Special thanks to Nate McNamara and Peter Wainwright for suggesting substantive organizational and structural improvements. Peter also reined in my default stream of consciousness, run-on style. None of them had any hand in the writing of this paragraph.

     

     

  • Heterogeneous Hardware Needs Universal Software

    Heterogeneous Hardware Needs Universal Software

    As hardware advances and diversifies, we’re entering what many see as a new golden age of computer architecture. However, the idea of an ever-expanding hardware landscape can be daunting for software developers, because while hardware diversity is wonderful for price and innovation, it can lead to spiraling software complexity on the back end. So many software tools are specific to the hardware they run on that a world where CPUs, GPUs, FPGAs, ASICs, DSPs and more are freely intermingled feels overwhelming; an invitation to interoperability headaches and spiraling tech overhead. The question on the minds of developers is how to navigate this constantly changing ecosystem to take advantage of the new world of hardware acceleration without driving themselves crazy in the process. 

    The challenge for developers is that, if we look historically, every time a new piece of world-changing silicon shows up, it often comes with a new software stack of tools. With heterogeneous hardware innovation moving at a nearly untrackable pace already, keeping up with the software end is beginning to feel untenable. In response, a new philosophy is emerging: We should be able to have a common set of tools, libraries, frameworks and compilers that can address a wide variety of accelerators in a “silicon agnostic” way. 

    Anyone doing DevOps work today should be asking, “Are the tools we are adopting ready for the heterogeneous hardware future?” Rather than using proprietary stacks exclusive to individual vendors’ concepts of how acceleration should work, we should be thinking about abstraction layers that allow us to move to open software stacks that work across multiple vendors and architectures. 

    This philosophy is central to a number of movements—oneAPI being one example—of asking for libraries, compilers and analysis tools that are more versatile. It’s incumbent on any future-conscious developer to analyze the tools they’re using and ask if they are preparing for the future or locking them into a single solution that excludes you from other innovations. 

    When it comes to navigating the changing developer landscape, it’s particularly important to look at the multiple-vendor piece of the puzzle. One of the signature strengths of computing and electronics is its ability to interoperate. Without compatibility between platforms, we end up locked into single solutions that don’t take advantage of the diverse landscape of innovation available to us. If systems are built from the ground up with mixing and matching in mind, we don’t have to change our software tools to adapt to new technologies and we can use the same languages, libraries and analysis tools we are used to. 

    It’s important we take on this work now because hardware diversity is not just at an all-time high, its momentum is only growing. In the past when the hardware landscape was more limited, switching to different toolsets for different CPUs was a feasible, if inelegant solution. Today’s computers, however, have computational capabilities from multiple vendors. If my tools don’t stretch across all of its ecosystems, I end up in a complex mess very quickly.  

    A particularly relevant example can be found in supercomputing, one of the most exciting and fast-moving areas of innovation in the developer community. What is the most exciting thing about supercomputing? It’s heterogeneous, allowing multiple architectures to take on specialized hardware acceleration for the different tasks for which they are best suited. The Aurora supercomputer, for example, is planned to achieve up to 2 exaflops of performance—a staggering number (the first supercomputer I helped design had 1 teraflop—or 2 millionths of Aurora’s performance). The amazing capabilities of these machines are only possible because of advancements in heterogeneous memory technologies, storage capabilities, sophisticated CPUs and GPUs all working together seamlessly.  

    The advancements we’re seeing in supercomputers, as always, will ultimately filter down into lower-level systems. In the near future, regardless of the type of work you do, enormous computing power is coming online. We need to make sure that our libraries and tools can take full advantage of it when it arrives.

    Two great places to learn more about activities to help software be open, multi-vendor and multi-architecture for heterogeneous programming are:  https://sycl.tech/ and https://www.oneapi.io. There are numerous online tutorials, a link to the SYCL book (free PDF download) and much more.

  • Intel Expands Scope of OneAPI Toolkit Initiative

    Intel Expands Scope of OneAPI Toolkit Initiative

    Intel has made available a 2022 edition of its oneAPI toolkit through which it makes processor extensions accessible to DevOps teams.

    James Reinders, chief evangelist for Intel software products, said the goal is to make it simpler for DevOps teams to expose extensions to any company’s processors in the runtimes developers use.

    At its core, the oneAPI toolkit is a collection of APIs that Intel created for x86, graphics processing units (GPUs), application-specific integrated chips (ASICs), field programmable gate arrays (FPGAs) and other classes of processors.

    Reinders noted that developers have never had as much ready access to so many different processor classes before. The challenge organizations face, however, is making it simpler for developers to take advantage of those capabilities in a way that doesn’t require them to write low-level code, he added.

    The oneAPI 2022 toolkit adds additional capabilities including the world’s first unified compiler implementing C++, SYCL and Fortran, data-parallel Python for CPUs and GPUs, advanced accelerator performance modeling and tuning and performance acceleration for artificial intelligence (AI) and ray tracing visualization workloads. Collectively, Intel said it has made 900 new and enhanced features available via the oneAPI toolkit in the last year, including deeper Microsoft Visual Studio Code integration, support for Microsoft Visual Studio 2022 and Microsoft WSL2 for Linux development on Windows.

    As IT continues to evolve it will become more routine for developers to invoke heterogeneous classes of processors within the same application, said Reinders. Those capabilities will drive an explosion of software development innovation through the rest of the decade and beyond, he added.

    Intel is especially focused on making it easier for developers to invoke inference engines that take advantage of processor extensions to run AI workloads, Reinders said. Going forward, most applications will take advantage of machine learning algorithms that run inference engines optimized by Intel via extensions to its processors. As those workloads are constructed, integrating AI models within DevOps workflows will become the next major challenge, noted Reinders.

    It’s too early to say how simplified access to heterogeneous processors, also known as XPUs, might change the application development landscape. The one thing that is clear is IT organizations now have more options than ever. The issue now is finding a way to invoke all those capabilities in a way that doesn’t overwhelm developers. DevOps teams will play a key role in deciding which processors and associated extensions to make available to their development teams.

    In the meantime, the battle for processor dominance will only intensify in the months ahead. DevOps teams will need to decide how comfortable they are using processors manufactured by a single company like Intel or whether they want to employ a mix of processors provided by multiple vendors. Regardless of what they decide, it appears APIs will play a critical role in making it easier to benefit from advances in processor technologies.

  • Intel Woos DevOps Teams with API and Processor Advances

    Intel Woos DevOps Teams with API and Processor Advances

    During the online Intel Innovation 2021 event, Intel announced today a unified Developer Zone for accessing application programming interfaces (APIs) that it makes available via a oneAPI toolkit to IT teams that want to more easily invoke a wide range of processor types and classes.

    At the same time, the company also revealed it is creating a set of oneAPI centers of excellence that will make it easier for DevOps teams to consume those APIs. Partners participating in that initiative include Oak Ridge National Laboratory, University of California Berkeley, University of Durham and University of Tennessee. Intel will also make the Intel Graphics Visualization Institutes of Xellence part of this program.

    Next year, Intel is also promising to deliver oneAPI 2022 toolkits that will add 900 additional capabilities including an ability to develop applications for CPUs and graphical processor units using what Intel described as the first unified C++/SYCL/Fortran compiler.

    The company is also planning on making available Data Parallel Python tools in addition to expanding its Advisor accelerator performance modeling tools to include support for VTune Flame Graph to visualize performance hot spots.

    There will also be additional oneAPI integrations to add support for Microsoft Visual Studio Code and the Microsoft Windows Subsystem for Linux 2 platform. Intel also created an Intel oneAPI AI Analytics toolkit to enable out-of-the-box AI development that is supported by workstations running Linux from Dell Technologies, HP and Lenovo. Additionally, Microsoft and Intel have agreed to create a complete data science toolchain to Windows 11 that will become part of the Surface Laptop Studio framework for building applications.

    Greg Lavender, chief technology officer, senior vice president and general manager of the software and advanced technology group at Intel, said it’s critical to make sure that DevOps teams add support for the acceleration engine and extensions that Intel makes available within its processors to the runtime environments they deploy. Otherwise, developers will never be able to routinely invoke those capabilities to build next-generation applications.

    The company also revealed that next-generation Intel Xeon Scalable Processors, code-named Sapphire Rapids, will have additional acceleration engines to both simplify deployments at scale and optimize processors’ core utilization in a way that reduces power consumption. Overall, Intel is promising up to a 30x total performance gain for artificial intelligence (AI) applications running on these processors.

    Other hardware advances announced today include a 12th generation Intel Core family based on 7-nanometer processors and a Mount Evans project through which Intel is collaborating with Google Cloud to build processors based on application-specific integrated circuit (ASIC) chips.

    The company also unveiled an Intel ArcTM Alchemist family of graphics systems on chips (SoCs) that take advantage of machine learning algorithms to optimize performance and touted an Intel Tofino fabric processor (IFP) that adds intelligence to network switching via support for P4 programming tools.

    Finally, the company noted that an Aurora supercomputer, due out in 2022, will exceed two exaflops of peak double-precision compute performance.

    As Intel seeks to regain the processor high ground from rivals such as NVIDIA, ARM and AMD, it’s clear that the future of IT will be defined by a broad range of processor classes that all have unique capabilities. The challenge now is deciding which of those capabilities to actually make available to developers that are largely dependent on the runtime environment that DevOps teams decide to make available.

  • ‘Old Man Yells At Cloud’

    ‘Old Man Yells At Cloud’

    These days, software programmers are increasingly estranged from hardware. Most millennials and Generation Z developers have never had to mount physical servers. They have never been lulled by the gentle hum of a server room chugging along in synchronous harmony. Today, you don’t even necessarily have to build your own PC or worry too much about disk space.

    Instead, cloud-native software developers conveniently offload the server burden to cloud service providers (CSPs). Hosting in a cloud, like AWS or Azure, for example, makes a lot of sense — it allows programmers to hone in on their own specializations. Elastic computing models are cost-effective for startups, and CSPs typically offer a vertical stack with helpful integrated platform-as-a-service (PaaS) and software-as-a-service (SaaS) offerings. However, what are we sacrificing?

    Well, first, research shows the cloud model becomes less cost-effective as a business scales. Second, as more companies transform into technology platforms, cloud usage could better enable their competition. Third, more direct access to bare metal could be an attractive option for certain service providers to increase ownership and control workload optimizations.

    These arguments are bolstered by large companies such as Dropbox, Facebook, Walmart, and others that chose to cut their cloud allegiances in favor of building their own data centers or adopting colocation (in other words, running your own hardware in someone else’s data center, also known as “colo”).

    I recently met with Zach Smith, managing director, bare metal, Equinix, to understand the current state of hardware-as-a-service and the future of hybrid server models. He believes that in the coming years, improved interfaces that access low-level infrastructure will arm IT with better consumer choice that brings them closer to the metal. Yet, to enable this, the hardware world will need to take some lessons from the upper stack regarding transparency, open source community, programmability and improved developer experiences.

    Sidestepping the Cloud

    Public clouds arose as a convenient one-stop-shop to host entire software application environments. As a result, “we have a generation of developers who will never physically touch that hardware,” Smith said.

    The cloud infrastructure services market was worth $39.8 billion in Q4 2020, estimates according to estimates from Statista. Having witnessed the “movement toward verticalization of hyperscale clouds,” Smith now sees the infrastructure-as-a-service (IaaS) market as an industry of high margins and escalating fees. This is echoed by others who seek to reduce rising cloud expenses.

    Though cloud computing usage is growing considerably, the data center isn’t dead. Instead, computing models are morphing and becoming hybridized. “By 2025, 85% of infrastructure strategies will integrate on-premises, colocation, cloud and edge delivery options, compared with 20% in 2020,” predicts David Cappuccio, distinguished VP analyst, in a Gartner report.

    As hybrid multi-cloud approaches emerge, Smith notices a resurgence of attention being paid to infrastructure. Some massive service providers are starting to construct their own data center estates; others, such as Box, now own and lease servers from data center providers.

    Benefits of Bare Metal

    As mentioned above, there are a few reasons to move back to the metal. First, it may simply be more cost-effective for large deployments. In his well-researched article, Antoine Beaupré compared monthly recurring costs for colocation versus an AWS bill, and found the break-even point rested at $100,000 per month, or 150 Amazon m4.2xlarge instances. His research also found that I/O-heavy applications are more costly in the cloud than via colocation.

    “While many services simply launch in the cloud without the organizations behind them considering other options, large content-hosting services have actually moved back to their own data centers,” Beaupré said.

    In addition to cost savings, automating colocation with a programmable layer could also offer more fine-grained control over computing optimizations. “I believe what we’re creating is an operating model for a rapidly changing hardware substrate,” said Smith. He described how bare metal opens up a neutral infrastructure substrate to control physical networking aspects such as cross-connects, IP transits and peering — a benefit Smith half-jokingly called ‘NetDevSecOps.’

    Another argument is that investing in hardware could, ironically, bring sustainability benefits. A report conducted by ITRenew estimates that, of the total embodied energy required within a server’s entire lifespan, 77% is used to manufacture the hardware itself. In contrast, the operational energy used over a server’s lifetime only accounts for 22%.

    Thus, there is an environmental benefit to reversing the trend of a make-dispose economy continually producing new servers. Instead, hardware ownership could encourage a more circular economy. “A circular economy aims to end our take-make-dispose economy, which is a very linear approach, and instead, keep products at their highest possible value for as long as possible,” according to the ITRenew report.

    Learning From the Upper Stack

    How can we build a fundamental non-opinionated infrastructure platform? According to Smith, the industry needs “hardware and networks that are programmable and usable by a developer generation.” To Smith, changing firewalls at a kernel level, altering how the runtime works or controlling the IP stack, should all be operational via API commands.

    Yet, hardware accessibility is still rather opaque and, historically, not as developer-friendly. “How can we take abstraction and community-driven ideas from up the stack?” Smith asks. To enable these changes, he believes the hardware world should model itself on the upper stack in a few key areas:

    1. Transparency: The hardware world is not as easy to comprehend. “When you get to the physical side of switches and routers, it’s still arcane,” Smith said. Increased transparency in functionality and ownership will be required to turn servers into something all developers can operate.
    2. Open-source community: In the same way the software world is built on strong developer communities, hardware will require an open community to normalize things like firmware, boot order and support for differences between drives.
    3. Programmability: “Self-service and programmability are requirements for most things these days — there’s no difference with physical infrastructure,” says Smith. Take the rise of the API economy — today’s mantra is, essentially, if something is possible in a UI, it should be possible through an API. Similarly, if it can be done in a server, it should be controllable with an API.
    4. Quality developer experience: Self-service, programmable infrastructure is just the beginning — interfaces to operate infrastructure require good developer experience. Smith feels that hardware requires an improved buying experience and better life cycle management capabilities. This could be enabled by open source tools like Tinkerbell, a provisioning and workflow engine supported by Linux Foundation.

    Back to the Metal — A Platformer’s Dream?

    All this is not to say the public cloud isn’t valuable – it absolutely is. The cloud has been necessary and has proven resilient enough to support online business and digital life as we know it throughout the pandemic and for years prior. “In 2020 and beyond, most organizations are going to be transforming into a completely cloud-based environment,” said Werner Vogel, CTO, Amazon, during an online AWS summit.

    But, even if most organizations are in the cloud, that doesn’t mean all of their assets will be. Service providers, especially, may cling to hybrid models for more control, to avoid supporting the competition and, perhaps, even to save money. Private data centers could also increase security. In an era where raw computing power is becoming a new asset (quite literally, in the case of cryptocurrency), the server rack is starting to look sexy again.

    “The magic happens when software and hardware work together,” said Smith. But, giving access to physical technology to a generation that has never touched it will require some ingenuity. As Smith says, this could be done by “automating colo” through more usable, standardized interfaces for hardware management.

    “We will see a future in next 10 years where operated software is separated from cloud infrastructure,” Smith said. That’s a bold statement, as such a dramatic change would, in effect, usurp the current cloud provider IaaS market share, altering the distribution and economic model associated with global software.

    While this dream may seem far-fetched, if more platforms decide to cut out the cloud middleman and go it alone, it could materialize.

  • SonicWall Adds Multi-Gigabit Switches to SD-Branch Portfolio

    SonicWall Adds Multi-Gigabit Switches to SD-Branch Portfolio

    SonicWall has added a line of multi-gigabit switches to its software-defined (SD)-branch offering that unifies the management of wireless access points and firewalls that include software-defined wide area network (SD-WAN) capabilities.

    Dmitriy Ayrapetov, vice president of platform architecture for SonicWall, said the switches will replace a series of switches that SonicWall had been reselling from Dell Technologies. SonicWall spun out of Dell at the end of 2018 after being acquired in 2012.

    SonicWall has been making a case for unifying the management of remote office networking via a central management console dubbed Capture Security Center. To achieve that goal, the company embedded SD-WAN capabilities in its firewalls while expanding its portfolio to include wireless access points and switches.

    Ayrapetov said in the wake of the COVID-19 pandemic, interest in centralizing the management of remote offices has increased for two reasons: First, IT teams are no longer inclined to travel to remote offices unless there is an absolute requirement; and second, following the economic downturn brought on by the pandemic, there’s more interest in reducing costs—an issue SonicWall addresses by consolidating the management of networking and security.

    As part of its SD-Branch offering, SonicWall also makes available client software in multiple forms, providing IT teams with capabilities such as URL filtering to block access to malicious websites that can be applied to any device on the network, including systems that employees are using to work from home. That capability makes it possible for IT organizations to enforce cybersecurity policies regardless of where employees are located or what network they are using to access IT resources in a branch office, said Ayrapetov.

    Longer-term, Ayrapetov said it’s clear organizations are moving toward zero-trust networking architecture to better ensure cybersecurity using SD-WANs and other technologies to access cloud applications. However, in the short-term, most organizations will continue to rely on virtual private networks (VPNs) to access IT resources located in a branch office, on-premises data center or public cloud.

    SonicWall has been making a case for its Boundless Cybersecurity platform, which can be broadly applied across a distributed computing environment. At the core of that platform is its Capture Advanced Threat Protection (ATP) cloud sandbox service that relies on deep memory inspection in real-time, machine learning and cyber threat intelligence to identify malware and other potential threats.

    As part of that architecture, SonicWall also provides zero-touch deployment capabilities, which eliminate the need for extensive cybersecurity expertise to deploy its appliances and services.

    It’s not clear to what degree organizations will be consolidating the management of networking and cybersecurity. However, the smaller the organization, the more likely one team is managing both networking and security. As the number of cybersecurity threats organizations face continues to rise at a time when IT budgets are under massive pressure, chances are high many larger IT organizations soon also will be consolidating the management of networking and security.

  • The Consumption-Driven Economy

    The Consumption-Driven Economy