Category: AI

  • IBM z16 Delivers AI and Quantum-Safe Levels of Security

    IBM z16 Delivers AI and Quantum-Safe Levels of Security

    Business transformation and disruption is all about data. But it’s not just about collecting huge amounts of data, it’s what you do with that data and how it can enable you to create powerful insights.

    While many people focus on the cloud unicorns, the reality is that industry incumbents have a huge advantage. These businesses have decades of experience which translates into massive amounts of data and business logic. Where do many of these businesses host their most important workloads and data? IBM zSystems—the system that is best known as “the mainframe.” 

    Today, IBM is announcing its latest IBM zSystem—IBM z16. Two weeks ago, IBM spent time briefing the industry analyst community on the business strategy and technical underpinnings of the IBM z16. We met with a variety of IBM business leaders including executives from the IBM Systems division, Distinguished Engineers and IBM cloud leaders. The most significant updates to the platform are the inclusion of a dedicated on-chip AI accelerator and the addition of quantum-safe cryptography.

    It’s important to understand the size of the IBM zSystems ecosystem. The platform is used by 45 of the 50 largest banks in the world, four of the top five airlines and eight of the top 10 insurers, among others. Likewise, the partner ecosystem is huge, ranging from the largest system integrators to small DevOps- and AI-focused technology vendors. Suffice it to say, some of the world’s largest businesses and governments run on IBM zSystems.

    In this article, I will briefly cover the five biggest takeaways from the pre-announcement industry analyst briefings.

    IBM is Bringing AI to the Mainframe

    Many cloud data warehouse and analytics vendors say “Bring me the data.” IBM is flipping the script, because IBM zSystems already houses clients’ most critical customer and operational data. IBM z16 will be shipped with an on-chip integrated accelerator for AI.

    Everyone knows that AI creates a competitive advantage, so why should you care about the on-chip AI accelerator? The last mile of deploying machine learning/artificial intelligence (ML/AI) models is still a huge challenge. Businesses are investing large amounts of money and skills in developing models; however, latency often kills. The difficulty is that ML/AI models need to be built into applications so that the model can be executed at the point of transaction with virtually no delay. This also must be done at a massive scale! Think about the latency and security concerns if near-real-time banking transactions needed to go from a back-end system to a cloud AI platform to check for fraudulent transactions.

    For those of you who have been following the IBM zSystems platform, IBM’s z14 unveiled a pervasive encryption capability. This encrypt-by-default approach without traditional encryption overhead was enabled through an encryption-dedicated chip. Similarly, AI workloads will be performed by the integrated AI accelerator.

    Your AI Infrastructure is Only as Powerful as Your Software Ecosystem

    Developers and data scientists want to use data, ML and AI tools that they know and trust. AI success requires an open stack of software options, not a walled garden. While IBM has a wide range of AI technologies, including Watson Machine Learning for z/os, IBM Cloud Pak for Data and IBM Cloud Pak for Watson AIOps, just to name a few, an IBM-only approach would be a non-starter for many data scientists.

    The IBM z16 AI accelerator supports a wide selection of ML libraries and compilers. Clients can build and train models in a variety of popular frameworks including TensorFlow, SAS, Microsoft CognitiveToolkit (Microsoft CNTK), PyTorch and Keras, along with others.

    The system uses ONNX, an open source tool to enable interoperability of ML/AI models to prepare and deploy the model on the dedicated AI accelerator.

    What does this mean? You can bring your existing data models to IBM z16. In addition, your teams can use their existing preferred vendors while also working with IBM zSystem’s integrated accelerator chip.

    Cybersecurity Arms Race: Quantum-Safe Cryptography is a Growing Need

    Cybercrime is a multibillion-dollar business. Although fairly routine exploits like phishing remain the number-one money generator for cybercriminals, these gangs and nation-state-sponsored organizations are getting more sophisticated. For example, cybercriminals are investing in AI and ML just like traditional businesses. In some cases, cybercrime syndicate engineering employees believe they’re working for a legitimate company.

    What is quantum-safe cryptography? The research is focused on identifying algorithms that are resistant to attacks by both classical and quantum computers. 

    Why is this a growing area of concern? IBM, along with Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP) all offer quantum computing-as-a-service. While quantum computing has massive potential for applications across a wide range of industries including drug discovery, chemical engineering and combating climate change, keep in mind that cybercriminals are looking at this technology to unlock complex cryptography algorithms. 

    IBM claims that IBM z16 is a “quantum-safe system.” The company uses a variety of different hardware, software and business process approaches to assure data security. For example, the system includes a new crypto express card with quantum-safe APIs to upgrade existing application security. At a firmware level, IBM z16 has a built-in dual signature scheme.

    In addition, for a number of years, the IBM zSystems platform has taken a zero-trust approach to security which includes a bring-your-own-key (BYOK) framework. IBM can never access the keys to your encrypted data–and neither can cybercriminals and/or governmental organizations—instead, those encryption keys remain inside your firewall.

    IBM zSystems Tech Goes to the Cloud; Mainframe Remains On-Premises

    What a difference a nine can make! Five nines mean 99.999% availability while four nines (99.99% availability) result in more than 47 minutes of downtime per year.

    When I work with C-level executives that have a mainframe within their environment, the role of IBM zSystems is always discussed. In many cases, businesses have explored the possibility of transitioning off of the mainframe, but resilience, predictability and security are the reasons why businesses remain on the platform.

    IBM mainframe services are increasingly being offered in the cloud. Most recently, IBM announced IBM Z testing in the cloud. Likewise, IBM’s Cloud Hyper Protect Crypto Services, a key management approach using a specific security chip module, is enabled by IBM zSystems technology.

    However, you shouldn’t expect mainframe-as-a-service. Between latency, security, resilience, governance and compliance, it’s going to be a long time until you migrate and/or send your mainframe workloads to the cloud.

    As the author of Hybrid Cloud for Dummies and Cloud for Dummies 2nd Ed. (among other books and articles) I can confidently say that the cloud is not a panacea. The business case for moving many mainframe workloads to the cloud doesn’t exist. 

    Digital Transformation Means More Than Slick Apps–Plumbing Matters!

    Don’t get me wrong, mobile check deposits, airline ticketing and contactless hotel check-ins are great, but in many cases, they rely on older systems that have not been updated. 

    A few weeks ago while traveling for work, my flights were delayed for hours. I saw that another flight was leaving within the hour. I was already at the airport and figured I could rebook it via the airline’s app. Of course, it wasn’t that easy—I finally went to the ticketing desk and told the agent that I’d like to book a different flight. The agent then went to work; they spent a long time entering information into their system, talking to a colleague, asking for help and asking me several questions. After about ten minutes my flight was finally rebooked.

    This airline has a great app experience when things are running smoothly (so great, in fact, that I prefer to book my travel on the app rather than the computer). However, even a slight disruption caused everything to go off the rails. It’s clear to me that the company still relies on old, broken processes and outdated technology.

    My Perspective

    Moving AI closer to the data just makes sense. The last mile of AI—moving models into transactions—continues to be a huge challenge. I work with many businesses that have tons of data and do an excellent job retrospectively analyzing that data. However, real-time data-driven decision-making remains a challenge because of latency and the fact that moving data between on-premises and cloud systems raises security and governance risks (in addition to costs).

    Quantum-safe security is a growing concern for highly regulated organizations in the banking, insurance and governmental segments. Over the last several years, financial industry CIOs and CISOs/CSOs at large financial services organizations were focused on establishing a zero-trust approach to security. However, as attacks become more sophisticated, security teams are starting to perform risk assessments on their approach to cryptography. The inclusion of quantum-safe cryptography is important for IBM zSystems customers.

  • CSS-Tricks Sells Out | Peloton Spins Out | ARM Axes 1,000

    CSS-Tricks Sells Out | Peloton Spins Out | ARM Axes 1,000

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: CSS-Tricks gets bought by DigitalOcean, Peloton pedals while its business burns, and Arm lays off up to 15% of staff. (more…)

  • What Does AIOps Mean for SREs?

    What Does AIOps Mean for SREs?

    It seems SREs are of two minds when it comes to AIOps. On one hand, AIOps’ potential is pretty exciting. By automating complex workflows and troubleshooting processes, AIOps could make your life as an SRE much easier.

    But on the other hand, some SREs may choose to view AIOps with disdain and distrust. They might think of AIOps as just another trendy buzzword that doesn’t live up to the hype and that can become a distraction from the SRE tools that really matter.

    Which perspective is right? Should SREs embrace AIOps with open arms, or should they resist marketers’ efforts to position AIOps as the latest, greatest tooling innovation in the IT industry?

    Those are subjective questions that we can’t answer definitively, but let’s at least gain some perspective by examining what AIOps means for SREs.

    What is AIOps?

    If you keep up with IT buzzwords, you’ve probably at least heard the term AIOps—which is short for artificial intelligence for IT operations—and that it refers to the use of AI and machine learning (ML) to help automate IT operations workflows.

    The big idea behind AIOps is that, by using AI and ML to perform advanced analysis of large volumes of data from IT systems, IT and SRE teams can solve complex problems more efficiently than they could using a traditional, manual approach.

    AIOps can, for example, help to surface the root cause of a performance issue in a complex, multilayered environment like Kubernetes. Or it could make recommendations about how best to resolve an incident.

    AIOps entered the IT lexicon in 2016, when Gartner coined the term. Now, six years later, it’s a relatively well-established tool domain.

    How SREs View AIOps

    Despite the fact that AIOps has been around for some time, it doesn’t yet appear that many SREs have bought into the AIOps revolution. Catchpoint found in a 2021 survey that just 7.5% of SREs reported that AIOps tools delivered high value to their organizations.

    It’s unclear exactly why there are such low rates of excitement about AIOps, but we’d speculate that there are a few key factors at play:

    • AIOps is a new term for an old idea: Many monitoring and observability tools have included at least basic AI and ML analytics features for a long time, even before tool vendors slapped the AIOps label on their products. SREs probably realize this and, to some extent, view AIOps as an effort by marketers to rebrand functionality that is not actually new.
    • AIOps is hard to implement: Setting up an AIOps tool requires integrating it with diverse data sources and customizing it to fit your workflows and environment. It’s possible some SREs view this setup work as requiring more effort than it’s worth.
    • AIOps can’t replace human insight: While AIOps tools may be useful to a point, it would be unwise to place blind trust in AIOps-based analyses or recommendations. For this reason, some SREs may believe that AIOps encourages organizations to rely too heavily on automated tools at the cost of the expert analysis and perspective that only SREs can provide. (This is similar to the fact that it sometimes makes sense to prioritize human intuition and expertise over playbooks.)

    From an SRE’s perspective, then, AIOps may appear over-hyped, overly complicated and underperforming compared to traditional approaches to SRE.

    What SREs can Gain From AIOps

    SREs’ wariness toward AIOps is valid—but only to a point. It’s important not to let suspicions about the limitations of AIOps turn into excuses not to use AIOps at all. AIOps has some value to offer to SREs, even if it’s not perfect.

    For example, AIOps can play a role in reducing toil. To the extent that AIOps tools can recognize complex patterns or discover interrelated data sets more quickly than human engineers, AIOps reduces the time SREs have to spend manually troubleshooting problems or poring over complicated information.

    AIOps also helps to enable a more proactive approach to monitoring and incident management. If AIOps tools can alert SREs to emerging issues before they would otherwise recognize them, AIOps can help the SREs to get in front of the problems before they turn into true incidents. That’s better for SREs and end users alike.

    There is also an argument to be made that AIOps can help SREs do more with fewer engineering resources. If you can use AI to automate some aspects of monitoring and incident response, you can maintain the same levels of availability and performance with fewer human engineers on hand.

    Conclusion: AI Won’t Replace SREs—But it can Help

    None of the above is to say that AIOps can replace SREs, or that it magically solves every problem SREs face. Anyone who believes AIOps is a silver bullet has bought into the marketing hype to an unhealthy degree.

    Nonetheless, AIOps tools do offer value to SREs by making their jobs easier in some respects and improving reliability outcomes.

    So, while it’s wise to maintain a healthy perspective about the limitations of AIOps, SREs shouldn’t rule out AIOps tools as one way to improve reliability engineering.

  • Tabnine, GitLab Build Custom Apps Faster With AI

    Tabnine, GitLab Build Custom Apps Faster With AI

    Tabnine today revealed it has integrated its code completion tool with the GitLab continuous integration/continuous delivery (CI/CD) platform. Tabnine’s platform employs artificial intelligence (AI) to accelerate application development.

    Brandon Jung, vice president of ecosystems and business development at Tabnine, said the goal is to make it easier for developers to automatically write code based on custom models created using AI. Those models would reside in the CI/CD platform provided by GitLab. Developers can now create custom models of their private source code, he noted.

    The Tabnine AI assistant can then be trained on a customer’s private code, style and security best practices that have been defined by an internal DevOps team, added Jung. That assistant, accessible from within a developer’s integrated development environment (IDE), is conversant in 35 programming languages.

    The code completion tool doesn’t eliminate the need for developers to write code; instead, it completes code once developers start writing it based on the model stored in the CI/CD platform.

    Nima Badiey, vice president of global alliances for GitLab, said the overall goal is to increase developer productivity at a time when organizations are launching more application development initiatives than ever. Many of those applications are being constructed using multiple programming languages that each individual developer may not have equally mastered.

    In addition, an AI model will, over time, improve the quality of the code being created, noted Badiey. That should make it easier to review code that is more accurate because it was created with the aid of a machine, he added.

    In general, GitLab has been moving to incorporate AI within DevOps workflows. The company last year acquired UnReview, a provider of a tool that uses machine learning algorithms to identify which code reviewers to assign to a project based on both the quality of their previous efforts and their current workloads.

    It’s not clear to what degree AI will be pervasively embedded within DevOps workflows. However, it’s now just a matter of time before algorithms are more widely employed. Organizations that employ DevOps today devote significant resources to maintaining platforms that could be simpler to manage if more tasks and processes were automated. In theory, more resources could be then devoted to building applications versus maintaining DevOps platforms.

    One way or another, more organizations are moving to increase developer productivity. Developers are both hard to find and retain, so organizations have a vested interest in finding ways to increase the rate at which an existing pool of developers can build applications capable of driving a wider range of digital business transformation initiatives. Of course, an increase in the rate at which applications are developed also has significant implications for the rate at which they might need to be deployed.

    Not everyone in IT is, of course, entirely comfortable with AI. Nevertheless, most DevOps organizations are highly committed to automating as many processes as possible. The only thing that will be different going forward is determining to what degree those processes will be automated by humans versus a machine.

  • IBM ‘is Ageist and Sexist’ | IBM Mainframe-aaS | IBM Vaccine Mandate

    IBM ‘is Ageist and Sexist’ | IBM Mainframe-aaS | IBM Vaccine Mandate

    Think! Welcome to Big Blue Week here at The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: IBM’s employment practices scrutinized, IBM z/OSaaS breaks cover, and IBM encourages staff back to the office.

    1. ‘Dinobabies’ and ‘Dated Maternal’ Workforce Woes

    First up this week: IBM’s defense in an employment age discrimination lawsuit looks to be shredded. Internal emails “appear to show high-level discussion about paring the ranks of older employees.”

    Analysis: Not a good look

    According to one of the plaintiff’s lawyers, unnamed “top IBM executives were explicitly plotting with one another to oust older workers … to make room for millennial employees.” On the face of it, they seemed to be doing the polar opposite of the Age Discrimination in Employment Act.

    Noam Scheiber: Making ‘Dinobabies’ Extinct: IBM’s Push for a Younger Work Force

    Top IBM executives were directly involved in discussions about the need to reduce the portion of older employees … sometimes disparaging them with terms of art like “dinobabies.” … Documents made public by a Federal District Court … show executives discussing plans to phase out older employees.
    …
    The newly unsealed documents, which quote from internal company emails … show top IBM executives specifically emphasizing the need to thin the ranks of older workers and hire more younger ones. [One] email refers to IBM’s “dated maternal workforce,” an apparent allusion to older women, and says: “This is what must change. They really don’t understand social or engagement.”
    …
    Adam Pratt, an IBM spokesman, [claimed,] “IBM never engaged in systemic age discrimination.” [He] said that some of the language in the emails “is not consistent with the respect IBM has for its employees” and “does not reflect company practices or policies.”


    Yikes, ageism and sexism? Experienced staff are valuable, says MrBanana:

    I did some time at IBM [and] remember being invited to a meeting with a new group, headed by a new “thinker,” who had something to present to myself and a colleague. We walked past the foosball table to the meeting room. Naturally the room had no chairs, and was covered in multi-coloured Post-it Notes. Once they had gone through their mood boards and the non-linear presentation of this new idea, I turned to the other old geezer in the room. “Didn’t we look at something like this 8 years ago?”

    “Yeah, we worked out that compute power just to run the UI for one session wouldn’t be available for a couple of decades.” The children were somewhat disappointed, and (fortunately) we were never asked back to their romper room.


    But experience can be both a blessing and a curse, learned aahortwwy:

    I experienced both sides of this in five years at a company in hypergrowth. … I learned so much at that job: … In year 1, I was determined to accomplish X and ignored the skeptical comments of some colleagues who’d been at the company longer than me. I failed and wasted some time. In years 4 and 5, new colleagues were determined to accomplish X and ignored my skeptical comments.


    2. IBM Wazi as-a-Service Confusion

    “Mainframe” is not a dirty word. It’s secure, vertically scalable, and it put the VM in VMware. Now, what IBM Wazi did for on-prem LPARs, Wazi aaS promises to extend to hosting z/OS instances.

    Analysis: Mainframe-as-a-service reinvented

    Mainframes and cloud-native DevOps need not be incompatible. To modernize legacy apps using hybrid cloud instances for dev/test—and even prod—can be a big productivity boost. But isn’t it weird that IBM spun out its mainframe-as-a-service zCloud offering only a few months ago?

    Mike Vizard: IBM Shifts Mainframe App Testing to the Cloud

    While many workloads over the years have been migrated off mainframes or simply retired, the applications that run on z/OS today are still among the most mission-critical that organizations run. … IBM this week revealed it has added the ability to set up testing environments on the IBM Cloud for z/OS mainframe applications.
    …
    IBM plans to make IBM Wazi as-a-Service (Wazi aaS) for z/OS generally available in the second half of this year, [which] will enable DevOps teams to self-provision an instance of a z/OS virtual server on a secure private space running on IBM Cloud. … At the same time, it will become easier to modernize mainframe applications by making them more accessible to other applications via a set of secure APIs … using a cloud-native set of DevOps tools that run in a Kubernetes environment.
    …
    Modernizing mainframe applications is a less complicated endeavor than migrating them. The challenge is that building, deploying, maintaining and securing all the APIs required to do so represents a significant amount of effort.


    OK, boomer, why are we talking about 1960s mainframes on devopsy.kinsta.cloud? Here’s ArchieBunker:

    Mainframes are different beasts altogether, almost alien when you compare them to PC hardware. You have binaries written for the S/360 system in the 1960s? Good news you can still run them natively without a recompile.

    You think virtualization and containers are a hot topic? IBM was running multiple OS instances inside a VM in the 1970s. … The engineering is incredible.


    But z/OSaaS? It’s déjà vu time for Beleagured Greybeard:

    “Cloud Managed Services for z” AKA “zCloud” has been around for over 10 years. … IBM offloaded it with the Kyndryl spin, so its odd that they now want to re-invent it.
    …
    If it’s a strategic platform, why didn’t they hold onto it instead of spinning it off to Kyndryl? Sounds like yet more daft leadership decisions.


    3. IBM’s Return to Work Plan has Vaccine Mandate

    IBM wants to “keep employees safe” as they return to office work after being fully remote. In practice, this means a vaccine mandate.

    Analysis: IBM Digital Health Pass is divisive to vocal minority

    Although there are exemptions on medical or religious grounds, IBM has promised to place dissenters on unpaid leave. As you might expect, the policy has a few employees up in arms.

    Cassie Werber: How Salesforce, IBM, and HSBC are approaching return to work

    When the pandemic hit, and IBM moved 95% of its workforce to fully remote, it was pleasantly surprised to discover no drop in productivity. … Still, they planned to return to offices.
    …
    The company saw this as a way to keep employees safe. … One example … is its development of the IBM Digital Health Pass as an alternative to vaccine cards or paper vaccine certificates, which they use to make sure all the staff that come into their offices are vaccinated.
    …
    Arvind Krishna, IBM’s CEO, meanwhile came up with what he called the IBM Work From Home Pledge. [It] included commitments to flexibility, kindness, sensitivity to family, setting boundaries, and acknowledged video fatigue. It concluded with a commitment to “be connected.”


    Tricky to “be connected” if you’re not allowed back in the building. Anne D’Innocenzio, Heather Hollingsworth, Tali Arbel, Mae Anderson, Ken Sweet, Matt O’Brien and Dee-Ann Durbin:

    IBM software engineer Justin Albano said his company recently notified employees that it would be revoking badge access to workers who aren’t vaccinated or haven’t proved that they are. “We’re not able to enter an IBM office at this time,” said Albano.
    …
    He said some workers have successfully sought religious exemptions but “my thought process was I shouldn’t have to explain to my company my faith” to get out of a policy that’s “immoral in its foundation.”


    What’s the alternative? With a perspective, it’s Chris Haney:

    Or, get the vaccine. Protect yourself and the people around you. You people really are detached from reality.


    The Moral of the Story: Those who know do not speak; those who speak do not know.


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Marcin Wichary—cc:by; leveled and cropped (ask your parents)

  • Unreliable Server Scare | Information Batteries | ARM IPO PDQ

    Unreliable Server Scare | Information Batteries | ARM IPO PDQ

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: We worry about chips failing randomly, we ponder a new way of thinking about workload shifting, and we grok Arm’s IPO.

    1. Smaller Features Lead to Random Failures

    First up this week: Could cosmic rays be causing increasing numbers of errors in cloud servers? As chips get faster and more densely packed, the worry is that undetected bit-flips could cause serious problems.

    Analysis: How can you mitigate the problem?

    As workloads grow in number and complexity, this is becoming a real issue—not merely theoretical. (See also: DevSecOps worries such as Rowhammer.) Think about where the critical data are and how to detect random errors. Even consider running important workflows in lockstep pairs.

    John Markoff: Tiny Chips, Big Headaches

    Imagine for a moment that the millions of computer chips inside … the largest data centers in the world had rare, almost undetectable flaws. … As the tiny switches in computer chips have shrunk to the width of a few atoms, the reliability of chips has become another worry for the people who run the biggest networks. … There is growing evidence that the problem is worsening with each new generation of chips.
    …
    Both Facebook and Google have published studies describing computer hardware failures whose causes have not been easy to identify. … Facebook researchers noted that some processors would pass manufacturers’ tests but then began exhibiting failures when they were in the field. … Smaller transistors, three-dimensional chips and new designs that create errors only in certain cases all contributed to the problem … according to Google.
    …
    Intel executives said they were familiar with the Google and Facebook research papers and were working with both companies to develop new methods for detecting and correcting hardware errors. [But] computer engineers are divided over how to respond to the challenge.


    Have we hit a wall? So asks Thutmose V:

    For more than 50 years, semiconductor manufacturers have been trying to keep up with Moore’s Law, which … has effectively served as the target for semiconductor engineers to push them forward. … The size of some parts of the transistors is now 10 nanometers or less. 10 nanometers is about equal to the width of 100 silicon atoms—100 atoms is not very many.
    …
    When things are that small, does quantum randomness start to be noticeable? Normally if the odds of something failing is a trillion to one, you don’t need to worry. But if you have 10 billion transistors operating at a clock speed of 3 billion times a second, a one in a trillion chance of failure is nowhere near good enough. … It is a miracle these things work as well as they do.


    We’ve known about the problem for years, but only now is it becoming reality, says dsgrntlxmply:

    We observed certain network failures to be caused by corruption of an on-chip routing table memory in a SoC that lacked parity protection for these tables. The failure rate was low, but was higher in units that were deployed at higher altitudes. … The problem was eventually mitigated by software table sweeps, and in newer SoC generations, by parity in critical tables.

    These Single Event Upsets are said to be caused by muons generated by high energy cosmic ray interactions in the upper atmosphere. Third semester undergrad physics.


    2. Mind-Bending Speculative-Compute Research

    Many of us are familiar with delaying a workload until a time when servers are idle—or when energy is cheap. But some fascinating research asks: What if we could predict a future workload and perform it in advance?

    Analysis: “Information batteries” can be cheaper than Li-ion

    The researchers argue that pre-caching even parts of a workload’s output when energy costs are low—or even negative—can save far more money than traditional grid-scale storage such as batteries or pumped hydro. For example, speculatively transcoding video versus generating it on the fly, or predicting future machine learning training workloads.

    Caitlin Dawson: A New Way to Store Sustainable Energy

    Renewable energy has an intermittency problem. … What if surplus renewable energy could be stored as computation? … That’s the thinking behind “information batteries,” a new system proposed by … Barath Raghavan, an assistant professor in computer science at the USC Viterbi School of Engineering … and Jennifer Switzer, a Ph.D. student from UC San Diego.
    …
    When renewable energy is available in excess, it is used to speculatively perform computations. … The stored computed results can then be used later when green energy is less plentiful. … For instance, every day, YouTube data centers transcode more than 700,000 hours of videos to different resolutions. Many of these computations are predictable and can be performed at a time when there is excess green energy.
    …
    [But] computations that are completed in advance do not need to match exactly with the computations completed at a later time. … The challenge, said the researchers, is determining what computation to perform, where and when, and how … to efficiently retrieve the results later. … It is only possible in some workloads and in some contexts. But Raghavan believes with improved prediction and integration into large systems, the technology points towards a promising future alternative.


    This isn’t just batch computing with a fancy name. cognizant_ape explains:

    On AWS we do a lot of batch processing when the data centers are low on traffic. This is just the standard SPOT instances which you can trigger once the cost goes below a certain amount and then end when it goes about a certain amount.

    The big issue is being effective with I/O and setup time. The article seems to be getting ambitious to the point of a distributed compiler with a flexible optimization metric.


    Cut to the chase. It hinges on prediction, says bradley13:

    Predicting loads is easy, but predicting the computations that will make up those loads? Not so much. … Imagine: Most employees start work in an hour — let’s predict which database queries they will generate. Your success rate better be high.


    3. Confirmed: Arm will IPO, Not be Sold to Nvidia

    SoftBank and Nvidia have bowed to the inevitable. Arm, designers of ubiquitous chip cores, will IPO sometime in the next 12 months.

    Analysis: The only sane choice, but ecosystem risk remains

    As I’ve said before, it seemed everyone was against the idea of Nvidia owning Arm. It put the kibosh on Arm’s open licensing stance, dooming the ecosystem to scraps from the table as Nvidia grabbed first dibs on everything. Now regulators must turn their attention to the very real risk that Nvidia will still acquire Arm anyway—via the stock market.

    Richard Waters, Arash Massoudi, James Fontanella-Khan and Antoni Slodkowski: $66bn sale of chip group Arm to Nvidia collapses

    The deal, which would have been the largest ever in the chip sector, was set to give … Nvidia control of a company that makes technology at the heart of most of the world’s mobile devices. A handful of Big Tech companies that rely on Arm’s chip designs, including Qualcomm and Microsoft, had objected to the purchase [arguing] that Nvidia would get an unfair advantage by having first rights to Arm’s technology.
    …
    Jensen Huang, Nvidia’s chief executive, hoped to use Arm’s processor designs to cement his company’s growing role in data centres. … Nvidia abandoned its pursuit of Arm at a board meeting on Monday, said a person familiar with the discussion.
    …
    The deal’s failure brought an immediate management shake-up at Arm, with chief executive Simon Segars replaced by Rene Haas, head of the company’s intellectual property unit. … In the UK, where politicians view Arm as a strategic national asset, attention is set to shift to whether the company will be listed in London [but] Haas said … no decisions had been made on where Arm would be listed.


    Reminding us why this matters to DevOps, here’s Timothy Prickett Morgan:

    Now there is no question that Arm Holdings will be going public. … The plan is to do so before the end of [its] fiscal year in March 2023. … Arm Holdings will be returning to its IP licensing roots, having spun off some uncore business back into SoftBank, and seeking out new markets as it has been able to do with hyperscalers [and] cloud builders.
    …
    When Intel was worried about AMD Epyc chips, what it really needed to worry about was boosting price/performance to keep Arm CPUs out of the hyperscaler and cloud builder datacenters. [But AWS] has Graviton Arm server chips on a pretty regular schedule, and you can bet Masayoshi’s last yen that Graviton4 will debut at re:Invent 2022 this year.


    But what happens next? Pun it up, HereAndGone:

    If Softbank is going to take ARM public, Nvidia will find it much cheaper simply to acquire a controlling interest. … Looks like a duck, walks like a duck, but cheaper than a duck—while ducking regulatory issues. This plan is not quacked up to solve what it pretends to solve and will likely prove most fowl.


    The Moral of the Story: Fall seven times and stand up the eighth


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Nicole Geri (via Unsplash; leveled and cropped)

  • Techstrong Research DevOps Radar Report 2022

    Techstrong Research DevOps Radar Report 2022

    You can’t control the pace of change; instead, consider that we are in a process of reinvention—both with regard to the way organizations use technology and how vendors position their offerings. At the same time, it’s never been more important for IT organizations to have complete visibility into the way technology is used from both a security and operations perspective. 

    This need to embrace reinvention while continuing to have broad and deep insight across all technology platforms led me to create a 2022 DevOps Radar Report. In this article, I highlight three near-term priorities that must be on every technology team’s radar. In addition, I’ll point out some long-range radar trends that should be on your agenda.

    Collision Warning! Your Current Radar View

    API Security

    APIs are the lifeblood of the cloud-native and app-based economies. Increasingly, poor API security across a variety of applications and systems puts your organization at risk. A compromised API or the use of unsupported or outdated APIs can prove to be an easy attack vector for cybercriminals. In addition, API vulnerabilities are calling into question how organizations approach application security and the usefulness of web application firewalls (WAF) and encryption protocols. Therefore, API security is rapidly becoming a top security priority for enterprise security teams and software professionals. 

    Failing to adequately manage the security of APIs creates a host of new challenges and questions that you must consider. For example, who is using which APIs? Are your APIs updated? Did your DevOps team evaluate the security of your APIs during their selection process? Have you formalized the way your organization is evaluating API security?

    It is increasingly clear that an API breach can lead to the downfall of your entire digital software strategy—but it doesn’t have to be this way.

    At Techstrong Research, we segment the market into “pure-play” API security vendors and larger security vendors with specific API security offerings. Our team will be publishing a Market Vista report on the API security landscape soon, so if you want to share your opinion or thoughts, please reach out.

    Below is a brief list of some of the vendors we are tracking in the API security market. Of course, expect consolidation—some of these pure-play vendors will likely be acquired in 2022.

    Pure-play API security vendors we’re tracking include:

    General security vendors with API security offerings that we’re tracking:

    Observability is Evolving to AIOps

    Modern applications are composed of containers and APIs that your organization can’t control, and into which visibility is difficult. Monitoring and application performance management (APM) solutions aren’t new; for many years log vendors built their businesses on collecting log files and sending alerts … so what’s changed?

    First, logs without context are meaningless. You need to understand upstream and downstream dependencies. Additionally, speaking for myself, I’m a mere mortal—if your teams are also made up of mere mortals, there is no way they can ingest, correlate and understand the tsunami of incoming DevOps log data quickly enough to take proactive measures.

    Besides the need to understand context, we are dealing with multiple cloud and on-premises platforms that each have their own logging protocols. Ultimately, the goal of observability and AIOps is to understand what’s happening across and among all these environments and technologies so you can detect and resolve issues as quickly as possible to keep your systems efficient and reliable and your customers happy. 

    Humans are the Biggest Security Vulnerability

    As an industry analyst, I see our clients pouring resources into cybersecurity efforts to prevent zero-day attacks and sophisticated advanced persistent threats (APTs). These threats, while real enough, are mostly external and may not pose as great a threat as many organizations believe. In fact, the vast majority of security vulnerabilities are caused by:

    • Well-meaning employees making innocent mistakes, including:
      • Falling victim to phishing attacks
      • Poor corporate password policies
      • Cloud misconfigurations and open ports
    • Technical debt—relying on older software and hardware that was never designed for a 100% mobile workforce, for example

    The first step in combating this problem is improving security awareness and building a cybersecurity culture. That starts with everyone in your organization understanding that security is a team sport and that it is a 24/7/365 responsibility. Even if you work from home; even when you’re cooking dinner while checking email on your phone, you need to have your “corporate security/Is it safe?” hat on.

    At the same time, machine learning, AI and automation have a role to play. There are emerging offerings that help enforce DevSecOps best practices and help security teams reduce the constant noise so they can focus on the most vital threats.

    Longview Radar: Plan Your Strategy Now

    To support emerging business models, what do DevOps leadership teams need to be thinking about from a long-term perspective?

    • Web3/Web 3.0 and the potential for change to the internet business model. But there is still a ton of opportunity with Web 2.0.
    • How will you support low-code/no-code? As a business model, vendors are selling directly to business units—it’s likely that your IT team has an existing solution to solve the business pain point(s).
    • Packaged software offerings that include blockchain, NFTs and other digital ledger technologies.
    • How to upskill teams, both from a technical perspective and a business perspective.
    • Providing developer teams with the tools they need while establishing guardrails to enforce governance and compliance requirements.
    • How to manage cloud costs in a predictable way no matter how many cloud vendors your organization uses.

    Conclusion: You Can’t Put the Brakes on Change

    Attempting to control change and the speed of reinvention is futile. You need to get comfortable riding the waves of industry change, or you’ll be swept away! Every IT leader needs to learn how to roll with and manage the ebb and flow of business, IT and personnel change.

    The best place to start is by building a successful culture within your organization. Although we often talk about technology-focused solutions to business problems, the first step on this journey doesn’t involve tech at all; it involves helping IT and business teams understand each other’s pain points and learn how they can each support a continual reinvention process.

    I will be tracking these incoming radar targets and other associated trends throughout the year and I look forward to your input. Do you think I missed something? Have a question or want to share what your organization is doing to support the need for continual reinvention? 

    Feel free to engage with me – @DDKirsch, LinkedIn or, of course, email. 

  • Google FLoC is Dead | Meta AI Supercomputer Lives | ARM Deal is Dead

    Google FLoC is Dead | Meta AI Supercomputer Lives | ARM Deal is Dead

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Google’s FLoC proposal is dead, Meta/Facebook is buying RSC—a huge AI supercomputer, and Arm “will IPO” instead of selling to Nvidia.

    1. Forget ‘Federated Learning of Cohorts’—Long Live ‘Topics’

    First up this week: Google bows to the inevitable. Third-party cookies will soon go away, because people are fed up with being tracked. But Google’s FLoC proposal wasn’t the answer.

    Analysis: FLoC is a flop

    Google doesn’t want to invade your privacy. But Google does want to sell well-targeted ads. The key difference between FLoC and Topics is that FLoC buckets were automatically generated—and hence opaque, risking overlap with “protected categories”—e.g., race or gender. But the topic classifications in the Topics API are curated and have user-visible, descriptive names.

    Frederic Lardinois: Google kills off FLoC, replaces it with Topics

    Google’s controversial project for replacing cookies for interest-based advertising … is dead. In its place, Google today announced a new proposal. … Your browser will learn about your interests as you move around the web.
    …
    When you hit upon a site that supports the Topics API for ad purposes, the browser will share three topics you are interested in. … The site can then share this with its advertising partners to decide which ads to show you. … Users will be able to review and remove topics from their lists [or] turn off the entire Topics API.
    …
    It’ll remain to be seen if other browser vendors will be interested in adding the Topics API. [But] they all quickly turned a cold shoulder to FLoC. … The plan is to start trialing the Topics API at the end of this quarter.


    A minor tweak and rename? “Yes and no,” equivocates mananaysiempre:

    Yes, it still amounts to building user-tracking functionality into user agents. No, it’s not the same approach.
    …
    FLoC was doing unsupervised clustering of users, whereas Topics allocates users into predetermined clusters. This makes it potentially more transparent and also seems designed to work around the “protected categories” objection.


    But Google has a long way to go to convince garett_spencley:

    I’m glad FLoC is being ditched, but as an end-user I’m still not asking for this Topics nonsense. It doesn’t solve any problem that I have. It solves problems that the advertising industry has.
    …
    I’ll choose to use browsers that don’t support this.


    2. Machine Learning at Metaverse Scale is Expensive

    Zuckerberg’s new toy is RSC: A $400 million Nvidia-based AI supercomputer. And that’s just Phase One.

    Analysis: 3.8×1018 integer operations per second is … a lot

    If you rely on Open Compute Project systems for your DevOps, know that it’s no longer the bee’s knees over at Hacker Way. Meta/Facebook is investing eye-watering sums into a custom configuration of off-the-shelf hardware.

    Timothy Prickett Morgan: Meta Buys, Rather Than Builds And Opens, Its Massive AI Supercomputer

    If you thought it took a lot of compute and storage to build Facebook’s social network, you ain’t seen nothing yet. … To build the Metaverse … is going to take an absolutely enormous amount of supercomputing power.
    …
    But what is a surprise, given Facebook’s more than decade long focus on … opening up its hardware designs through the Open Compute Project, is that the … Research Super Computer [RSC] system is built from … commercially available … servers, storage, and switches. [It] probably comes down to … working a deal with some vendors who want the Facebook publicity and are willing to work the price and the parts availability.
    …
    [So Meta bought] 760 DGX A100 systems from Nvidia—which each … have a pair of AMD “Rome” 64-core Epyc 7742 processors … 1 TB of main memory … eight A100 GPU accelerators [and] eight 200 Gb/sec HDR InfiniBand controllers—all-flash storage from Pure Storage, and … caching servers from Penguin Computing. … At list price this machine would cost around $400 million … Phase one of the RSC machine is rated at 59 petaFLOPS … (FP64), and 118.6 petaFLOPS FP64 using the Tensor Core matrix engines. … With INT8 processing on the Tensor Cores, which is needed for AI inference, the phase one machine is rated at 3.79 exaOPS.


    How much??? It doesn’t seem like news to kleiba:

    I used to work at a university where my professor had been in automatic speech recognition for a long time, but basically gave up on that line of research about 10 years ago because he figured that universities simply cannot compete budget wise with the big industry players. I suppose the same will soon be true for most ML-related areas of research sooner or later.
    …
    Already, a substantial amount of research innovation in [Natural Language Processing] and [Computer Vision] has been coming from big companies in recent years. Of course there is a discussion to be had about what that means for society at large.


    But 3.79 exoOPS? That’s the way the wind is blowing, according to sungazer:

    Give it 15 or so years—this level of computation will be on your desktop. If desktops still exist.
    …
    Or desks.


    3. Arm Bid: No Legs

    Speaking of Nvidia, as I’ve said before, the Arm merger really shouldn’t go through. And it’s looking less and less likely. ARM chips are an increasing fixture in the data center—especially those that value “performance per Watt.”

    Analysis: ARM M&A DOA—IPO PDQ

    A veritable hornets’ nest of Bloomberg journalists have put their names to an anonymously-sourced report. The story argues that the proposed Nvidia buyout of Arm from SoftBank Group is dead. Next stop: IPO.

    Ian King, Giles Turner, Peter Elstrom, Dina Bass, David McLaughlin, Ruth David and Dinesh Nair: Nvidia Quietly Prepares to Abandon … Arm Bid

    According to people familiar with the matter … Nvidia … doesn’t expect the transaction to close. … SoftBank, meanwhile, is stepping up preparations for an Arm initial public offering.
    …
    If Nvidia manages to get the deal over the line, it would be a massive coup. … But it will be an uphill fight. … The world’s biggest tech companies rely on Arm technology, and they fear they could lose unfettered access under Nvidia.
    …
    Within SoftBank, there are factions. [And] the ordeal has created divisions within Nvidia. … “We remain hopeful that the transaction will be approved,” a SoftBank spokesperson said in an emailed statement.


    Post-Brexit Britain would no doubt like to keep Arm independent. Here’s simonh:

    Arm has I think ⅔ of its employees in the UK—mainly Cambridge—and they’re expanding the campus there. The Semiconductor Physics Group at the Cavendish Laboratory at the University is at the cutting edge globally with 18 PhD projects currently.
    …
    Guess where scads of Arm’s top scientists are recruited from? Arm and the University are joined at the hip. That company’s not going anywhere.


    More broadly, it’s good news, thinks hatchet:

    Hopefully this means NVIDIA will focus on RISC-V instead. This will be good for everyone.


    The Moral of the Story: Zeal should not outrun discretion.


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Reed Geiger (via Unsplash)

  • Investing in Artificial Intelligence: How MLOps Drives Enterprise AI Wins

    Investing in Artificial Intelligence: How MLOps Drives Enterprise AI Wins

    Insight Partners had a banner year in 2021, with more than $50 billion in capital commitments across over 200 investments. While we invest in founders across a wide spectrum of businesses, this series focuses on outlining our theses on four verticals that we’re particularly excited about in 2022: Artificial intelligence (AI), fintech, cybersecurity and healthtech. 

    Artificial intelligence has remained a super-hot category for good reason: It has the potential to transform nearly every industry and business.

    At Insight, we’ve long been bullish on the many use cases for AI. In the past year, we’ve invested in image recognition software from Netherlands-based ScreenPoint Medical, which improves early detections of breast cancer, Covera Health, which provides a quality analytics platform to reduce medical errors in radiology, CARTO, which helps companies use and understand spatial analytics, and Laminar, a cloud data security platform to continuously monitor and protect against data leaks – among other game-changing companies.  

    In total, Insight invested in 49 different companies across a broad spectrum of artificial intelligence and machine learning use cases in 2021, which represents a 172% increase from the year prior.  

    As we look ahead into 2022, we expect artificial intelligence tools to continue to dominate. We see the ecosystem dividing into three primary categories:  

    • Layer 1 – Base platform companies: Algorithms; frameworks; infrastructure and workbenches for creating ML systems   
    • Layer 2 – Cross-industry capability companies: Turnkey machine learning-based systems that solve specific problems spanning multiple industries (e.g., cybersecurity)  
    • Layer 3 – Industry-specific companies: Applications powered by prediction or classification systems that target specific, niche use cases in one domain    

    Companies and investors will find valuable AI/ML software across all three layers. At Insight, we initially focused on layers two and three. We invested in startups creating robust ML systems that addressed specific problems, either vertically (like credit underwriting company Zest AI) or horizontally (like cybersecurity company SentinelOne). We thought that economic moats were hardest to build at layer one; in part as a result of robust open source ecosystems and because large public cloud vendors deliver many of these tools at low prices.

    Insight Partners’ three-layer framework of machine learning companies

    Throughout this year, however, we realized we were missing a piece of the puzzle. While there will always be market demand for cross-industry capabilities and industry-specific applications, 2021 taught us that as companies try to productize AI for the first time, base platforms will become increasingly important as well. 

    After all, more companies than ever are charging forward on artificial intelligence projects to try to guide business decisions and save time and money. A whopping 90% of enterprises are either actively running AI projects or plan to in the next 12 months, according to a recent survey.   

    But here’s the catch: Most of these projects will fail.  

    There are many reasons why including a lack of sufficient data or black-box models that spit out indecipherable results. Regardless of the reason, high failure rates can hurt the ecosystem if they result in corporations’ hesitation to move forward with future AI projects.  

    Accordingly, Insight is particularly excited about a subsection of AI tools that can significantly increase companies’ likelihood of succeeding at their project goals: Machine learning operations, or MLOps. 

    MLOps tools can improve a machine learning pipeline from start to finish by helping gather, manage and label data, experiment and test the model selection, deploy multiple models in production at once and protect against model and data drifts and attacks. 

    Holistically, the goal is to improve communication and collaboration between data scientists, data engineers and business analysts through the machine learning life cycle, similar to how DevOps tools help improve communication in the software development life cycle. Machine learning is an iterative process with constant feedback loops and the need for continuous monitoring, which makes MLOps tools to manage this process even more critical. 

    Overview of some of the key areas of an MLOps pipeline

    While every company rolling out AI projects can benefit from MLOps, the type of tools they require will depend on their needs, resources and strategies.   

    Companies should ask themselves: How sophisticated is our data science team? How mission-critical do we want our models to be? Are our data sources structured or unstructured? Do we want open source, commercial or homegrown tooling?   

    The answers to those questions will point companies to specific MLops tools that are best suited for their specific challenges. 

    For example, some platforms—like Dataiku or DataRobot—are end-to-end products with wide capabilities, which are typically easier to use and thus encourage model creation and experimentation from workers across job functions, regardless of technical knowledge. These so-called ‘citizen data scientists’ can boost analytic workflows for companies with fewer data science experts and create significant value, but there’s a risk that the deployed models are not as well understood or controlled. These types of tools are best for companies that want to use machine learning to drive business insights and analytics rather than as part of their core business function.  

    If a company has a sophisticated data science team, however, it may lean toward specific point solutions, or what Insight calls best-in-class MLOps solutions. While these products require greater technical knowledge to maintain and deploy, they allow for much more control, depth and sophistication in a system. One startup offered the apt analogy that using platforms versus best-in-class solutions is like driving an automatic car versus a stick shift. 

    There are multiple MLOps tools for each part of the pipeline, and we see a world where each tool has enough market space to support it becoming a large company. 

    At Insight, we spent 2021 learning about and investing in many of what we consider to be the best-in-class MLOps tools, as determined by highly satisfied customer feedback, clear market momentum and well-respected and knowledgeable teams. This perspective led to our investments in Explorium, Rasgo, Weights & Biases, Deci, Run AI, Fiddler, Tonic, Dataiku and Databricks, as well as several others that have yet to be announced. 

    Insight Partners’ MLOps market map 

     While Insight expects MLOps to continue to be an essential driver of AI success in the new year, we also expect to see several other trends take shape in 2022:  

    • First, we will see the emergence of two machine learning pipelines, one for structured and one for unstructured data. Unstructured data sources—like images, videos and audio—require specific data warehouses, data management, pipelines and model management tools that maximize productivity and accuracy. Given the nuances and complexities between the two data types, we expect to see more MLOps tools shift toward owning their category for one or the other. 
    • As enterprises scale from dozens of models to hundreds or even thousands of models, we expect the increased industrialization of AI. This will result in the rise of orchestration layers that sit on top of machine learning pipelines to help manage all the different tooling. These layers will integrate across multiple tools in the ML pipeline—whether open source, commercial or homegrown—and provide an integrated environment (or so-called “single pane of glass”) to better track and manage ML pipelines. Orchestration layers will help enterprises gain more control over their pipelines while still supporting best-in-class tools. 
    • Insight believes we are at an inflection point of moving from a model-centric to a data-centric world. In a model-centric approach, you ask how you can change a model’s code to improve the performance of the system, whereas, in a data-centric model, you ask how you can change the data to improve it [system performance]. We expect the most successful tools of 2022 to be focused on supporting the new tasks, workflows, and jobs spawned by the data-centric movement.
    • We will also see a new class of MLOps tools focused on closing the growing AI skills shortage and gap across businesses. These tools will focus on increasing the efficiency of the model production process and lower the bar for ML development.
    • Finally, we expect to see a significant increase in the overall enterprise adoption of ML, with a shift from experimentation to deploying models in full production. Last year, we saw many enterprises form AI research teams and hire data scientists to experiment with machine learning. In 2022, we believe these enterprises will start realizing the full capacity of their AI projects and driving significant business value. This increase in enterprise adoption will also be driven by the explosion of edge devices that will enable new use cases of ML adoption. 

    As active machine learning and artificial intelligence investors, we are excited to continue to watch the market and support new entrants in what will surely be another dynamic and exciting year. 


    George Mathew, Lonne Jaffe and Sophie Beshar co-authored this piece.

  • AI: It’s What Staff Crave!

    AI: It’s What Staff Crave!

    If you’ve seen Idiocracy (Side note: I am not recommending the movie; I’m using a phrase from it we’ve all heard. The movie itself was a comedy not at all meant to be family-friendly when it was made, and changes in social mores since its 2006 release make it far less acceptable today), you are familiar with the title phrase I’ve used here. That’s what is repeated almost religiously to justify why they buy/use/drink Brawndo, an energy drink.

    One of the scenes involves people telling the star that plants need Brawndo—“It’s got electrolytes … That’s what plants crave!” This is repeated over and over until he asks, “What are electrolytes?” After some stuttering and failures to answer, someone says, “It’s what they make Brawndo out of.” Someone else inevitably tacks on, “It’s what plants crave.”

    Now, substitute “AI” for “Electrolytes”. That’s where we seem to be. Ask a vendor what algorithms they’re using, where their training data comes from, the difference in results between the test reservations and the real world. Most will end up saying a lot, not answering your questions and ending with, “It’s what staff crave!” or some such.

    A few companies will lay it all out there, assuming you can parse through the information and are familiar with the AI training algorithms they are using or the algorithms they’re using to classify inputs against the dataset. And these companies are grand because they (a) know what they’re actually doing beyond “It’s got AI/ML, it’s what staff crave,” and (b) Expect that you are knowledgeable about the data you’ve asked for or are capable of looking it up in this age of wonder that has boundless information at your fingertips.

    Don’t let your vendors get away with just repeating “It’s got electrolytes” over and over. AI and ML are a massive help in some areas, but they’re not yet a panacea and success rates vary wildly whether we’re talking problem domains, datasets, algorithms, training style, etc. Make them tell you the details. Where the data comes from and how big that dataset is. How much was reserved for testing and whether it was reserved randomly. Ask them what they hope AI will achieve, then ask them how that correlates to human interaction. Most AI today is still evaluating or replicating human interactions with something—most AI in IT particularly so. What algorithms were used to validate? And finally, make sure to ask for real-world results. I know people in data science and one has confidently said to me, “My models are 95% accurate.” When I finally dug into that claim (because my experience is that real-world use of AI/ML is almost never that accurate) it turned out that claim was true … for the reserved test data. So, the same dataset used to train got the same results on the reserved subset. That’s a good thing, but it is no indication of how said algorithm will do in the wild with unfiltered data. The big problem really is the switch from test data to real-world data.

    In fact, in IT, we are a little better off than most spaces wrestling with AI/ML, because very often there is no difference between test and training data in IT. A network security vendor will test against actual traffic—because they have access to massive volumes of it and it shortens the iterations to use what’s at hand and what the system is being trained for.

    And keep rocking it. The current iteration of IT is the opposite of the dark comedy future depicted in Idiocracy. You all are putting up systems and keeping them running at increasingly fast cadences; sometimes remotely because of COVID-19 or other work-related issues. Don’t stop turning out success for your coworkers and customers and take a moment to force vendors to explain what they’re doing with AI. If you’re a generalist like me, ask over email, though, so you can go look up the answers and make sure you understand the details.