devopsy.kinsta.cloud

Author: Diego Oppenheimer

  • Enterprise ML Investments on the Rise

    Enterprise ML Investments on the Rise

    The second wave of enterprise artificial intelligence and machine learning (AI/ML) adoption is building, and is expected to crest in 2021, as many organizations increased and accelerated their investments in tools and people.

    That was one of the unmistakable conclusions of our 2021 enterprise trends in machine learning report: 83% of the companies we surveyed said their AI/ML budgets had increased year-over-year, and we found that the average number of data scientists they employ has increased by 76% year-over-year. These changes came even as many organizations tightened their overall spending and refocused their strategic priorities amidst all of 2020’s disruption and uncertainty related to the COVID-19 pandemic. These companies aren’t simply moving forward with their AI/ML initiatives in spite of the pandemic; they’re doubling down on their investments because of it.

    Many organizations have begun to understand just how critical ML is to remaining competitive in uncertain and rapidly evolving conditions. When we ran a separate survey during the summer of 2020 to find out how businesses adapted to the pandemic’s impacts, it revealed a renewed sense of urgency around AI/ML investments: 43% of respondents said that AI/ML mattered “way more” to their business than they’d previously realized.

    Our independently conducted 2021 report included input from more than 400 leaders and practitioners across a diverse mix of technical and business functions, all of whom have a stake in their organization’s AI/ML strategy. They’re not just increasing their spending and staffing, either – they’re accelerating their pursuit of a growing number of enterprise ML use cases, with customer experience and automation leading the way.

    You need to be doing this, too. The second major wave of enterprise AI/ML adoption is underway, and you can still capitalize on the immense opportunities ahead. But the time to act is now. Organizations that take a “wait-and-see” approach are at risk of being left behind, particularly as so many other companies are scaling up their investments. Not only are your competitors increasing their AI/ML budgets (evident in the 83% YoY increase mentioned above), but they are moving their initiatives to the top of their strategic priority lists. Approximately three out of every four respondents (76%) in our 2021 report said they are prioritizing AI/ML over other IT initiatives, with 28% reporting that it is their top priority overall. These companies have realized that their success is directly tied to their ability to adapt to rapidly changing business conditions — and that AI/ML is the key to doing so.

    This is no longer the sole purview of elite technology companies. Companies of all sizes, across just about every industry, can get up and running with AI/ML, thanks in large part to the spread of easy-to-use tooling. The opportunities are more accessible than ever. You don’t need a ton of time or resources to get started — and it’s absolutely crucial that you do so soon.

    Your competitors are actively pursuing AI/ML initiatives today, regardless of your specific industry. We’re seeing many longstanding enterprises begin to drive huge transformations of their businesses and industries with AI/ML. More and more use cases are being pursued in legacy industries, from insurance to agriculture, healthcare to financial services and more. The bellwether financial firm JPMorgan Chase, for example, has even built an entire AI research program and data platform. There are virtually no industries where this is irrelevant. So, you need to act with urgency, as this second wave of enterprise adoption intensifies.

    You also need to build for operational efficiency and scale. We think of this as industrializing your machine learning operations (MLOps). While easy-to-use tools reduce the barrier to entry, they only get you partway to your destination. Invest in repeatable, automated processes or you’ll hit a wall of technical debt and other potential issues. MLOps paves the road to tightly integrated, well-aligned technology stacks, processes and teams that make it easier, not harder, to deploy more models in the future. This isn’t something you can wait on; if you’ve manually bootstrapped the deployment of even a handful of models to production, you need MLOps. Without it, you’ll quickly accrue unnecessary technical debt that will inevitably hinder your organization’s ability to scale and deliver meaningful business value.

    This is already a significant issue. Our 2021 report found that, while many organizations are increasing budgets and prioritization, it’s taking them longer to get models into production. Data scientists are still spending too much of their time on operational tasks, a problem that worsens as the number of models in production increases. This tells us that even as many organizations increase their spending and hiring, they’re actually allocating more of their growing resources toward manually scaling their AI/ML initiatives instead of investing in the operational efficiency that will maximize results.

    There’s a better way. Organizations no longer need to take a DIY approach to MLOps. There are robust solutions that can simplify many of the complex issues involved in scaling your AI/ML initiatives in production. Our report found that companies that use third-party MLOps solutions see improved outcomes compared with companies that go it alone and build and maintain everything themselves. They spend less on infrastructure, deploy models to production faster and tie up less of their data scientists’ time on model deployment.

    The pattern here is clear. Regardless of your particular industry, more and more businesses are identifying a growing list of use cases for AI/ML that can help transform their businesses and boost their competitiveness. They’re industrializing their AI/ML initiatives with MLOps to ensure their technology stacks, processes, and people are well-aligned. And, by doing so, they’re positioning themselves to rapidly adapt to and act on changing signals in their businesses.

    Don’t make the mistake of thinking AI/ML doesn’t apply to you or your industry. Doing so would mean missing out on potentially massive strategic opportunities, from new or growing revenue streams to increased efficiencies throughout your business operations. Your competitors are doing it; it’s easier than ever to get started and third-party MLOps solutions can help minimize technical debt and other operational complexity that will otherwise limit the positive business impacts of your initiatives.

    You can absolutely get in on the second wave of enterprise adoption, but the time to act is now. You can’t afford to wait.

  • Five Challenges of Machine Learning DevOps

    Five Challenges of Machine Learning DevOps

    As organizations add machine learning (ML) to their workflows, it’s tempting to try to squeeze model creation and deployment into the existing software development lifecycle (SDLC). However, ML is fundamentally different than traditional applications, and it’s important to account for that in a new, unique process called the machine learning development lifecycle.

    We have identified five challenges every organization should keep in mind as they begin to support ML development.

    Making Heterogeneity Work

    Machine learning is successful when the right tool is selected for a given job. Depending on the use case, a data scientist might choose Python, R, Scala or another language to build one model, and another language for a second model. Within a given programming language, there are numerous frameworks and toolkits available, adding complexity to versioning and consistency.

    For example, TensorFlow, PyTorch and Scikit-learn all work with Python, but each is tuned for specific types of operations, and each outputs a slightly different type of model.

    ML-enabled applications typically call on a pipeline of interconnected models, often written by different teams using different languages and frameworks.

    Figure 1: An ML pipeline that extracts text from scanned documents and analyzes its sentiment passing through several languages and frameworks.

    For example, a public relations firm looking to identify news and reports critical of one of its customers might use the following pipeline:

    1. Extract all text from hundreds of scanned documents with an Optical Character Recognition (OCR) model.
    2. Identify the languages of that text with a language-identification model.
    3. Translate the text to English.
    4. Prepare the text for sentiment analysis with a vectorization model that turns English text to numbers.
    5. Use a sentiment analysis model to score the text.

    Planning for ML Iterating

    In machine learning, your code is only part of a larger ecosystem—the interaction of models with live, often unstructured or unpredictable, data. Interactions with new data can introduce model drift and affect accuracy, requiring constant model evaluating, tuning and retraining.

    Figure 2: The continuous ML Lifecycle with iterations.

    As a result, ML iterations are faster and more frequent than traditional app development workflows. This places greater demands on versioning and other operational processes and requires a greater degree of agility from DevOps processes, tools and staff.

    Organizations interested in adopting ML-enabled applications need to be prepared to handle the burden of iteration without breaking the budget or introducing needless infrastructure tasks.

    Developing the Right Infrastructure

    The model training process typically involves multiple instances of the following:

    • A long and intensive compute cycle.
    • A fixed, inelastic load.
    • A single user.
    • Concurrent experiments on a single model.

    After deployment, models from multiple teams enter a shared production phase characterized by:

    • Short, unpredictable compute bursts.
    • Elastic scaling.
    • Many users consuming many models.

    Operations teams must be able to support both the training and deployment environments on an ongoing basis, because it’s a continuous lifecycle. Selecting the right infrastructure for that job is complicated by a rapidly evolving tech stack, from the increasing variety of processors available for ML workloads to advances in cloud-specific scaling and management, to a never-ending stream of new data science tooling.

    It’s important to future-proof your ML program to ensure a tool or framework you use today does not lead to lock-in or worse—obsolescence—down the road.

    How Will You Scale?

    To address the unpredictability of ML workloads and the premium on low latency, organizations must build compute capacity to support substantial bursts.

    There are three typical architectural approaches, each with benefits and drawbacks:

    Traditional Capacity Planning: In a traditional architecture, operations reserve compute resources capable of scaling to maximum anticipated demand.

     

    Pros:

    • Reserved capacity is always available
    • Easy to administer

     

    Cons:

    • Extreme waste and cost
    • Unanticipated demand can exceed the capacity

     

    Elastic Scaling: Standard elastic scaling designs for a local maximum, scaling machines up and down based on step functions.

     

    Pros:

    • Cost improvements over traditional architecture

     

    Cons:

    • Inefficient hardware utilization
    • Difficult to manage heterogeneous workloads on the same hardware
    • Slightly more management overhead than traditional architecture

    Serverless: Serverless architecture approaches spin up models as requests come in.

     

    Pros:

    • Significantly improved hardware utilization
    • Dramatically lower costs
    • Easier to run and scale different types of workloads on the same machines

    Cons:

    • Substantial administrative overhead for custom-built, on-prem solutions
    • Some third-party services without extensive APIs and reporting provide limited explainability
    • Performance tuning to meet latency

    Architects can implement any of the three approaches in the cloud or in a physical datacenter, though elastic scaling or serverless approaches requires far more effort to manage in-house.

    Trusting the Results with Auditability and Governance

    Explainability—understanding why models make given predictions or classifications—is an essential part of ML infrastructure and the lifecycle. Equally important and often overlooked, however, are the related topics of model auditability and governance—understanding and managing access to models, data and related assets.

    Figure 3: An audit trail can help inform decisions.

    To make sense of complex pipelines, a multitude of users and rapid model and data iteration, deployment systems should attempt to identify:

    • Who called which version of which model.
    • At what time a model was called.
    • Which data the model used.
    • What result was produced.

    Thinking about and planning for these five challenges before beginning a machine learning initiative is a good place to start to ensure a valuable return from your investment.

    To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon NA, November 18-21 in San Diego.

    — Diego Oppenheimer