devopsy.kinsta.cloud

Tag: MLOps

  • Learn a Bit About AI

    Learn a Bit About AI

    Every once in a while, a trend becomes broad enough that I feel the need to offer career advice. This year it will be simple: Know enough about AI/ML to be dangerous. We are past the point where not having at least a basic understanding is a good idea. Looking around the data center, AI is in use from the earliest stages of development through production network performance evaluation. Somewhere in your environment, you are using AI and ML. More to the point, somewhere in the places you may interview going forward, they are using AI and ML.

    I’m not suggesting you become an AI/ML expert—though you certainly could; I have a relative that is a data scientist and that’s all he does—design and tweak algorithms. He’s learning new things pretty much daily, so if you like that (as I do), it’s an option. But for those of us doing DevOps or SRE or even regular old operations, we’re only going to have to understand a bit.

    Understand what tools like Jupyter are, and what the most commonly used ML algorithms in your space are. Then, learn about how AI and ML help with what you are doing. Just as knowing more about the compiler/interpreter of your chosen language helps you solve more odd problems, AI is going to have that same issue. Think about how those folks who understand the guts of routing are the ones we want solving our routing problems. Again, same scenario.

    Moving forward, AIOps, NetOps, DevSecOps … All intend to increase the amount of work offloaded onto the AI in question. In a lot of cases that is great. We have enough to do; if you can paw through those logs and give me relevant summary info, go for it. But it doesn’t end there—some AI engines are making the calls and implementing changes. In both network and application security, a fair number of vendors will tell you there is no time to wait for humans to stop an attack, and they encourage their customers to turn on automated responses. That’s the first of many applications for AI/ML. We will see an increasing number of AI/MLOps tools being used for full-on automation. We all do difficult jobs with a complex—sometimes bewildering—number of inputs. But we use pattern recognition and experience to make determinations and do what’s best for our organization. Our software has reached the point where sometimes, it can do that for us on the easier tasks so we can focus on the more complex ones. When the AI/ML fails—and it will, because pattern recognition and historical trends are not always indicative of normal usage—knowing what is going on under the covers can help you determine if it did the right thing in the wrong way, or simply did the wrong thing. And it can help you to quickly recover. I can’t speak for every organization, but I’ve never seen one that lamented, “Sure wish that outage had lasted another couple of hours!”

    And keep rocking it. Somewhere in the distant future, an  AI may be able to do your job as well as you. But that isn’t today. Even if it can handle one bit, it certainly isn’t rocking the entire breadth of your responsibilities like you are. Thank you, from all the users that don’t realize you’re keeping them live.

  • Why You Should Use GitOps to Experiment With AI

    Why You Should Use GitOps to Experiment With AI

    As the pace at which artificial intelligence (AI) models are being constructed and inevitably updated starts to increase, it’s becoming more apparent that the pace at which data science teams are currently experimenting also needs to dramatically quicken.

    Many data science teams today are fortunate if they manage to successfully deploy two AI models in a production environment over the course of 12 months. It’s already apparent, however, thanks in part to the rise of digital business transformation, that there are potentially hundreds of use cases for AI models within each enterprise IT environment. Truth be told, it’s hard to envision an application being developed today that would not incorporate one or more AI models. The challenge is, the demand for AI capabilities is now far outpacing the ability of data science teams to deliver.

    The best way to solve this issue is to recognize that AI models and the components employed to construct them are, in reality, just another kind of software artifact. In fact, the need to accelerate the development of applications using artifacts is largely a problem already solved. Rather than reinventing the wheel, the time has come to apply what we already know to the development of AI models.

    The Rise of GitOps-Based Processes

    Modern software development teams today routinely make use of Git-based repositories to share code and collaborate. These repositories are jam-packed with vetted software artifacts that development teams continuously reuse via an opinionated instance of a DevOps workflow based on a set of GitOps best practices.

    As a subset of DevOps, a GitOps workflow enables data science teams to employ a Git repository that already exists within their software development environment to manage artifacts. Git enables versions of files to be created using what is known as a Git branch. They also employ concepts known as tags (Git Tag) and remotes (where the server is located). Those three different concepts, or references, exist under the hood in Git. In fact, the entire flow is based on those concepts. When a new feature is implemented, a new branch is created and then merged into the production branch with a tag that says something like “This is a new version that needs to be deployed.”

    An AI model, like any other software artifact, needs to be updated. As new data sources become available, an AI model is likely to drift beyond the parameters originally intended. A rapid change in business conditions may also require changes to an AI model. The same processes that DevOps teams apply to update any software artifact can also be applied to rapidly update an AI model. Employing a Git repository to also manage AI artifacts extends the benefits that software development teams enjoy to data science teams.

    Often, organizations will have an MLOps solution to streamline AI model development that is completely separate from their DevOps tools. These solutions will have dedicated features like version tracking for data and experiments specifically for AI models. But many MLOps solutions exist in a closed ecosystem, meaning they don’t work off of existing tools like GitHub used on the DevOps side. 

    Also, data science teams have to separately buy, deploy, and manage these closed MLOps solutions. Going with a GitOps-oriented solution streamlines management of added MLOps tools and makes it easier for both application developers and data science teams to gain visibility into each other’s respective efforts. The end result is not just faster AI experimentation at lower cost; the likelihood that more AI models will successfully be deployed in a production environment increases dramatically. 

    Software and AI Teams Working Together

    The convergence of DevOps and MLOps is already inevitable. The longer organizations continue to employ isolated MLOps platforms to build and deploy AI models, the more expensive it will become to meld MLOps and DevOps processes. 

    Data science teams are conservative by nature. The difference between success and failure often comes down to how thoroughly an AI model is validated. However, time to market doesn’t have to be sacrificed to ensure an AI model works as promised. There is no such thing as a great AI model that arrives late when rivals are racing to develop similar AI capabilities—there are only missed opportunities. Speed enabled by the reusability of artifacts that drives faster experimentation using GitOps workflows is of the essence now more than ever.

  • Enterprise ML Investments on the Rise

    Enterprise ML Investments on the Rise

    The second wave of enterprise artificial intelligence and machine learning (AI/ML) adoption is building, and is expected to crest in 2021, as many organizations increased and accelerated their investments in tools and people.

    That was one of the unmistakable conclusions of our 2021 enterprise trends in machine learning report: 83% of the companies we surveyed said their AI/ML budgets had increased year-over-year, and we found that the average number of data scientists they employ has increased by 76% year-over-year. These changes came even as many organizations tightened their overall spending and refocused their strategic priorities amidst all of 2020’s disruption and uncertainty related to the COVID-19 pandemic. These companies aren’t simply moving forward with their AI/ML initiatives in spite of the pandemic; they’re doubling down on their investments because of it.

    Many organizations have begun to understand just how critical ML is to remaining competitive in uncertain and rapidly evolving conditions. When we ran a separate survey during the summer of 2020 to find out how businesses adapted to the pandemic’s impacts, it revealed a renewed sense of urgency around AI/ML investments: 43% of respondents said that AI/ML mattered “way more” to their business than they’d previously realized.

    Our independently conducted 2021 report included input from more than 400 leaders and practitioners across a diverse mix of technical and business functions, all of whom have a stake in their organization’s AI/ML strategy. They’re not just increasing their spending and staffing, either – they’re accelerating their pursuit of a growing number of enterprise ML use cases, with customer experience and automation leading the way.

    You need to be doing this, too. The second major wave of enterprise AI/ML adoption is underway, and you can still capitalize on the immense opportunities ahead. But the time to act is now. Organizations that take a “wait-and-see” approach are at risk of being left behind, particularly as so many other companies are scaling up their investments. Not only are your competitors increasing their AI/ML budgets (evident in the 83% YoY increase mentioned above), but they are moving their initiatives to the top of their strategic priority lists. Approximately three out of every four respondents (76%) in our 2021 report said they are prioritizing AI/ML over other IT initiatives, with 28% reporting that it is their top priority overall. These companies have realized that their success is directly tied to their ability to adapt to rapidly changing business conditions — and that AI/ML is the key to doing so.

    This is no longer the sole purview of elite technology companies. Companies of all sizes, across just about every industry, can get up and running with AI/ML, thanks in large part to the spread of easy-to-use tooling. The opportunities are more accessible than ever. You don’t need a ton of time or resources to get started — and it’s absolutely crucial that you do so soon.

    Your competitors are actively pursuing AI/ML initiatives today, regardless of your specific industry. We’re seeing many longstanding enterprises begin to drive huge transformations of their businesses and industries with AI/ML. More and more use cases are being pursued in legacy industries, from insurance to agriculture, healthcare to financial services and more. The bellwether financial firm JPMorgan Chase, for example, has even built an entire AI research program and data platform. There are virtually no industries where this is irrelevant. So, you need to act with urgency, as this second wave of enterprise adoption intensifies.

    You also need to build for operational efficiency and scale. We think of this as industrializing your machine learning operations (MLOps). While easy-to-use tools reduce the barrier to entry, they only get you partway to your destination. Invest in repeatable, automated processes or you’ll hit a wall of technical debt and other potential issues. MLOps paves the road to tightly integrated, well-aligned technology stacks, processes and teams that make it easier, not harder, to deploy more models in the future. This isn’t something you can wait on; if you’ve manually bootstrapped the deployment of even a handful of models to production, you need MLOps. Without it, you’ll quickly accrue unnecessary technical debt that will inevitably hinder your organization’s ability to scale and deliver meaningful business value.

    This is already a significant issue. Our 2021 report found that, while many organizations are increasing budgets and prioritization, it’s taking them longer to get models into production. Data scientists are still spending too much of their time on operational tasks, a problem that worsens as the number of models in production increases. This tells us that even as many organizations increase their spending and hiring, they’re actually allocating more of their growing resources toward manually scaling their AI/ML initiatives instead of investing in the operational efficiency that will maximize results.

    There’s a better way. Organizations no longer need to take a DIY approach to MLOps. There are robust solutions that can simplify many of the complex issues involved in scaling your AI/ML initiatives in production. Our report found that companies that use third-party MLOps solutions see improved outcomes compared with companies that go it alone and build and maintain everything themselves. They spend less on infrastructure, deploy models to production faster and tie up less of their data scientists’ time on model deployment.

    The pattern here is clear. Regardless of your particular industry, more and more businesses are identifying a growing list of use cases for AI/ML that can help transform their businesses and boost their competitiveness. They’re industrializing their AI/ML initiatives with MLOps to ensure their technology stacks, processes, and people are well-aligned. And, by doing so, they’re positioning themselves to rapidly adapt to and act on changing signals in their businesses.

    Don’t make the mistake of thinking AI/ML doesn’t apply to you or your industry. Doing so would mean missing out on potentially massive strategic opportunities, from new or growing revenue streams to increased efficiencies throughout your business operations. Your competitors are doing it; it’s easier than ever to get started and third-party MLOps solutions can help minimize technical debt and other operational complexity that will otherwise limit the positive business impacts of your initiatives.

    You can absolutely get in on the second wave of enterprise adoption, but the time to act is now. You can’t afford to wait.

  • How MLOps Can Help Get AI Projects to Deployment

    How MLOps Can Help Get AI Projects to Deployment

    Did you know that most AI projects never get fully deployed? In fact, a recent survey by NewVantage Partners revealed that only 15% of leading enterprises have gotten any AI into production at all. Unfortunately, many models get built and trained, but never make it to business scenarios where they can provide insights and value. This gap – deemed the production gap – leaves models unable to be used, wastes resources and stops AI ROI in its tracks. But it’s not the technology that is holding things back. In most cases, the barriers to businesses and organizations becoming data-driven can be reduced to three things: people, process and culture. So, the question is, how can we overcome these challenges and start getting real value from AI? To overcome this production gap and finally get ROI from their AI, enterprises must consider formalizing an MLOps strategy.

    MLOps, or machine learning operations, refers to the culmination of people, processes, practices and underpinning technologies that automate the deployment, monitoring and management of machine learning models into production in a scalable, fully governed way. Laying an MLOps foundation allows data, development and production teams to work collaboratively and leverage automation to deploy, monitor, manage and govern machine learning services and initiatives within an organization. In short, MLOps can help enterprises see AI projects through to deployment at scale by addressing the three previously mentioned barriers: people, process and culture.

    People

    There’s a skill, motivation and incentive gap between teams developing machine learning models (data scientists), operators of those models (DevOps, software developers, IT, etc.), and the business leaders who look for the results of that work. By providing actionable visibility into the different steps of the machine learning model life cycle for these different roles, MLOps can increase collaboration across departments so that enterprises can streamline the process of getting models operational and finally meet strategic goals.

    Process

    Operations teams are geared towards ensuring overall quality of service, optimizing runtime environments on their cloud, resource management and role-based access, among many other tasks. Data science teams are often unaware of the considerations these dependencies require, and, hence, models they create do not take these tasks into consideration. Additionally, lack of a proper native governance structure as it pertains to machine learning models – with system, performance, life cycle, and user logs – stifles troubleshooting, as well as hinders legal and regulatory reporting. Organizations that don’t properly monitor their models end up introducing potential risk to their organizations. When production models don’t reflect the ever-changing patterns in data and user behavior, these factors may affect the accuracy and overall performance of the model. Machine learning-based applications are far more sensitive to production environments than typical software applications. Therefore, machine learning models need to be validated and updated over time to ensure that they’re performing properly and leveraging the most accurate and recent data. As part of implementing an MLOps strategy, by having the right monitoring processes in place to detect drift – and many other machine learning-specific metrics – enterprises can ensure their models are delivering the most accurate results, and that their projects are more likely to move forward.

    Culture

    Fear of failure (or worse) can delay organizations from collaborating to operationalize machine learning models, or prevent this entirely. Leadership needs to be clear about their commitment to the process and fully support teams in enduring the hardships involved. This is only compounded by the fact that machine learning applications typically tend to involve processes that might introduce material risk to the company. To stay on top of such transformations while still producing trustworthy value, enterprises must keep careful tabs on all model deployment processes and the updates they entail. Many operations professionals are not aware of the unique characteristics and sensitivities of machine learning. As their role is focused on managing mission-critical production environments, they are often solely concerned with the implications on their work that deploying machine learning into production might have. While these are justifiable concerns, in the final analysis, they serve as trivial reasons for significant delays in progress and deployment. The way to mitigate all these risks and get all teams aligned around these common goals is to implement a solid MLOps strategy that matches the goals and unique position of the organization and its various teams’ responsibilities.

    MLOps: All Systems Go

    MLOps allows organizations to alleviate the people, process and culture issues posed by machine learning. MLOps provides a strategy and a set of processes and best practices, a technological backbone, for managing and scaling machine learning-based services through automation. It also provides for seamless collaboration between the data science teams responsible for generating models with the teams that are accountable for running the services in production environments. This combination of process automation with role-based collaboration and governance capabilities helps pave the way toward strategic AI goals. Specifically, MLOps can help organizations move from anticipating the management and scaling of machine learning services into production and turn that into reality.

  • AWS Looks to Meld MLOps and DevOps

    AWS Looks to Meld MLOps and DevOps

    The rate at which machine learning operations (MLOps) and DevOps best practices are converging is poised for rapid acceleration in 2021. At the recent online AWS re:Invent conference, Amazon Web Services (AWS) announced it is adding a bevy of capabilities to Amazon SageMaker, a managed MLOps service, that includes a continuous integration/continuous delivery (CI/CD) service for MLOps it is calling Amazon SageMaker Pipelines.

    Amazon SageMaker Pipelines enables developers to define a machine learning workflow that encompasses everything from loading data to training configuration; algorithm set up and debugging algorithms. Rather than having to acquire and deploy a standalone CI/CD platform for building artificial intelligence (AI) models, AWS is making the case for using a managed service that enables developers to manage workflows from within the Amazon SageMaker Studio tools it already provides for building AI models.

    Workflows can be shared and reused between teams to either re-create a model or can serve as a launch point from which to create another AI model.

    Amazon SageMaker Pipelines also logs each event in Amazon SageMaker Experiments, which enables IT teams to organize and track machine learning experiments and versions of AI models. In addition, a Deep Profiling for Amazon SageMaker Debugger now makes it possible for developers to more quickly train AI models by automatically monitoring system resource utilization and generating alerts whenever bottlenecks are detected.

    AWS has also added a Distributed Training capability to Amazon SageMaker that it claims makes it possible to train large, complex deep learning models up to two times faster than current AWS approaches by enabling data and AI parallelism across its cloud services.

    The cloud services provider has also added an Amazon SageMaker Edge Manager service that enables developers to optimize, secure, monitor and maintain machine learning models deployed on fleets of edge computing devices. The Amazon SageMaker JumpStart service enables developers to search for AI models, algorithms and sample notebooks.

    Finally, AWS has added additional data preparation tools that come pre-configured with support of AWS and third-party data sources, a repository for storing AI models, a preview of additional tools for automatically matching AI models to data sources such as the Amazon RedShift data lake, the ability to match an AI model to a graph database and a tool that detects bias within AI models.

    Bratin Saha, vice president and general manager for machine learning services at AWS, said as usage of AI models increases, the scope of these initiatives naturally lends itself more to cloud platforms. Many applications are starting to employ multiple classes of AI models in combination with each other; Saha noted this requires massive amounts of data and compute resources to train those models that are only readily available in the cloud.

    Overall, AWS claims it already has more than 100,000 customers building AI applications on its platform using various tools and processor classes. AWS said in 2021 it will add G4ad Graphics Processing Unit (GPU) instances based on processors from AMD and revealed its intention to build its own GPUs next year. The company also plans to add an Amazon EC2 instance based on up to eight of Intel’s Habana GAUDI accelerators to deliver up to 40% better price-performance than current GPU-based EC2 instances.

    At the same time, AWS has extended its alliance with NVIDIA to make software modules for building AI models available on the AWS Marketplace.

    In effect, Saha said AWS is inviting IT teams to either build AI models on AWS or bring their own AI models, encapsulated in containers, to its platform.

    AWS is clearly aiming to dominate AI, regardless of how such capabilities are infused into enterprise applications. In fact, in many cases, AWS will automatically construct the AI model for customers, or simply return the results generated by an AI model via an application programming interface (API). It will be up to each organization to determine when building their own AI model makes the most sense, and to what degree they want to rely on AWS to automate the deployment process.

    What’s certain is that many of the same DevOps principles—created long before AI models ever came along—will still be relevant as the applications that use those models are continuously updated for years to come.

  • Algorithmia Allies With Datadog on MLOps Observability

    Algorithmia Allies With Datadog on MLOps Observability

    Algorithmia today launched a performance monitoring for machine learning (ML) model that tracks algorithm inference and operations metrics generated by the enterprise edition of its namesake platform for building these models.

    Company CEO Diego Oppenheimer said Algorithmia Insights provides a level of observability into ML models that DevOps teams have come to expect from applications. To make that monitoring capability more accessible, Algorithmia has partnered with Datadog to stream operational and user-defined inference metrics from Algorithmia to the open source Kafka data streaming platform, from which the Datadog Metrics application programming interface (API) can consume that data.

    That capability will enable DevOps teams that have adopted the Datadog monitoring platform to employ a single pane of glass to both monitor their IT environments as well as detect data drift, model drift and model bias with an ML model, Oppenheimer said.

    As organizations infuse ML models into applications many of them are encountering DevOps workflow challenges. It can take six months or more to develop an ML model, which then needs to be embedded with an application that often is updated several times a month. At the same time, organizations are discovering that ML models often need to be replaced either because assumptions about business conditions have changed or additional relevant data sources have become available. In many cases, data scientists have adopted DevOps principles to create a set of best practices known as MLOps to update their models.

    The alliance with Datadog is significant because it enables IT teams to start melding DevOps and MLOps processes, noted Oppenheimer.

    At this juncture, it’s apparent that almost every application to varying degrees will eventually be enhanced using machine learning algorithms. Most of the data scientists that create these models, however, have little experience deploying and updating ML models in a production environment. It’s now only a matter of time before ML models become just another artifact flowing through a DevOps workflow. Organizations that have invested in ML models, however, will need to spend some time bringing together a data scientist culture that is currently distinct from the rest of the IT organization.

    To further that goal, Algorithmia earlier this year made it possible to write and run local tests for algorithms as shared local data files. Desktop tools that have been integrated with that process include PyCharm, Jupyter Notebooks, R Shiny, Android, iOS, Cloudinary, Datarobot and H2O.AI.

    A recent survey of more than 100 IT directors conducted by Algorithmia finds the COVID-19 pandemic has required 40% of survey respondents to make adjustments to at least half of all their AI/ML projects either in terms of priority, staffing or funding. Over half the respondents (54%) said AI/ML projects prior to the pandemic were focused on financial analysis and consumer insight. In the wake of the pandemic, survey respondents said more of their AI focus is now on cost optimization (59%) and customer experience (58%).

    The survey also notes respondents were spending at least $1 million annually on AI/ML prior to the pandemic, with 50% saying they are planning to spend more than that going forward. Overall, nearly two-thirds (65%) said that AI/ML projects were at or near the top of their priority list before the pandemic, with 33% now saying these applications are now higher on their list.

    It may be a while before every DevOps team regularly encounters ML models, but at this point, it’s more a question of when rather if before DevOps processes expand to include ML model artifacts.

  • MLOps: More Than Automation

    MLOps: More Than Automation

    Recently, there has been a lot of talk about MLOps from the DevOps and software engineering perspective. The message often comes across as, “Hurry, DevOps community, let’s set some standards before MLOps reinvents the wheel,” or “The purpose of MLOps is to allow anyone who knows how to code to get ML into production.” Let me tell you, as a data scientist this disappoints and sometimes scares me. Why? Because neither of these points of view captures the pain of the data scientists or the people that are subject to ML algorithms.

    A good example of this attitude toward MLOps is Google’s recent announcement. If you have read Google’s blog, you’ll see that some of the pain points they have identified are needs for data scientists: We do need continuous evaluation, continuous monitoring, metadata management and lineage and a feature store for collaboration. But where I find the announcement falls down for me is in the company’s goal “to make machine learning act more like computer science.”

    Here’s the reality. DevOps wasn’t created to make software development behave more like a different field. It was created as a set of principles to enable software engineers to quickly and easily apply incremental improvements based on the needs of their customers, with a focus on quality. MLOps is more than automation. Centering on the needs of the users and the customers is what made developing products, ideas and a set of principles successful as DevOps.

    For MLOps to learn from DevOps, we must center the needs of data scientists and the people that are impacted by their models first. It isn’t enough to say that practicing MLOps means advocating for automation and monitoring at all steps to do things faster. Without this focus, we will see an increase in the deployment of models that have uninspected and unintended consequences that often disproportionately impact marginalized communities.

    So, as a data scientist, what is it that I need? Keeping up with the latest and greatest event streaming services, distributed systems or methods of continuous deployment of integration isn’t where my mind lights up. I would like to spend most of my time understanding the domain space of the model I’m about to build, the nuanced impact of that model and whether it’s going to meet the needs of my customers and the people they serve.

    There are a few ways to notice if you’re applying MLOps basically as a Band-Aid, a way to just go faster, that will ultimately break down. When looking for a solution to automate, consider if you’re only reducing the work required for manual processes or if you’re also enabling data scientists to focus on the hard problems they’re trained to tackle. Another way to look at your MLOps practice and tools end-to-end is to see if they allow your data scientists to know the impact of their models across more dimensions than just the outcome. Let’s take a few examples.

    Automation of Data Pipelines Can Slow You Down

    Today, data engineers have been reluctant to allow data scientists to have access to all of the data that exists in all of its forms. There are many reasons I’ve been given in my years of experience, some of which are good, such as not all of the tools that data scientists have available for experimentation support all of these data sources. Other reasons, however, might include grumbling about creating additional work for data engineers. It’s a continuous challenge to get data such as event streams into a format that can be explored by data scientists. Each new feature I hope to create comes at a cost to other teams.

    With our shiny new culture of MLOps, someone proposes a solution: Automate! But, remember, MLOps is more than just automation. This is a great place to stop and look at the data scientist’s workflow and find a way to automate a solution that fits their needs and gives them new capabilities while reducing the work on the data engineering team.

    A simple automation instinct might look like this: Ask data scientists to list out all the data they’ll ever need, and create batch jobs to serve up those needs. A better solution would be to find a way to connect multiple types of data sources to a single environment, allowing data scientists to explore data as it becomes available. Then create features and make them available for production without having to change development environments or languages or to be proficient in reading crystal balls to know what data they’ll need into the future.

    Continuously Monitoring the Wrong Things

    Another common pitfall of applying MLOps without focusing on the people your ML models will ultimately impact is using continuous monitoring only to measure performance, without tracking the impact of your features across various demographics. It’s not enough to check—one time—that your models aren’t systematically disadvantaging marginalized groups. We need to do this continuously.

    People change, so do our biases and patterns. It’s easiest to conceptualize this for me in our use of language over time: The words are the same but their impact changes. We often don’t know that the underlying data has changed because society itself has evolved. But we can commit to measuring the impact across dimensions to react quickly, ultimately reducing harm.

    Ultimately, MLOps isn’t just about automation. It is also a set of principles to guide how we build tools for sharing outcomes. At my company, we are thinking about ways to help identify and flag data and features that could introduce bias in our feature store product.

    The gains from efficient ML development are significant. Data scientists and data engineers are sharing new tools to speed development and foster collaboration at higher velocity. And with better tools and processes, we are able to share responsibility for productionizing ML. MLOps, at its best, incorporates these components and more into your organization.

    If we’re not careful, however, teams will focus on making data scientists more like software engineers, vendors will create tools that fail to address the entire problem, and companies will spend millions of dollars on quickly releasing models that fail to serve their customers—ultimately at the cost of humanity.

  • Algorithmia Looks to Meld MLOps and DevOps

    Algorithmia Looks to Meld MLOps and DevOps

    Algorithmia, in the latest update to the enterprise edition of its namesake machine learning operations (MLOps) platform, is enabling software development lifecycle practices to be applied to the building of algorithms by making it possible to debug them using desktop tools that are widely employed.

    The enterprise edition of Algorithmia will enable users to write and run local tests for algorithms as shared local data files. Desktop developer tools that can now integrate with that process include PyCharm, Jupyter Notebooks, R Shiny, Android, iOS, Cloudinary, Datarobot and H2O.AI.

    In addition, Algorithmia MLOps has extended its support for the latest graphical processor units (GPUs) available on Amazon Web Services (AWS) and Microsoft Azure.

    Finally, the company has added support for AWS C2S, a private cloud for intelligence services, and AWS GovCloud.

    Algorithmia CEO Diego Oppenheimer said IT teams building artificial intelligence (AI) applications are applying many of the same best DevOps practices employed elsewhere in the organization. Making it possible to debug those applications on a local desktop makes it easier for organizations to construct those DevOps workflows, he said.

    The Algorithmia platform enables IT teams to control the provenance of all components of ML operations, including certificate authorities, operating systems, container images, code, dependencies and ML models employed.

    Many organizations today are struggling with how to meld MLOps and DevOps processes. The rate at which AI models are created and updated is typically much slower than the rate at which the applications those models are infused is typically much slower. Data scientists are now coming to terms with how to deploy and update AI models within the context of a larger set of DevOps processes.

    In the wake of the COVID-19 pandemic, interest is AI is rising. Organizations are looking for ways to leverage machine learning algorithms to both drive down costs and increase revenue by automating a wide range of tasks. A recent global survey of 2,737 information technology and line-of-business executives conducted by Deloitte finds nearly two-thirds of the survey respondents (61%) said AI will substantially transform their industry in the next three years. More than half (53%) also noted they have spent more than $20 million during the past year on AI technologies and the talent required to master them.

    However, the same survey noted over half the respondents (56%) said their organization is slowing adoption of AI technologies because of the emerging risks. Obviously, improving the debugging process for applications that incorporate machine learning algorithms could go a long way toward reducing those risks.

    It’s not clear to what degree AI might one day transform the application experience. It is safe to say that eventually, almost every application will incorporate machine and deep learning algorithms to automate one or more processes. Those algorithms won’t eliminate the need for humans as much as they will automate rote processes that today consume more time than anyone prefers. The challenge, of course, will be making sure algorithms perform those tasks consistently as more data becomes available.

  • Cloudera Calls for MLOps Standards Initiative

    Cloudera Calls for MLOps Standards Initiative

    Cloudera put out a call this week for the IT industry to define a set of open standards for machine learning operations (MLOps) and machine learning model governance that could be universally applied.

    Santiago Giraldo, senior product marketing manager for data engineering for Cloudera, said the idea is to create the equivalent of a set of best DevOps practices to create the equivalent of a set of best DevOps practices for employing machine learning models employed by artificial intelligence (AI) applications that will serve to foster portability, interoperability and explainability.

    In the absence of such standards, most AI models are essentially black boxes optimized for specific platforms that many organizations are reluctant to adopt because no one is quite sure how they operate. Furthermore, it’s not all that feasible right now to integrate multiple AI models built using different classes of tools.

    At the same time, Giraldo said the further down the AI path organizations go, the more they realize the AI models they are building need to be either updated frequently or outright replaced as new data sources become available. Most AI models are trained to respond to specific events based on access to a finite set of data. As new data sources are introduced, organizations are discovering the accuracy of recommendations being made by an AI model start to degrade. In fact, only a limited number of AI models are making it into production environments today largely because they didn’t scale as expected.

    Cloudera is already moving down the MLOps path with Apache Atlas, an open source framework designed to integrate data management across explainable, interoperable and reproducible MLOps workflows. Now Cloudera is inviting the industry, including competitors, to collaborate on a broader set of metadata specifications that Anduago said is more a call to action than a specific proposal.

    It’s unclear precisely how such an initiative might be managed. There is no widely recognized consortium initiative focused specifically on MLOps. Thus far, the closest thing might be The Deep Learning Foundation, an arm of The Linux Foundation that connects organizations working on innovative technical projects focused on AI and machine learning.

    Despite all the hype, it’s apparent that AI is still in its infancy in terms of practical use cases. Most organizations can’t afford to hire the data scientists required to build AI models, let alone find them. Even when they do, the competition for AI talent is so fierce that organizations are also finding it difficult to retain that talent. Giraldo said the goal should be to make it easier not only to build AI models but also slipstream them into both new and existing applications.

    Obviously, many of the best DevOps practices that have already been defined can be applied to AI models. The next big challenge might very well be educating data scientists on how to work more closely with developers to build, test and deploy AI models with applications at scale. In the meantime, the existing DevOps community might want to do as much as possible to help prevent a nascent MLOps community from reinventing some wheels that might already exist.

    — Mike Vizard

  • Are Self-Service Machine Learning Models the Future of AI Integration?

    Are Self-Service Machine Learning Models the Future of AI Integration?

    DevOps teams seeking to step up their mojo in developing cutting-edge artificial intelligence (AI) features are facing a big skills bottleneck when it comes to data analytics and machine learning modeling. As a result, the market is seeing an influx of self-service machine learning models and machine learning-as-a-service offerings designed to help development teams more easily integrate AI capabilities into their software.

    This is coming in direct response to an explosion in demand for AI capabilities in the enterprise. According to Gartner analysts, AI adoption in the enterprise tripled in the past year. A report last fall from MIT Sloan Management Review and Boston Consulting Group found that 91% of enterprises believe that AI will deliver new business growth to them by 2023.

    The trouble is that folding AI functions and predictive analytics into software requires a whole new level of expertise in data science and machine learning from cross-functional DevOps teams. They need added skills to choose the right algorithmic approaches, acquire and manage data, train the models and integrate them into the code base and underlying infrastructure so that everything works properly under the hood.

    “We have heard from customers everywhere that they want to adopt machine learning but struggle to actually get models into production,” said Eric Boyd, vice president of cognitive and AI for Microsoft.

    Ali Farhadi Xnor
    Ali Farhadi, Xnor.ai CEO

    DevOps leaders today already face an uphill battle to keep their developer ranks staffed with well-trained software engineers. Piling on additional requirements for very specialized machine learning and data science expertise further strains those recruitment efforts. According to the most recent Harvey Nash/KPMG CIO Survey, the top one and two technical realms that suffer the biggest skills shortages today are in data science and AI, with 46% and 38% of CIOs respectively reporting recruiting pain in those areas.

    “There’s a huge imbalance between the demand in the market and the supply of the very best AI experience to actually train these models,” said Ali Farhadi, CEO of Xnor.ai, a Seattle firm that just last week released a self-service platform for developers. Called AI2Go, the platform provides pre-trained deep learning models to quickly integrate AI features such as facial recognition and object classification directly into their software.

    “We want to enable anyone who can code to benefit from AI,” Farhadi said.

    The Xnor release comes just a few weeks on the heels of a big launch from Microsoft of several self-service machine learning model tools aimed firmly at MLOps teams—those DevOps teams implementing machine learning capabilities. The new services from Microsoft roll out capabilities to manage the code, data and environments used by MLOps teams throughout the machine learning deployment life cycle.

    “Azure Machine Learning service’s MLOps capabilities provide customers with asset management and orchestration services, enabling effective ML lifecycle management,” explained Jordan Edwards, senior program manager for Azure MLOps, in a blog post on the topic. “With this announcement, Azure is reaffirming its commitment to help customers safely bring their machine learning models to production and solve their business’s key problems faster and more accurately than ever before.”

    Tools such as these are not always going to obviate the need for data scientists and machine learning/AI specialists on DevOps teams. As Farhadi noted, even a simple service like AI2Go requires development teams to choose the right models for the right situations. Nevertheless, this burgeoning market could take the pressure off teams by minimizing the scale of specialized recruiting, while reducing bottlenecks and dreaded re-work.

    “Machine learning projects usually involve several people—data engineers, data scientists, DevOps and others. Conventionally, at each stage of the machine learning life cycle, most, if not all, of these people end up doing an excess of custom coding, redundant work and manual trial and error,” said Craig Stewart, senior vice president of product management and product marketing for SnapLogic, which last fall introduced a self-service machine learning add-on for its integration platform. He noted that one of the primary goals is speed, to help cross-functional teams “build, train, validate and deploy high-performing models faster.”

    — Ericka Chickowski