devopsy.kinsta.cloud

Tag: data science

  • Are LLMs Leading DevOps Into a Tech Debt Trap?

    Are LLMs Leading DevOps Into a Tech Debt Trap?

    Everyone thought AI would free us from toil. Unfortunately, that’s not true. What it will do, however, is lead to an explosion of technical debt. Some of the most alluring uses of LLMs have the potential to lead to some of the worst outcomes for enterprises and their DevOps and platform engineering teams.

    SK Ventures and Adrian Cockcroft raised the alarm bells in their warning posts about the tipping points LLM introduced into work and society. And in my own Gluecon 2023 presentation, I challenged just how powerful and disruptive the new generation of AI/ML will be for ITOps and DevOps efforts.

    Working Faster is not Working Smarter

    There is no doubt that LLMs are powerful and will change how we think about expertise and knowledge work.  While they are not truly innovating, their ability to draw on such a vast pool of human expertise makes it difficult to distinguish from true creative work. Since they are unbounded, they will cheerfully use the knowledge to churn out terabytes of functionally correct but bespoke code.

    It’s easy to imagine a future where LLMs crank out DevOps scripts 10x faster. We will be supercharging our ability to produce complex, untested automation at a pace never seen before! On the surface, this seems like a huge productivity boost because we (mistakenly) see our job as focused on producing scripts instead of working systems.

    But we already have an overabundance of duplicated and difficult-to-support automation. This ever-expanding surface of technical debt is one of the major reasons that ITOps teams are mired in complexity and are forever underwater. Generating even more scripts, even if they were better written, does not reverse the trend.

    Can AI/ML Avoid This Technical Debt Trap?

    It depends on how we use the expertise in the models. Instead of asking it to generate new code, we could ask it to interpret and modify existing code. For the first time, we have tools to take down the “not invented here” barriers we’ve created because of the high cognitive load of understanding code. If we can help people work more effectively with existing code, then we can actually converge and reuse our systems. By helping us expand and operate within our working systems base, LLMs could actually help us maintain less code.

    Imagine if the teams in your organization were invested in collaborating around shared systems! We haven’t done this well today because it takes significantly more time and effort. Today, LLMs have thrown out those calculations.

    Taking this just one more step, we can see how improved reuse paves the way for reduction of the number of architectural patterns. If we improve our collaboration and investment in sharing code, then there is increased ROI in making shared patterns and platforms work. I see that as a tremendous opportunity for LLMs to improve operations in a meaningful way.

    It comes down to whether we ignore complexity by replacing it or embrace complexity by understanding it. I see this as a choice between independence or collaboration, as illustrated by the table below.

    LLM Driving Independence

    LLM Driving Collaboration

    Create new code

    Interpret existing code

    Replace existing code

    Maintain existing code

    Design new architectures

    Be a guide and mentor

    Build new stacks

    Evaluate alternatives

    Bypass documentation

    Improve documentation

    Can AI Help us Collaborate?

    We keep taking the same hardware and software components and then building unique systems. That variation makes it impossible to leverage each other’s work. At its heart, the reusability problem lies in the difficulty of understanding systems, and that makes it nearly impossible for us to work together, collaborate, reuse and accelerate.

    Breaking this cycle has been our mission at RackN for nearly a decade. We’ve seen tremendous ROI from availability, reliability and speed when customers are able to leverage standardized infrastructure pipelines. We’ve seen firsthand that it is possible to guide teams into collaboration with sufficient guidance, mentoring and training. Collaboration creates 10x outcomes even without AI, but it takes building expertise, deliberate effort and leadership discipline to implement.

    Why does collaboration create 10x ROI?  It works because true innovation actually comes from understanding and building on top of systems rather than rebuilding them from scratch over and over again. The key is being able to leverage LLMs to better understand existing automation and platforms instead of creating yet another bespoke stack.

    Enterprises Should use LLMs to Drive Collaboration

    I believe that the most effective enterprises will use these tools to elevate their IT leaders and architects to focus on collaboration and reuse. They must actually direct them to produce less bespoke automation; instead, they can use expertise in these tools to find and maintain existing systems. LLM presents options, but collaboration requires new thinking and strong leadership.

  • MLOps Vs. DevOps: What’s the Difference?

    MLOps Vs. DevOps: What’s the Difference?

    Machine learning operations, or MLOps for short, is a key aspect of machine learning (ML) engineering that focuses on simplifying and accelerating the process of delivering ML models to production and maintaining and monitoring them. MLOps involves collaboration between different teams including data scientists, DevOps engineers, IT specialists and others.

    MLOps can help organizations create and improve the quality of their AI and machine learning solutions. Adopting MLOps allows machine learning engineers and data scientists to collaborate to improve model performance by implementing continuous integration and continuous deployment (CI/CD) practices. It accelerates the ML model development process by incorporating the appropriate monitoring, governance, and validation of ML models.

    What Is DevOps?

    DevOps combines the concepts of development and operations to describe a collaborative approach to performing the tasks usually associated with separate application development and IT operations teams. DevOps, in its broadest sense, is a philosophy that encourages improved communication and cooperation between these (and other) teams within an organization. 

    In its narrowest sense, DevOps refers to adopting practices that enable the deployment and maintenance of iterative application development, automation, and programmable infrastructure. It also includes changes in the workplace culture, like trust-building and bonding between developers, system administrators and other team members. DevOps aligns technology with business objectives and can transform the software delivery chain, job functions, services, tools and best practices.

    MLOps Vs. DevOps: Key Differences

    Here are some of the main differences between MLOps and traditional DevOps.

    Development

    The concept of development refers to different things in each model, with a slightly different CI/CD pipeline.

    DevOps:

    • Usually, the code creates an interface or application. 
    • The code is wrapped into an executable or artifact before being deployed and tested with a set of checks. 
    • Ideally, this automated cycle will continue until the final product is ready.

    MLOps:

    • The code enables the team to build or train machine learning models. 
    • The output artifacts include serialized files that can receive data inputs to generate inferences. 
    • Validation involves checking the trained model’s performance based on the test data.
    • This cycle should also continue until the model reaches a specified performance threshold.

    Version Control

    DevOps:

    • Version control typically only tracks changes to code and artifacts.
    • There are few metrics to track.

    MLOps: 

    • MLOps pipelines usually have more factors to track. Building and training an ML model involves an iterative experimentation cycle, requiring tracking of various metrics and components for each experiment (essential for later audits). 
    • Additional components to track include training datasets, model building code and model artifacts. 
    • Metrics include hyperparameters and model performance indicators, such as error rates.

    Reusability

    DevOps:

    • DevOps pipelines focus on repeatable processes.
    • Teams can mix and match processes without following a specific workflow.

    MLOps: 

    • MLOps pipelines repeatedly apply the same workflows. The common framework across projects helps improve consistency and allows teams to progress faster because they start with familiar processes. 
    • Project templates offer structure, enabling customization to address the unique requirements of each use case.
    • Uses centralized data management to consolidate the organization’s data to accelerate the discovery and training processes. Common approaches to centralization include a single source of truth and data warehouses.  

    Continuous Monitoring

    Monitoring is essential for both DevOps and MLOps, but for slightly different reasons.

    DevOps:

    • Site reliability engineering (SRE) has been trending over the past few years, emphasizing the need for monitoring software from development through to production deployment. 
    • The software does not degrade in the way an ML model does. 

    MLOps:

    • Machine learning models can degrade quickly, requiring constant monitoring and updating.
    • Conditions in the production environment affect the model’s accuracy. After deployment to production, the model starts generating predictions based on new data from the real world. This data is constantly changing and adapting, reducing model performance. 
    • MLOps ensures that algorithms remain production-ready by incorporating procedures to facilitate continuous monitoring and model retraining.

    Infrastructure

    DevOps and MLOps both rely heavily on cloud technology, but have different operational requirements.

    DevOps relies on infrastructure such as:

    • Infrastructure-as-code (IaC)
    • Build servers
    • CI/CD automation tools

    MLOps relies on infrastructure such as:

    • Deep learning and machine learning frameworks
    • Cloud storage for large datasets
    • GPUs for deep learning and computationally-intensive ML models

    DevOps and MLOps Trends

    Here are some of the major trends driving the development of DevOps and MLOps.

    GitOps

    A new evolution of the DevOps workflow, GitOps is a new paradigm for controlling and automating infrastructure. A Kubernetes-oriented paradigm enables developers and operations teams to manage Kubernetes clusters and deliver containerized applications using Git. Implementing Git workflows for operations and development teams allows developers to leverage Git pull requests to manage software deployments and infrastructure.

    GitOps incorporates existing development tools to manage cloud-native and cluster-based applications with CI/CD. It automatically deploys, monitors, and maintains cloud-native applications using a Git repository as the single source of truth.

    GitOps is a way to implement and maintain clusters in Kubernetes. Continuous delivery and deployment allow developers to build, test, and deploy software faster through incremental releases. Kubernetes continuous integration and runtime pipelines must be able to read and write files, update container repositories, and load containers from Git. GitOps helps businesses manage their infrastructure with version control, real-time monitoring, and alerting of configuration changes.

    Synthetic Data

    Synthetic data is any information generated artificially instead of collected from real events. Algorithms generate synthetic data for use as a replacement for operational and production test datasets. Synthetic datasets are also useful for validating mathematical models and training machine learning models.

    Benefits of synthetic data include:

    • Minimizing the constraints associated with using sensitive and regulated data.
    • Customizing data to specific requirements and conditions not available in real-world data.
    • Generating data for testing software quality and performance for DevOps teams.

    Codeless Machine Learning and AI

    Machine learning often involves computer code to set up and process model training, but this is not always the case. Codeless machine learning is a programming approach that eliminates the need for ML applications to go through time-consuming processes.

    Codeless ML eliminates the need for experts to develop system software. It is also simpler and cheaper to deploy and implement. Using drag-and-drop input during machine learning processes can simplify training efforts in the following ways:

    • Evaluating results.
    • Dragging and dropping training data.
    • Creating predictive reports.
    • Using plain-text queries.

    Codeless ML gives developers easy access to machine learning applications, but it is no substitute for an advanced, nuanced project. This approach is suitable for small businesses that lack the capital to maintain an in-house data science team.

    TinyML 

    TinyML is a new approach to machine learning and AI model development. It involves running models on devices with hardware restraints, such as microcontrollers powering smart vehicles, refrigerators and electric meters. This strategy is the best fit for these use cases because it speeds up the algorithm—there is no need for the data to go back and forth to a server. It is especially important on large servers and can speed up the entire ML development process.

    Running TinyML programs on an IoT edge device has many benefits:

    • Lower energy consumption.
    • Reduced latency.
    • User privacy guarantees.
    • Reduced bandwidth requirements.

    Using TinyML offers greater privacy because the computation process is entirely local. It consumes less power and bandwidth, resulting in lower latency because it doesn’t require sending data to a central location for processing. Industries that are taking advantage of this innovation include agriculture and healthcare. They typically use IoT devices embedded with a TinyML algorithm to monitor and predict real-world events using collected data.

    Conclusion

    In this article I covered the key differences between MLOps and DevOps:

    • Development—DevOps pipelines focus on developing a new version of a software product, while MLOps focuses on delivering a working machine learning model.
    • Version control—DevOps is mainly concerned with tracking binaries and software artifacts, while MLOps tracks additional factors like hyperparameters and model performance.
    • Reusability—DevOps and MLOps both strive to create reusable processes and pipelines, but use different strategies to achieve repeatability. 
    • Continuous monitoring—Monitoring is important to DevOps, but even more important in MLOps because models degrade in performance due to model and data drift.

    Finally, I covered a few key trends that will transform DevOps and MLOps in the near future. I hope this will be useful as you discover your place in the new and exciting development ecosystem.

  • Why You Should Use GitOps to Experiment With AI

    Why You Should Use GitOps to Experiment With AI

    As the pace at which artificial intelligence (AI) models are being constructed and inevitably updated starts to increase, it’s becoming more apparent that the pace at which data science teams are currently experimenting also needs to dramatically quicken.

    Many data science teams today are fortunate if they manage to successfully deploy two AI models in a production environment over the course of 12 months. It’s already apparent, however, thanks in part to the rise of digital business transformation, that there are potentially hundreds of use cases for AI models within each enterprise IT environment. Truth be told, it’s hard to envision an application being developed today that would not incorporate one or more AI models. The challenge is, the demand for AI capabilities is now far outpacing the ability of data science teams to deliver.

    The best way to solve this issue is to recognize that AI models and the components employed to construct them are, in reality, just another kind of software artifact. In fact, the need to accelerate the development of applications using artifacts is largely a problem already solved. Rather than reinventing the wheel, the time has come to apply what we already know to the development of AI models.

    The Rise of GitOps-Based Processes

    Modern software development teams today routinely make use of Git-based repositories to share code and collaborate. These repositories are jam-packed with vetted software artifacts that development teams continuously reuse via an opinionated instance of a DevOps workflow based on a set of GitOps best practices.

    As a subset of DevOps, a GitOps workflow enables data science teams to employ a Git repository that already exists within their software development environment to manage artifacts. Git enables versions of files to be created using what is known as a Git branch. They also employ concepts known as tags (Git Tag) and remotes (where the server is located). Those three different concepts, or references, exist under the hood in Git. In fact, the entire flow is based on those concepts. When a new feature is implemented, a new branch is created and then merged into the production branch with a tag that says something like “This is a new version that needs to be deployed.”

    An AI model, like any other software artifact, needs to be updated. As new data sources become available, an AI model is likely to drift beyond the parameters originally intended. A rapid change in business conditions may also require changes to an AI model. The same processes that DevOps teams apply to update any software artifact can also be applied to rapidly update an AI model. Employing a Git repository to also manage AI artifacts extends the benefits that software development teams enjoy to data science teams.

    Often, organizations will have an MLOps solution to streamline AI model development that is completely separate from their DevOps tools. These solutions will have dedicated features like version tracking for data and experiments specifically for AI models. But many MLOps solutions exist in a closed ecosystem, meaning they don’t work off of existing tools like GitHub used on the DevOps side. 

    Also, data science teams have to separately buy, deploy, and manage these closed MLOps solutions. Going with a GitOps-oriented solution streamlines management of added MLOps tools and makes it easier for both application developers and data science teams to gain visibility into each other’s respective efforts. The end result is not just faster AI experimentation at lower cost; the likelihood that more AI models will successfully be deployed in a production environment increases dramatically. 

    Software and AI Teams Working Together

    The convergence of DevOps and MLOps is already inevitable. The longer organizations continue to employ isolated MLOps platforms to build and deploy AI models, the more expensive it will become to meld MLOps and DevOps processes. 

    Data science teams are conservative by nature. The difference between success and failure often comes down to how thoroughly an AI model is validated. However, time to market doesn’t have to be sacrificed to ensure an AI model works as promised. There is no such thing as a great AI model that arrives late when rivals are racing to develop similar AI capabilities—there are only missed opportunities. Speed enabled by the reusability of artifacts that drives faster experimentation using GitOps workflows is of the essence now more than ever.

  • Why We’re Swimming in Dashboards and Drowning in Data

    Why We’re Swimming in Dashboards and Drowning in Data

    Knowing how to use a dashboard is table stakes in the enterprise. Information technologies generate countless volumes of data from multiple sources, and dashboards are meant to harness all of that information, reduce the noise and display the stats you need to do your job. Yet businesses are still struggling to unlock the value of their data, with over half of global businesses (57%) claiming they can identify and capture just 70% of their data, according to a study conducted by IDC and commissioned by Qlik.

    As in “The Rime of the Ancient Mariner,” there is water, water everywhere, but not a drop to drink. Enterprises are inundated with dashboards and still struggle to make effective decisions with the information in front of them.

    The Limits of Analytics

    There are many reasons why a dashboard may communicate ‘bad’ or ‘useless’ information. Analytics can’t do it all. A data pipeline may be parched with too little data or flush with the wrong kind of data, limiting the volume and/or value of outputs that can be drawn into the dashboard. The organization may be heavily siloed, rationing the range of information available to the dashboard. Or, the tooling itself might be inadequate for leaders to quickly respond to opportunities and disruption.

    When a decision-maker looks at a dashboard, they want to see things as they could be, as well as how they are. That requires a dashboard that depicts the data itself, but that also generates views of, for example, how it looks when sales activity in the Southwest is increased in the face of a weather event or other externalities. Results don’t have to be taken at face value – they can be analyzed more carefully and intuitively to uncover meaningful insights.

    Investigatory analysis is what modern, interactive dashboards and applications are all about. They support non-linear exploration and search and provide advanced features, such as augmented analytics, to help subject matter experts explore data and generate insights without relying on IT, data scientists or expert analysts. Interactive dashboards also make it possible for business intelligence to be truly self-service, simplifying aspects that were once limited to experts working in much more complex environments. They give all employees access to the insights they need to make better decisions, regardless of their skill level, essentially democratizing analysis across organizations.

    How to Choose the Right Dashboard

    For any business looking to really expand decision making across the business, here’s how to find a dashboard suited for investigatory analysis:

    Accessibility and timeliness: In the past, business users who wanted insights had to submit a request to an analyst, wait for their turn in the queue and go back and forth answering clarifying questions before they could review a dashboard and make decisions. The process took too long, and insights came too late. With modern, interactive dashboards, users can explore all their data using interactive selections (filters), search, and NLP (natural language processing) powered by AI. In the end, they can ask and answer all their questions leading to informed decisions.

    Flexibility: Modern dashboards are not just static displays of content with limited filtering. They are rich analytics applications that provide unmatched interactivity and exploratory power. Business users aren’t limited to rigid, linear analysis on partial subsets of data. Instead, they have the freedom to explore all their data however they like. Analytics are always in context together, giving users peripheral vision, and all data relationships are exposed – including what’s related to your selections and what’s not.

    Performance: Interactivity should be driven by outcomes. Novel techniques to optimize and store data, index and understand relationships and deliver high-performance calculation all help to go beyond the limits of query-based and RDBMS architectures. Interactivity should also be enhanced with AI to deliver next-level analytical power, in new forms, across a full range of user experiences.

    A Crucial Tool in the Stack

    Turning data into business value is what a dashboard should—and can do—for the business. When a business user looks at a static dashboard or visualization, they usually end up with more questions than answers. To make an informed decision, they need to go through a process of questioning and evaluating – and then questioning again – an interactive display of business-critical trends until the right insight is fully uncovered. This iterative process is unique to every user and their lines of thinking, and modern dashboards allow all users, beyond data experts and analysts, to build visualizations and queries.

    In addition to creating business value, interactive dashboards help rally key stakeholders in organizations and allow them to act quickly to changing market needs. Take, for example, a major luxury fashion retailer; parent company of three well-known apparel brands. Their businesses hinge on having the products consumers want and need, when they want and need them. Their assortment planners need to analyze their sales mix and inventory mix by product attribution to identify mismatches and, more importantly, understand the drivers behind the mismatches to make more accurate inventory recommendations. Interactive dashboards allow them to investigate their data in different ways and ultimately inform strategic decisions for the business moving forward.

    In addition to empowering users to transform data into business value, dashboards ultimately drive the need for data literacy. Sharing dashboards throughout all levels of an organization fosters a data-driven culture by encouraging exploration of data – and that’s how new discoveries and innovations can be made.

    So while some say the dashboard is dead, I say long live the interactive dashboard.

  • From the Bookshelf #1

    From the Bookshelf #1

  • Top Open Source Tool: Data Science

    Top Open Source Tool: Data Science

  • 5 Best Machine Learning Use Cases

    5 Best Machine Learning Use Cases

    Many industries are struggling with the ongoing COVID-19 pandemic, and the IT industry itself and the broader trend of transition to remote work over the last year revealed many areas where traditional approaches to managing businesses created unnecessary waste. Data science and its counterpart, machine learning, revealed that expansion in the ways technology can facilitate new ways of working is nearly limitless.

    Machine learning uses powerful algorithms to discover insights based on real-world data. These insights can then be used to make predictions about future outcomes. As new data becomes available, machine learning-enabled programs can automatically adapt and produce updated predictions. As with any tool, machine learning is not a silver bullet. However, there are many situations in which the technology can outperform linear and statistical algorithms.

    Here are five of the most common use cases where machine learning can make a big difference:

    When engineers can’t code rules for certain problems. Many human-oriented tasks (such as recognizing whether an email is spam) aren’t solvable using simple (deterministic), rule-based solutions. Because so many factors may influence an answer, engineers would have to write and frequently update billions of lines of code. In addition, when rules depend multiple factors and when those rules overlap or need fine-tuning, it becomes difficult for humans to code precise rules. Fortunately, machine learning programs don’t require users to encode actual patterns. These programs only need proper algorithms to extract patterns automatically.

    When you need to scale a solution to millions of cases. In finance, for example, you might be able to manually categorize a few hundred payments as either fraudulent or not. However, this becomes tedious or impossible when dealing with millions of transactions. As user bases grow, it’s no longer feasible for organizations to process payments by hand – end-users today want answers about their money in milliseconds, not minutes or hours. Machine learning solutions are effective at handling these types of large-scale problems with little to no human intervention.

    When you can do it manually, but it’s not cost-efficient. There are situations in which in-house experts could process many requests quickly and accurately, but at a high cost. For instance, imagine you assess DMV forms for in-state and out-of-state car purchases to determine their validity before passing them on. In this situation, the business processes are well-defined, optimized and serialized. It may take only a few minutes to check each form thoroughly. But allocating so much manual labor to this work is likely not the best use of your budget. Machine learning, on the other hand, offers predictable, pay-as-you-go pricing for fully scaled operations.

    When you have a massive dataset without obvious patterns. Consider this – you’ve successfully prepared a well-curated dataset and know the underlying problem you’re trying to solve. However, you don’t see any explicit patterns in the data, preventing you from encoding those validations. Plus, there are many typos, missing fields, and other human-caused errors with no validation in place. You may even know the data is poor quality and can manually determine every affected row. But you can’t see any actual connections between valid and invalid records. Machine learning algorithms can solve this problem. They can find hidden connections between data points that aren’t clear to humans. Tools like interpreting tracers can even describe how machine learning models arrived at their conclusion(s).

    When you live in an ever-changing universe (adaptive). The world, and its problems, are always changing. A problem you solved yesterday can, today, easily mutate into something else entirely, rendering your previous solution inefficient or even useless. For example, if your organization processed medical appointment records to extract diagnoses, procedure information and billing codes, your rules might have to evolve constantly. However, you can’t make updates in real-time, 24/7. Meanwhile, incorrectly labeled items could lead to insurance rejections, huge fines and legal penalties. One major advantage of machine learning methods is that they can learn from data across the entire life cycle of your application – from the first line of code written to the moment when the model is finally shut down. Moreover, it’s important for production-grade systems to have feedback loops so that you can catch the moment when your model no longer solves problems correctly.

    It’s important to remember that machine learning is a tool – it’s not magic. Machine learning models are, essentially, advanced math-based algorithms, which identify patterns in data and learn from them. However, when properly applied to the right use cases, machine learning can reduce the amount of time spent on error-prone manual IT operations, adding significant business value and greatly reducing IT costs.

  • Data Measured in Terms of Real Aggregate Value

    Data Measured in Terms of Real Aggregate Value