devopsy.kinsta.cloud

Tag: AI models

  • JFrog Extends DevOps Reach to Manage and Secure AI Models

    JFrog Extends DevOps Reach to Manage and Secure AI Models

    At its swampUP 2023 conference, JFrog today added additional security capabilities to its continuous integration/continuous development (CI/CD) environment as well as making it possible to manage machine learning models within the context of a DevOps workflow.

    The latest edition of the JFrog Software Supply Chain Platform adds an integrated static application security testing tool (SAST) to scan source code alongside the existing JFrog Xray tool for scanning binaries.

    JFrog has also added an Open-Source Software (OSS) Catalog to the JFrog Curation service to make it simpler for DevOps teams to discover specific packages that have been determined to be secure. JFrog is also adding Release Lifecycle Management (RLM) capabilities that enable aggregating software artifacts into an immutable software package that serves as the single source of truth as multiple iterations of an application are developed. JFrog RLM uses anti-tampering systems, compliance checks and evidence capture to collect data and insights to ensure the immutability of signed binaries.

    Finally, ML Model Management capabilities, available in beta, are being added to the platform to make it possible to store AI models, run AI models from Hugging Face, a provider of open source AI models, as a proxy in cache via an application programming interface (API) and apply DevSecOps best practices to govern, secure and ensure licensing compliance.

    JFrog CTO Yoav Landman said that as DevOps teams become more familiar with AI models, it’s becoming increasingly clear they are managed like any other software artifact incorporated into a DevOps pipeline. He noted that the approach provides the added benefit of helping to ensure AI models have not been poisoned by malicious data and prompts that would deliberately cause a hallucination.

    Most AI models today are created by data scientists using machine learning operations (MLOps) best practices that each organization defines. In some instances, those processes include what is known as a feature store that enables data scientists to track which components are being used at any given time. However, once a model is completed, it needs to be integrated into an application—a process that, from a DevOps perspective, is best facilitated by storing an AI model in a Git repository alongside other components that need to be managed.

    It’s still early days as far as the convergence of DevOps and MLOps practices are concerned. Going forward, as it becomes apparent almost every application will be invoke some type of AI model, the need to integrate these workflows is becoming a more pressing concern.

    It’s now only a matter of time before AI models become a pervasive element of the software development life cycle as they are continuously retrained and updated. It will generally fall to DevOps teams to manage the deployment and updating of those AI models in collaboration with the data science teams that train them. The immediate challenge, of course, will be to bridge the significant cultural divide between two very different types of IT professionals.

  • OpenAI to go Open Source — Elon Musk was a ‘Huge Idiot’ ¦ Mojo Risin’

    OpenAI to go Open Source — Elon Musk was a ‘Huge Idiot’ ¦ Mojo Risin’

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Sources say it’s an open source future for OpenAI, and improving AI perf with Mojo. (more…)

  • Why MLOps Hurts: 3 Pain Points for DevOps Teams

    Why MLOps Hurts: 3 Pain Points for DevOps Teams

    Machine learning operations (MLOps) is a practice that aims to improve the collaboration and communication between data scientists and IT professionals in the development, deployment and maintenance of machine learning models.

    MLOps aims to streamline the process of building, testing and deploying machine learning models by automating and standardizing the various tasks involved. This includes automating the training and testing of models and the deployment and management of models in production.

    MLOps involves using tools and processes to automate the various steps involved in machine learning model development and deployment. This includes tools for version control, continuous integration and continuous deployment (CI/CD), and tools for monitoring and managing the performance of machine learning models in production.

    By adopting MLOps practices, organizations can improve the speed and efficiency of their machine learning model development and deployment and reduce the risk of errors and inconsistencies in the process.

    What Role do DevOps Engineers Play in MLOps?

    In MLOps, DevOps engineers play a crucial role in supporting the development and deployment of machine learning models. Their responsibilities may include:

    • Setting up and maintaining the infrastructure needed to support machine learning models, including hardware and software environments, data pipelines, and monitoring systems.
    • Automating the various tasks involved in machine learning model development and deployment, including training, testing, and deployment, using tools such as continuous integration and continuous deployment (CI/CD).
    • Monitoring the performance of machine learning models in production and troubleshooting any issues that may arise.
    • Collaborating with data scientists to understand the requirements and constraints of machine learning models and to help them implement and deploy their models effectively.
    • Managing the release process for machine learning models, including coordinating with different teams, conducting testing and validation, and rolling out updates and improvements.

    Challenges to Implementing MLOps

    MLOps is not without its challenges. Here’s what to consider when developing and deploying your machine learning pipelines. 

    Data Security

    Data security is a key challenge of MLOps for DevOps teams because machine learning models rely on large amounts of data to function effectively. Ensuring the security and integrity of this data is critical to the success of the model and the overall MLOps process.

    There are several risks to data security that DevOps teams need to consider in MLOps:

    • Data breaches: Machine learning models often work with sensitive data, such as personal information or financial data. If this data is not secured properly, it could be accessed by unauthorized parties, leading to data breaches and potential legal and financial consequences.
    • Data tampering: Machine learning models rely on the accuracy and integrity of the data they are trained on. If the data is tampered with or manipulated, it could affect the model’s performance and lead to incorrect or unreliable predictions.
    • Data privacy: Machine learning models may work with data that is subject to privacy regulations, such as the General Data Protection Regulation (GDPR) in the European Union. Ensuring compliance with these regulations is critical to avoid fines and other consequences.

    Model Deployment

    Model deployment is a key challenge of MLOps for DevOps teams because it involves integrating the machine learning model into an existing application or system and setting up the infrastructure to support it. This can be complex, especially if the model is part of a larger application or system, and requires careful planning and coordination.

    There are several challenges that DevOps teams may face when deploying machine learning models:

    • Integration: Machine learning models need to be integrated into an existing application or system in a way that is seamless and does not disrupt the overall functionality. This can be a complex task that requires careful planning and testing.
    • Testing: It is important to thoroughly test the machine learning model before it is deployed to production to ensure that it functions correctly and meets the required performance standards. This may involve testing the model on a variety of data and in different environments to ensure that it is robust and reliable.
    • Performance: Machine learning models may require specialized hardware and software environments to support their performance. Setting up and maintaining these environments can be a challenge for DevOps teams.
    • Scalability: Machine learning models may need to scale up or down depending on the volume of data and the demands placed on them. Ensuring that the model is able to handle different workloads and scale appropriately is a key challenge for DevOps teams.

    Model Performance and Maintenance

    Model performance and maintenance is a key challenge of MLOps for DevOps teams because machine learning models require ongoing monitoring and maintenance to ensure that they continue to perform well in production. This can be a complex and time-consuming task that requires close collaboration between data scientists and IT professionals.

    There are several challenges that DevOps teams may face when it comes to model performance and maintenance:

    • Model drift: Machine learning models can suffer from model drift over time, where the model’s performance begins to degrade due to changes in the data or the business requirements. This can be difficult to detect and can significantly impact the model’s accuracy.
    • Data changes: Machine learning models rely on the accuracy and integrity of the data they are trained on. If the data changes over time, the model may need to be retrained or updated to ensure that it continues to perform well.
    • Hyperparameter tuning: Machine learning models often have several hyperparameters that need to be set to optimize their performance. Identifying the optimal values for these hyperparameters can be challenging and the model may need to be retuned over time to maintain its performance.
    • Model versioning: Machine learning models may need to be updated or replaced over time to improve their performance or to address issues. Managing different versions of the model and coordinating the rollout of updates can be a challenge for DevOps teams.

    5 MLOps Best Practices to Help DevOps Teams

    There are several best practices that DevOps teams can adopt to help overcome the challenges of MLOps:

    1. Automate as much as possible: Automating tasks such as model training, testing and deployment can help streamline the MLOps process and reduce the risk of errors and inconsistencies.
    2. Use version control: Version control systems such as Git can help track changes to the machine learning model and its supporting infrastructure, making it easier to roll back changes or deploy updates.
    3. Use CI/CD: CI/CD pipelines can help automate the process of building, testing, and deploying machine learning models, improving the speed and efficiency of the MLOps process.
    4. Monitor and log model performance: Setting up monitoring and logging systems can help DevOps teams track the performance of machine learning models in production and identify any issues that may arise.
    5. Collaborate and communicate effectively: MLOps requires close collaboration between data scientists and IT professionals. Establishing clear communication channels and effective collaboration processes can help ensure that all parties are working towards a common goal.

    Conclusion

    In conclusion, MLOps can be a challenging process for DevOps teams due to the complexity and specialized nature of machine learning models. There are several pain points that DevOps teams may face when implementing MLOps, including the complexity of setting up and maintaining the necessary infrastructure, the difficulty of collaborating and communicating effectively with data scientists and the challenges of ensuring the security and integrity of data used to train and test machine learning models. 

    By adopting best practices such as automation, version control, CI/CD pipelines and effective collaboration and communication, DevOps teams can help overcome these challenges and improve the speed and efficiency of machine learning model development and deployment.

  • Stable Diffusion Goes Public — and the Internet Freaks Out

    Stable Diffusion Goes Public — and the Internet Freaks Out

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters. (more…)

  • Top 3 Requirements for Next-Gen ML Tools

    Top 3 Requirements for Next-Gen ML Tools

    As the machine learning market matures, new tools are evolving that better match data science and machine learning teams’ needs. Vendors, both open source and private, have been quick to introduce new products that better meet requirements making it easier and faster to develop models and enable collaboration.

    These new offerings range from cloud-based platforms that make it easy to build and deploy models to specialized software that can speed up the training process. As an example, the existing field of ML experiment tracking is a multi-billion dollar market with a crowded amount of vendors: Comet ML, Weights & Biases, Neptune AI, Databricks’ MLflow and more. These tools will need to evolve to move beyond meeting only the basic requirements of ML teams like experiment tracking, visualizations and model training to address collaboration and technical requirements so these teams can get models to market faster.

    The types of teams that are driving modern requirements for data science are the bet-the-farm types, where it’s all about finding new technology and solutions to drive efficiencies and get models to market faster for their organizations to maintain a competitive edge. Going forward, we’re seeing the following as requirements for ML teams as they evaluate tools to add to their MLOps stack.

     An Open, Modular Approach

    Data scientists and ML engineers use multiple tools to do their jobs, from Jupyter notebooks for model coding to DVC for experiment and data versioning, to automation tools like AirFlow to model visualization tools. Solutions will need to be open to integration with other tools instead of a walled garden that many ML tools, like those offered by cloud vendors (e.g. Amazon SageMaker, etc.) are now.

    Familiar Developer Experience

    ML developers and engineers are familiar with standard DevOps tools and workflows like GitOps (think GitHub or GitLab) and live in CLI-oriented solutions. ML tools should integrate tightly with these existing tools instead of complicating the developer experience with a separate ecosystem. And as a part of the developer experience, workflow is also important. Data scientists can run hundreds of experiments using their preferred CLI-based solutions but, often, all those experiments are streamed and posted into existing ML tools. A more optimal approach would be to offer data scientists the experience of running experiments locally and letting them commit only the most important experiments for further discussion and collaboration so the team isn’t overwhelmed.

    Strong Connection to Source Code

    This approach makes a stronger foundation for something like tracking ML experiments. As ML teams race to get models to market and continually improve them, being in lockstep with DevOps teams to deploy those ML models into production is critical. An example would be a developer tracking five experiments with changing hyperparameters. For these experiments, they may have a link to a Git repo for a particular version of source code. But what if the source code changes or data set changes are not committed or reported in their existing ML solution? Then, the connection is lost and existing metrics would be based on a previous version of source code or data that’s no longer accurate. That lost connection slows down time-to-market and makes it harder to optimize models.

    As seen with the above examples, ML teams juggle myriad tools and workflows as they develop models for various applications and services. The requirements discussed will help these teams to collaborate better and build models faster in an increasingly heterogeneous MLOps landscape.

  • Computer Vision is Building the Future

    Computer Vision is Building the Future

    Developing an accurate, regulated and purpose-driven facial recognition software is the challenge of computer vision engineering. It is also the future of how people do their everyday tasks: from unlocking a phone to signing into a bank app or logging the time they get to the office. Facial recognition amazes more and more with its new applications. Today, people can go to a concert without the tickets they bought online, they just have to bring…their face and show it at the gate.

    Computer vision science has been working on facial and object recognition software for some decades now. From the beginning, the main goal of this engineering, framed in the AI field, has been to give computers a visual understanding of the world.

    The goal gets fulfilled through a three-step process: image acquisition, image processing and image understanding. Each step can be done thanks to different kinds of technology. For example, image acquisition can be done using a web camera, an embedded phone camera or a 3D camera, among other types of lenses.

    With the help of algorithms computer vision is currently capable of recognizing and tracking objects. But it can go much further than that. Now, neural networks are at such a high level that they are capable of restoring and even creating new images. All of these features that seemed to be available only in futuristic or science fiction movies are now in the market thanks to AI algorithms, such as Generative Adversarial Networks (GAN).

    Today, big cloud providers have released their own AI and vision models including facial recognition APIs. Nevertheless, these models are conceived with rather general purpose algorithms that might be sensitive to specific environmental details, but don’t offer the possibility to retrain models with data gathered in a productive environment.

    The good news is some companies can offer AI models retraining in customers’ environment to accommodate any need or requirement, including usage of depth cameras or ToF sensors. But of course, these developments bring along some challenges. The way to succeed is to be ready and thoughtful in every step. This way, any surprises along the process will be controlled and won’t destabilize the project offered to a client.

    Last year at intive, we built a cognitive department with our internal AI experts as well as the best people we could find in the market. Eleven people shaped the core team that developed advanced facial recognition software in only one year of work. Our objective was to create an AI model fast suited to specific environmental conditions of small, medium-sized and big companies. We created AI models capable of using RGB, depth cameras and ToF sensors.

    Depth cameras enable the API to distinguish a real human face from flat images such as screens, tablets and paper. The new intive’s API can estimate age, detect sex and recognize a fraudulent face. It identifies eyes, nose and the ratio of length and distance of individual facial elements. It can even discern the differences between identical twins. The only face characteristics not taken into account are hair, mustache and beard; this is to make sure shaving or changing the hair style won’t impact the recognition capabilities.

    Here are the three major challenges for computer vision development.

    Legal and Compliance

    First, at any computer vision development, it is crucial to have the General Data Protection Regulation (GDPR) in mind. You must ensure you are allowed to manage or process certain biometric data. Imagine the city government installs cameras over the city parks, these are useful not only for recording but they are also capable of tracking people, thus enhancing the streets security. In this case, people’s identity has to be somehow recognized, and the way of doing this is by processing biometric data. If the person tracked does not know this is being done, a GDPR violation happens.

    Anytime a system needs to process people’s biometric data that can identify them, they need to give consent to process this data. To solve this, intive installed a button on our API that releases the biometric calculations and the person’s identification. By pressing this button, the person automatically gives consent to start the process. Additionally, while enrolling to the system, employees need to sign the consent.

    Same Environment, Always

    In every computer vision case, environment is an aspect you will always have to deal with. This refers to the technology used for acquiring images during the development, testing and productive stages of the project: the type of camera and its calibration, the drivers, libraries, settings and the light conditions.

    To ensure the API’s best accuracy, all stages must be developed with the same environmental conditions. Otherwise, the system loses its best accuracy and could come-up with worse results in the productive environment and in the development one. When environments change, the error rate rises.

    To ensure success, it is crucial to collect all possible data from the final environment and retrain the developed model with this data. For our computer vision project, we created several different production environments and we gathered all the data that came up. There will always be differences, for example, the lighting conditions are hard to control in every stage. But the more data you have, the fewer errors will appear.

    Set Acceptance Criteria and Meet It with the Data You Have

    From the beginning of the project, it is very important to set the accuracy you are aiming for. This means you need to decide how often the system recognizes an enrolled user correctly. To understand this better, if you have 10 users, and you set a classification accuracy of 90%, this means nine people are going to be recognized accurately.

    False positive rate (FPR) is linked to this and it needs to be decided too. This value represents how often the system recognizes a person that enters the system with different characteristics than the enrolled one. For example, if you are using the system to control employees’ entrance to an office, how often will a person who is not an employee (and not enrolled in the system) gain access to the building?

    Both values, accuracy and FPR, are strictly bound. Better accuracy is directly related to an increase in FPR. You need to decide if you want better accuracy and higher FPR or lower accuracy and lower FPR. This can be controlled smoothly along the course of the development if you set it according to the available data.

    Accuracy and FPR will change based on the quality of data–the photos used. The more pictures (data) per enrolled person, the more accurate the system is. Pictures from different angles allow the system to have a better chance of being effective. In our case, we took a minimum of five pictures for every enrolled person.

    Working on a computer vision project with facial recognition software can seem intimidating. But it is just a matter of being well organized. Take into account the three challenges highlighted above and you will be on the right track to success.

    — Grzegorz Tymiński