devopsy.kinsta.cloud

Category: Cloud-native

Microsoft GitHub Alignment Fuels Birth of the AI Developer

Microsoft wants developers to evolve into AI developers. It’s a predictable enough progression i.e. first, developers were mainframe developers, then they became PC developers, mobile developers and then cloud developers. Next, logically, software engineers will have to become automation-centric AI developers with closer proximity to the models and engines that enable our new intelligence services.

To facilitate this shift, Redmond is of course championing the use of Microsoft Azure AI services as a route to building custom AI applications. To achieve the critical mass it so dearly seeks, Microsoft is partnering with GitHub to enable its 100 million+ developers to build AI applications directly on the GitHub platform.

Although Microsoft acquired GitHub in 2018 and the platform has subsequently been bolstered by CI/CD pipeline tools and wider security capabilities, it remains a comparatively independent entity in the style of a Red Hat inside IBM.

GitHub Models Marketplace

Microsoft has now made Azure AI Studio model selection functions available to developers through GitHub Models, a “marketplace” portal where software engineers who want to develop a generative AI application can find and experiment with AI models for free before considering token-based payment once an application moves to production. Efforts from Microsoft here also include the provision of APIs designed to enable production-ready AI applications.

GitHub Marketplace itself exists as a “free playground” where programmers can adjust model parameters and submit prompts to see how an AI model responds. This work also features integrations with Codespaces and Microsoft Visual Studio Code. GitHub Codespaces provides a secure development environment with a selection of prebuilt built-in resources that enjoy native integration with the GitHub platform.

GitHub Models came about in August of this year as a means of enabling the rise of the AI engineer with access to leading large and small language models.

The Rise Of The AI Engineer

“As AI model innovation accelerates, Azure remains committed to delivering the leading model selection and greatest model diversity to meet the unique cost, latency, design and safety needs of AI developers. Today, we offer the largest and most complete model library in the market, including the latest models from OpenAI, Meta, Mistral and Cohere and updates to our own Phi-3 family of small language models,” notes Asha Sharma, corporate vice president of AI platform, Microsoft.

With GitHub Models, Sharma suggests developers can explore the latest models along with other AI innovations such as next-generation “frontier models” i.e. those models that rank as large scale with many parameters and complex algorithmic logic such as Google Gemini, Anthropic’s Claude, AlphaFold’s DeepMind and indeed GPT-4 from OpenAI.

“For most of us, learning to be a developer didn’t happen on a linear path in the classroom. It took [plenty of] practicing, playing around and learning through experimentation. The same is true today for AI models. In the new interactive model playground, students, hobbyists and startups can explore the most popular private and open models from Meta, Mistral, Azure OpenAI Service Microsoft and others with just a few clicks and keystrokes,” blogged GitHub CEO Thomas Dohmke.

Many Modes & Modals Of Models

This entire dovetailing operation has likely come about as a result of the sheer size and scope of AI model development today. Twinned with the need to straddle deployment attack surfaces spanning cloud, edge, general-purpose and perhaps more task- or industry-specific application and data service deployments, the complexity factor out there is dizzying. GitHub Models aims to open the door for developers to experiment with multiple models, simplifying model experimentation and selection across the Azure AI catalog so that engineers can compare models, parameters, and prompts.

By making Azure AI an open, modular platform, Microsoft’s Sharma says the company aims to help our customers rapidly go from idea to code to cloud. With Azure AI on GitHub, developers can use Codespaces to set up a prototype or use the Prompty extension to generate code with GitHub Models directly in Microsoft Visual Studio Code.

“Increased model selection gives developers the broadest range of options for the individual applications they are building. But each model naturally brings with it increased complexity. To counteract this, we’re making it incredibly easy for every developer to experiment with a range of models through the Azure AI model inference API. Using this single API, GitHub developers can now access a common set of capabilities to compare performance across a diverse set of foundational models in a uniform and consistent way, easily switching between models to compare performance without changing the underlying code,” said Sharma.

On the road ahead, Microsoft says it will expand its integration even further. Surprisingly specific about what this means, this is Redmond’s commitment to bringing Azure AI’s language, vision and multi-modal services to GitHub, along with additional Azure AI toolchain elements. Organizations with an existing Azure subscription can purchase GitHub products via self-service, directly through Microsoft sales or via third-party solution providers and can adjust the number of GitHub seats as needed.

August 5, 2024
IT Ops Trends: Cloud, Containers and COVID

Almost all companies are modernizing their IT infrastructure to remain competitive, with the focus firmly on migration to the cloud and the use of dynamic infrastructures. In a recent Trends in IT Ops 2021 survey, more than 160 IT professionals were asked how the implementation of IT strategies affected their day-to-day work. The results provided an impressive insight into the current work priorities of system administrators and IT Ops in general.

According to the survey, most administrators believe that monitoring cloud (79%) and container (73%) environments will take on a more important, or much more important role for them in the future. Additionally, 59% of those surveyed agree that their job role has evolved over the years due to cloud computing and containerization.

Not all IT Teams are Equal

However, the data also shows that the changes are not affecting every IT team equally. For example, 37% of all respondents say that cloud solutions are not relevant—or less relevant to them on a day-to-day basis. This response shows that on the one hand, system administrators see cloud computing as an essential topic. However, on the other, the impact in practice is not felt to the same extent by every administrator. These responses show that not every administrator has to deal with cloud platforms on a day-to-day basis, with some being ‘siloed’ into dedicated teams that may not interact with the broader corporate cloud strategy. In addition, not every organization has fully committed to the cloud, with some finding their migration hindered by applications that are not suited to cloud migration.

In the survey, 45% of respondents also said they believe the importance of monitoring physical infrastructure will not change in the next few years. However, 43% believe that monitoring physical systems will become more important or much more important. Conversely, a mere 1% replied that they thought physical infrastructure monitoring was less critical.

Taken together, these responses demonstrate that, despite the excitement around the cloud, many organizations take a hybrid approach to infrastructure. IT Ops and administrators recognize the importance of monitoring cloud systems and dynamic assets while, at the same time, the monitoring of local systems remains relevant or is even expected to gain in relevance.

Network Management and Automation

Cloud and containers enable faster growth of IT environments and promote digitalization. However, they also lead to a sharp increase in data traffic and more complex networking of systems. As a result of these changes, the demands on local networks are growing. This increasing demand demonstrates the importance of network management tools. 81% of respondents said that solutions in this area are relevant, or very relevant to them in their day-to-day work.

This year, the move to a home office joined the trend towards migration to the cloud, containers, and network monitoring. In the wake of the COVID-19 pandemic, many IT departments had to adapt their infrastructure to allow employees to work from home offices. However, most administrators were relatively stress-free, with 54% saying they had no trouble adapting infrastructure to meet remote work requirements.

Automation is also a key issue. For example, 81% of all respondents say projects that deliver deployment and configuration automation have been relevant in their organizations in the past 12 months. In addition, 92% say that automation tools are relevant, or very relevant to them on a day-to-day basis.

Cloud, Containers and COVID

The Trends in IT Ops 2021 survey shows that system administrators are concerned with cloud computing and containers and several other topics. In addition, most administrators recognize the increasing importance of monitoring cloud environments and containers. That said, administrators continue to see monitoring networks and other physical infrastructure assets as critical, and some system administrators see increasing importance in monitoring this vital area.

In addition to the topics outlined here, respondents also provided feedback on topics such as virtualization, application performance monitoring and network management.

January 12, 2022
CNCF Takes LitmusChaos Platform to the Incubation Level

The technical oversight committee (TOC) for the Cloud Native Computing Foundation (CNCF) announced today it is elevating the open source LitmusChaos application testing platform to the incubation level.

LitmusChaos is a chaos engineering platform donated to the CNCF by ChaosNative in 2020. Since then, it has seen adoption within production environments by more than 25 organizations, including Intuit, Lenskart, Orange, Red Hat and VMware.

ChaosNative CEO Uma Mukkara says LitmusChaos was developed to provide DevOps teams with a chaos engineering platform built using containers that makes it easier to scale up and down than existing monolithic alternatives. The platform itself, however, can be used to test both monolithic and microservices-based applications.

The platform consists of a Chaos Operator based on a software development kit (SDK); a ChaosHub for hosting and sharing experiments; Litmus Workflows that declaratively chains experiments (either in sequence or parallel) to build a chaos scenario; a centralized control plane, dubbed ChaosCenter, to design, schedule and monitor Litmus Workflows; Litmus Probes for creating chaos scenarios that automate steady-state validation and remediation actions and a Chaos Observability tool to export Prometheus metrics.

In total, there are now more than 400 contributors to the LitmusChaos project and more than 4,000 pull requests have been made. Mukkara said since the beginning of this year, Litmus operator installations have grown to more than 2,000 per day. Going forward, the goal for 2022 is to significantly increase the number of LitmusChaos integrations there are for a wider range of DevOps platforms, added Mukkara. Additional sets of experiments for Kubernetes and non-Kubernetes targets, improved observability and integration with other platforms—via the open source OpenTelemetry agent software also being advanced by the CNCF—is also planned.

In general, it’s still early days as far as the adoption of chaos engineering with DevOps workflows is concerned. However, Mukkara said that as more organizations embrace observability to better understand IT processes, the number of organizations that also use chaos engineering to test IT resiliency should also increase. In fact, as IT environments become more complex—thanks mainly to the rise of microservices-based applications—chaos engineering approaches to testing may someday be required, noted Mukkara.

In the meantime, there are still a lot of IT professionals that are uncomfortable with chaos engineering approaches to testing that deliberately break underlying services to test the resiliency of an application. The core idea, however, is that no application should have a single point of failure that could lead to it becoming unavailable. In theory, a microservices-based application won’t fail because when a service becomes unavailable calls are automatically rerouted to another service. The performance of that application will degrade, but it won’t go offline. The challenge is that building an application that achieves that level of resiliency using microservices requires a significant amount of DevOps expertise.

Regardless of how an application is constructed, the one thing that is clear is how dependent organizations are on applications to drive revenue. As such, the tolerance for application failures—or performance degradation—has never been lower.

January 11, 2022