devopsy.kinsta.cloud

Category: Enterprise DevOps

Microsoft Power Platform Integrates ChatGPT: What This Means

Microsoft recently announced it integrated ChatGPT into its low-code Power Platform developer suite. This follows the news of ChatGPT-powered Bing search and Microsoft’s pledge to invest billions into its partner company, OpenAI.

ChatGPT, the large language model (LLM), has reached viral status due to its ability to generate impressively complex textual outputs given simple prompts. And now, enterprises are hoping to cash in on its power. Microsoft’s inclusion of ChatGPT in its Power Virtual Agents and AI Builder tools will likely signal more native inclusion of AI models within other low-code development environments throughout the industry.

However, not all is peachy in the world of generative AI. Users have compiled a laundry list of ChatGPT failures, ranging from simple arithmetic errors and inaccurate information to social biases and even one attempt to dissolve a journalist’s marriage. AI seems sure of itself, but the “hallucinations” it produces can be mind-bogglingly off-base. If such inaccuracies persist within production use cases, it could lead to false suggestions that confuse users and even harm a company’s reputation.

Below, I’ll consider the implications of this new paradigm of AI-driven development, exploring the benefits and potential risks of incorporating LLMs into low-code development frameworks. We’ll also highlight how this may upset the competition and consider things leaders should keep in mind as they seek to leverage this new technology.

Benefits of Incorporating ChatGPT

Low-code development platforms (LCDPs) excel at abstracting complex functionalities into usable components. They typically offer drag-and-drop capabilities and reusable templates to enable citizen developers and professional programmers alike. And there are many potential benefits of incorporating ChatGPT into such a low-code environment. For example, as a feature within Power Virtual Agents, ChatGPT can be used to create chatbots that connect to public company resources and internal company data. This could help users seamlessly develop a contextually-aware AI trained on a company’s documents.

Utilizing AI within low-code can spur development with deep inferences and guidance. Such AI assistance can drastically accelerate development efforts through enhanced auto-completion and intelligent insights, quickly generating boilerplate code and making suggestions more customized to the task at hand than your typical static templates.

GPT-based chatbots leveraging AI trained on a company’s documents could be incredibly more valuable than an off-the-shelf model without context. In the realm of customer support, an in-house trained AI could be beneficial in summarizing company information and automating manual support activities. Providing a low-code means to generate such chatbots also brings advanced AI to a broader range of users. All in all, this could advance the pace of development, especially for mobile interfaces, automated workflows and contextually-aware chatbots.

Implications For the Low-Code Space

In the wake of ChatGPT, the market has been scrambling to respond. Tech giants have debuted their own generative AI to mixed reviews. And some, myself included, are left pondering how low-code and AI will coexist in this new era. Will natural language-driven code generation replace the need for programming entirely, whether it’s traditional or ‘codeless’?

This remains uncertain. But what appears most plausible is that these recent developments will raise the bar throughout the software industry and augment how LCDPs are built and used. AI has the power to enhance LCDPs in a few ways, including augmenting the developer experience, training bespoke ML models and building more intelligent end-user experiences.

Yet, with Microsoft funneling billions into AI research and development, it could be challenging for mid-size low-code platforms to keep pace without their own AI-driven workflows. Thus, the LCDPs that do not integrate robust AI of their own may lose out on new subscribers. Plus, engineers hoping to create AI models that run on internal datasets may prefer working with a larger cloud technology suite in which their data is natively stored.

Downsides of AI-Infused Programming

ChatGPT and other generative models are absolutely impressive, but the results can’t be 100% trusted. And at the time of writing, ChatGPT usage within PowerApps is still experimental, which is indicative of ChatGPT and generative AI in general—still in the experimentation phase.

This doesn’t bode all that well for programmers relying on its outputs, given the potential for inaccuracies. Because although the results of ChatGPT sound authoritative, they are generated from knowledge scoured from the public web, which often contains bugs, errors and inefficiencies.

The outputs of ChatGPT may even suggest entirely nonexistent features! This is, unfortunately, a current issue with geocoding API provider OpenCage. ChatGPT routinely recommends users integrate OpenCage’s “phone lookup service” for certain prompts. But … OpenCage doesn’t even offer this feature. The team has received so many complaints from angry developers that they even had to issue a statement explaining what happened.

With AI code generation, the focus shifts from hardcoding toward creating and organizing prompts and debugging errors. And although it’s giving programmers newfound agility, it doesn’t solve all software development problems—namely, the DevOps side of the equation. There will still be friction in deploying code and piecing together an optimal architecture. Also, due to rising software supply chain vulnerabilities, code automation that loops in third-party dependencies must require careful security forethought. There’s also the chance of breaking changes and ongoing maintenance hurdles with any software.

Another important element is that low-code requires governance to be secured correctly. No-code business users may not have the security oversight to understand the implications of spinning up new services. And when you add AI into the mix, the technical implications escalate. AI can result in ethical violations, and bots have been found to communicate anger and irrational ideas when poked and prodded.

Are We All Just Hallucinating?

‘Hallucinating’ is when an AI model confidently spouts nonsense or inaccurate information. And although this is the case with some models currently, they are actively accepting feedback and being retrained. Over time, their outputs will improve. But for now, engineers should be aware that they are still under experimentation. As such, enterprises should use caution and test new AI innovations on internal processes first. And AI adoption must be governed carefully to avoid misconfigurations and abuse.

AI is here, and the future is now. But although it’s dismantling some coding barriers, developers still have their work cut out for them. This reality is a win for low-code solutions offering standard software delivery pipelines and centralized collaboration features. If LCDPs can maintain pace with AI and embed it into their workflows, both sides should fare pretty well in this new era.

Surprise — this entire article was written by ChatGPT. Just kidding! None of it was. I’m just tired of seeing this “gotcha” moment at the end of news segments…

This post was written by a human.

March 10, 2023
Raise Those (Feature) Flags

I’ve written about feature flags before, but I think it’s time for a solid bit of advice: “If you are not yet using feature flags for DevOps, it is past time to reconsider.”

I say this because feature flags have been a slow-growth item, with a few spikes here and there in usage, but mostly on the development side. While many of you are using feature flags to do dynamic toggling of features–definitely an advanced use case, but still very development heavy–few organizations are using them for deployment options–blue/green without separate code, environment changes for different targets (deploy this to the cloud provider, but do this for local installation type stuff), etc.

Feature flags are starting to be supported by DevOps tooling for this precise use case, and it is glorious. While the ability to say, “run this code,” and if the new code fails, change something like an environment variable to say, “stop running that bit,” is huge. We have enough time with this process that the industry is well aware of the need to clean up as soon as the correct code options are known, and that is the biggest weakness; it makes spaghetti code that must be cleaned up. But the benefits are well worth a bit of cleanup at the end of a project, even if far too many of you don’t bother.

Now we can use them in deployment, and in full life cycle toolsets, we have them available for both–so a set of source code can be gated behind a feature flag and deployment can use the same feature flag. This enables the ability to say, “If our FF environment variable is ‘TestDeployment,’ connect to ‘Test Database’ … then use the same variable to say, “If our FF environment variable is ‘TestDeployment,’ spin up the container with test data instead of real data.” This simple example is something that could be (and has been) done manually, but doing it this way makes it part of the process. Automated testing can then follow without the need for human intervention, and from “commit” to “test results,” IT can focus on other things instead of spinning up the right containers and making sure the right DB is accessible.

After all, we are in a time of massive automation, which should not worry us. There are globally too many IT positions and too few people to fill them; automation should be exercised as much as possible to address that simple fact. And there is a good chance that you are already using tools in the DevOps toolchain that support feature flags–a growing number do support them. If not, it is easy to research and find a good tool to do so and while it doesn’t offer as much functionality, a stand-alone feature flag product will still offer more options than not having any.

And keep rocking it. The applications you wrote years ago are still powering the enterprise, even if you are at a different organization today. We leave a legacy of applications behind us that is a testament to quality. Keep leaving quality, and use all of the tools you reasonably can.

March 8, 2023
AWS Delivers on Latest Graviton3 Price/Performance Promise

Amazon Web Services (AWS) has expanded the number of virtual machines based on the latest generation of its Graviton processors to include instances that support DDR-5 memory.

The latest Amazon EC2 instance types, dubbed M7g and R7g, that are now generally available are based on the AWS Graviton3 processors developed using an architecture created by Arm. Previously, these instances were only available as part of a technology preview program.

Rahul Kulkarni, director of product management at AWS, said these latest offerings take advantage of DDR-5 memory to provide 25% better performance than the previous sixth generation of Graviton instances. DDR5 memory increases the amount of bandwidth available by 50%, he noted.

Graviton processors now provide the lowest-priced instance of virtual machines in addition to the best performance and overall price/performance across the AWS portfolio, Kulkarni added.

AWS is making a case for employing its AWS Graviton processors for general-purpose workloads as more IT teams are being tasked with reducing costs as the overall number of applications being deployed in the cloud continues to increase. In fact, all of the top Fortune 50 companies have already deployed workloads on Graviton processors, noted Kulkarni.

While it’s not clear how many applications are being developed to run natively on Arm processors, the process of refactoring existing applications to run on Arm processors is not as intensive for most applications as it was when previously moving applications from one class of processors to another.

In addition, Kulkarni noted AWS has committed to providing a full range of DevOps services, including a continuous integration/continuous delivery (CI/CD) platform, to help facilitate any transition to Graviton processors.

There are, of course, multiple strategies for containing cloud costs that range from committing to consuming a specified amount of compute resources annually at discounted rates to relying more on spot instances that are available for limited amounts of time. Regardless of approach, the days when developers were allowed to invoke cloud resources at will appear to have come to an end as enterprise IT organizations become more aware of the total cost of IT. As a result, many of them are adopting financial operations (FinOps) best practices to maximize the utilization of cloud infrastructure resources.

AWS, like most cloud service providers, gives organizations a wide range of processor options. However, it’s apparent that AWS views Arm processors in the form of Graviton to be the most efficient from both a price and performance perspective. It’s not clear how quickly organizations are embracing Graviton in the cloud given the dominance of x86-based platforms, but it’s apparent there is now a clear alternative for general-purpose workloads. If fact, the more data-intensive those workloads become, the greater the interest in more efficient processor architectures.

Of course, x86 platforms are not going away anytime soon. But as more Arm-based processors find their way into the enterprise, DevOps workflows will only become more challenging to manage.

March 6, 2023
Before ChatGPT, AIOps Powered Enterprises for Years

Led by OpenAI and its ChatGPT software, a new era of consumer use cases for artificial intelligence is here. These exciting new tools have made AI more accessible to more users than ever before. No longer is this technology the exclusive domain of wealthy corporations.

However, while the world is now focused on the potential use cases for AI in consumer applications, it’s important to remember that this technology has been part of the business world for years. Behind the scenes, AI has quietly been powering solutions that companies of all sizes have harnessed to improve business outcomes. These technologies have specifically been used by IT teams in the form of artificial intelligence for IT operations (AIOps) to help modernize their companies and operate more efficiently.

By leveraging AIOps-powered solutions in use cases like observability, overworked and understaffed technology teams are improving their ability to manage complicated, hybrid and multi-cloud IT environments, automate processes and optimize their businesses. As companies prepare for a potential economic downturn and a reduction in resources, these solutions and AIOps are more important than ever.

Here are four ways that AIOps is helping businesses overcome today’s IT challenges.

Providing Unified Visibility Into Complicated Networks

Companies have accelerated their digital transformation efforts in recent years. This has included migrating workflows to public and private clouds, implementing new productivity tools and adopting next-generation technologies. The average business now relies on geographically dispersed workforces and IT environments that use 2.6 clouds and leverage more than 200 apps!

The result is that organizations lack visibility into these environments. A recent SolarWinds survey found that 54% of technology professional respondents stated they only have visibility into about half or less of their apps and infrastructure. This lack of visibility can lead to problems that negatively impact the return on investment for projects.

AIOps helps overcome this challenge by providing comprehensive visibility into an organization’s systems, regardless of whether those environments are hosted in on-premises, hybrid or multi-cloud environments. AIOps automatically collects and integrates insights and information across geographies, apps and infrastructures into one dashboard. This provides technology teams with single-pane-of-glass visibility into the state of IT environments.

Automating Processes & Remediating Issues

While the visibility provided by AIOps and observability is integral to managing today’s complex IT environments, the technology goes a step further.

With AIOps, teams can automate processes including the identification and remediation of issues. Given the stakes, this is critical. Companies must ensure their environments are always available and that their performance meets and exceeds customer demands. Even short service outages can lead to frustrated users and a damaged customer experience.

This is challenging, as an IT anomaly often results in many alerts and a flood of log messages, making the problem difficult for a human to understand and address. AIOps quickly ingests massive amounts of information, provides rapid root cause analysis, data/event correlation and incident management which allows for fast and automated remediation of issues as they arise.

Additionally, since this technology is always observing and gathering information on potential incidents, it can predict and prevent future issues proactively. By implementing AIOps, companies are taking a step toward autonomous operations or environments that can manage themselves without human involvement.

Protecting Revenue

In the face of an uncertain economy, every organization is taking steps to protect its revenue. With AIOps, enterprise technology companies have a powerful tool in this fight.

By implementing AIOps and observability, companies can quickly resolve and prevent IT issues before they arise. This ensures that IT environments are operating at maximum efficiency.

With AIOps, organizations can reduce downtime and prevent outages. This ensures that business-critical applications and infrastructure are available and functioning. These capabilities are more important as even short service outages have reportedly been capable of causing millions of dollars in damages.

Freeing Up Employee Time for Innovation and Efficiency

Over the last few years, companies have experienced severe staffing challenges, with many organizations finding it difficult to fill critical technology roles. This has led to smaller teams with more responsibilities and less free time. The result has been overworked technology professionals who spend far too much time on manual and menial tasks that do little to support the business’s bottom line.

By implementing AIOps, organizations can streamline workflows by automating repetitive and routine tasks. Doing so can cut down on the amount of overwhelming metrics technology professionals are required to analyze and, instead, empower those individuals and teams to again focus on innovating and completing projects that support key business objectives.

March 6, 2023
LinkedIn Job Scams: Out of Hand ¦ 4-Day Workweek: Let’s Get Serious

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: Don’t get scammed looking for a job, and momentum grows for the 32-hour working week. (more…)

March 2, 2023
In DevOps, Chatbots are a People Problem

By now, you are undoubtedly aware that you can call up a chatbot or install an AI assistant into your IDE and generate entry-level code and deployment scripts.

Indeed, if you just go to a site like ChatGPT and enter “generate Python code to do X” where X is something you need to be done–or an example like “read a file and print out the contents one line at a time,” it will do it. And if you keep it simple, it will do it well. If you say “again, in Java,” it will comply and do it well.

Then say “new chat,” and tell it to “write a bash script to deploy an app named X written in Java and using Gradle to a cluster named Y,” it will do that also, informing you of the names of the deployment and service YAML files. Then you can say, “generate the deployment file mentioned,” and it will. And then you can say, “parameterize the Java version,” and it will do that, too. This iterative revision can continue until the deployment code, containers and deployment files are complete. If you have not yet, I urge you to spend the half hour going through these steps once required, so the potential is obvious.

It is not yet to the point where I would (or you should) trust it with complex systems, but all of that busy work we’ve been trying to overcome with templates and preconfigured containers? We can actually get around the bulk of it now. The same is true for testing of all kinds. There are tools that will auto-generate test scripts for a given set of source code. They will also generate test execution code. That means that the only step left—and the one that has held testing back because of the volume of work—is reviewing results … But wait, there’s more! In security testing, AI can help eliminate false positives. In performance or QA testing, it can filter out all but the most egregious issues (as defined by the org or by a larger pool of apps).

The problem we need to address today is the people issue. What we have traditionally used entry-level people for—and trained them in Dev/Ops/DevOps with—is no longer necessary. Because the AI can do it. That is stunning and really very cool, if you think about it.

But it heralds a danger that every org needs to be aware of. What we do and how we do it is changing. For the better, but change always brings uncertainty, and the foreseeable bit we need to confront is, “Now that we don’t need entry-level people for these steps, how do we recruit and train them?” I do not yet have a definitive answer for this question, because I am a product of my time, so I see things in terms of learning the bits that make up the whole—advising people interested in DevOps to learn at least one Linux command shell, to understand how networking works in containers, to write the scripts to get containers (and/or public cloud instances) up and running, etc.

I do think that every org should start using these tools at a pace that makes them comfortable because it allows us to continue our “more with less” growth without significant pain. But we need to make certain we are training the next generation of DevOps engineers in the process. For the foreseeable future, we’ll still be the ones bug fixing and troubleshooting, so we need to understand the environment and the applications. We are starting to get some sweet tools to help us at a higher level, though. Anyone who has inherited complex systems knows that the learning curve of tons of source files that make up many subsystems and all of the spaghetti code that these can be comprised of is painful. I don’t normally mention brand names here, so I’ll just say there is at least one tool out there that you can ask to “describe what this code does” and it will run with it and give you an answer in plain English. This might be the training tool of the future, with the ability to compare what you see in code with what the AI describes in English.

But for now, even if you don’t hunt down app description AIs, having entry-level people review the files that the AI generates will serve two purposes: First, to validate what the growing AI space is capable of and second, to get entry-level engineers into the entry-level bits of the systems.

And keep rocking it. Make sure you have a path for DevOps in the AI future, but remember that the creation of the calculator did not statistically change the number of mathematicians, just made the bulk of them more productive. That’s all we’re doing here, just a bit more flexibly … But in the end, like a calculator, the AI has an equation that it fills and reports back with.

March 1, 2023
Day in the Life of a Chief Developer Experience Officer (CDXO)
Developer experience (DX) has become essential to modern enterprise software development. As more companies become software companies, more are opening up their software both for internal reuse and externalized consumption. As part of this process, making developer-facing tools consumable and friction-free becomes more necessary to both perform well and stand out in the market.

To oversee developer experience quality within an organization, an entire new role is beginning to emerge — the chief developer experience officer (CDXO). A CDXO is in charge of ensuring the intuitiveness of developer-facing toolsets and being an advocate for engineering needs. Such a focus could improve internal platform usability, like within platform engineering initiatives. Or, it may involve improving the developer experience for public or partner APIs through quality documentation, SDKs, libraries, forums, tutorials and more. A CDXO might interface with many departments to monitor and improve DX across the organization.

Since the role is in its very early stages, few CDXOs exist. But I tracked down a few to gather insights on the emerging position—I had the unique opportunity to sync with Alessandro Cauduro, chief developer experience officer at Azion and Ivan Burazin, chief developer experience officer at Infobip. They both shared some fantastic insights on why DX is so important these days and what qualities make a good developer experience leader. Below, I’ll distill their wisdom and take a peek into the life of a CDXO.

Why DX is Important

First, it’s good to reiterate that developer experience is increasingly important for several reasons. Organizations want to build and ship applications faster and more securely. Yet, it can be difficult for programmers to keep up with new technology trends. “Technology is always changing, and developers are always having to try new things,” said Cauduro. “As a result, you rarely become an expert on a single platform.”

Developer experience helps you get up and running quickly with new tools. The mindset is similar to user experience (UX) but more applicable to different interaction points like APIs, CLIs, SDKs, VSCode plugins, libraries or other developer-facing technologies. Improving usability for these packages helps ease the daily grind, increases productivity and can be a competitive advantage, said Cauduro. He added that a benchmark of great developer experience is when you don’t even have to read the documentation to understand what you have to do.

“I think that developers are the lifeblood of innovation,” said Burazin. “They are the ones creating solutions and implementing transformative technologies. Yet, they are often let down when it comes to the tools they use to achieve this innovation, so providing a great developer experience is paramount if we want to foster that innovation as much as possible.”

CDXO Roles and Activities

So, what does a day in the life of a CDXO officer look like? What are some of their roles and responsibilities? According to Cauduro, the overarching goal is to put himself in the developer’s shoes—he constantly thinks about common developer workflows and considers how to create a seamless experience throughout the entire development life cycle.

Next is spreading awareness throughout the company about DX principles and how to increase DX within their offerings. A CDXO will likely answer directly to executive leadership but might interface with many departments. A CDXO may direct teams to construct developer-specific tools, like libraries, documentation, SDKs and self-service environments. “DX is a mindset,” said Cauduro. “The whole company needs to be engaged in it.”

“As with any C-level position, your job is on the one hand to make your team’s life easier in any way you can,” said Burazin. “And on the other to convey the developers’ issues or ideas to the company in hopes of nudging the company in the correct direction.”

A CDXO may also oversee developer relations efforts, which focus on fostering a developer community. There are many ways to interface with partners and clients to exchange ideas on how to build against the platform, Cauduro explained. This includes promoting open source projects, encouraging user meetings or interacting on Discord and forums to provide answers to community questions.

Qualities That Make a Good CDXO

The CDXO position isn’t for everyone. So, what personality traits make a good candidate for the role? Here are some of the necessary qualifications for the type of work:
- Experience as a programmer. A CDXO should have past experience as a professional developer. Engineering knowledge will help them understand the persona and empathize with their needs.
- Leadership qualities. The role also requires leadership expertise, as a CDXO must have the ability to coordinate and influence many different areas of the company.
- Creativity. Since the role is pretty nascent without a written path, it will require creativity. Having the ability to experiment, accept feedback and reiterate is also important, said Cauduro.
- Empathy. Communication skills and empathy also go a long way. “Deep engineering knowledge crossed with great people skills is the mix you would need for such a task,” said Burazin.
Developer Experience in Practice

At Azion, Cauduro is interested in how its tools can help developers move solutions from the cloud to the edge more seamlessly. To that end, they’ve found that Jamstack fits well with the needs of edge computing. Thus, some of his efforts focus on making the process of deploying a Jamstack app to the edge as seamless as possible for engineers.

While DX efforts have helped address development bottlenecks at Azion, the role of CDXO may not be necessary within all companies, admitted Cauduro. It’s more of a fit for API-first or SaaS companies that target developers as a primary consumer base. That being said, other companies can still benefit from a developer experience focus to enhance their internal IT platforms.

At InfoBip, Burazin explained that developer experience efforts center around helping external developers consume the InfoBip API, an API for all communication channels, such as SMS, voice, WebRTC and more. Sitting under the CTO, Burazin helps run three teams and DX has its own department. Impressively, he reports revenue on self-service APIs has increased fivefold in the past two years.

Burazin, also the co-founder of Codeanywhere, explained that his role is to “convey the worries or needs of the developer customers to management and help steer the ship in the right direction.” To discover those needs, he recommends ‘dogfooding’ what the customer is experiencing and streamlining the onboarding process to be as self-service as possible. He also recommends having measurable insights on metrics you can directly impact, like decreasing time from discovery to ‘hello world.’

Final Thoughts

Developer experience is becoming a unique selling point for technology companies—so much so that it’s entered the C-suite. “Whoever has the better DX will have the leading edge,” said Cauduro. For him, it’s about finding small things to optimize that remove pain points from the developer’s process.

To summarize, the role of CDXO is still evolving and not set in stone. And, granted, many different labels are being applied to describe similar objectives. But to get an idea of what the CDXO role entails, some common activities include:
- Considering the perspective of the developer.
- Engaging the entire company to prioritize DX.
- Overseeing the construction of DX-related assets and resources such as tutorials, documentation and guides.
- Collecting feedback and reiterating platforms.
- Overseeing developer relations and advocacy teams.
Looking to the future, developer experience will likely become more of a necessity as developer-driven purchasing habits become a real revenue driver. “This bottom-up go-to-market motion will help awareness and increase sales,” said Burazin. And, to helm this movement, more companies may adopt DX within a C-suite title. But of course, the role may look different from company to company as DX is applied in different ways.

“If we look at the largest technology companies, almost all of them have a developer experience department in their organization. And this is due to the fact that so much of a developer’s productivity is directly linked to their experience while working,” said Burazin. Things like internal developer portals and standard cloud deployments will help improve developer experience and help them focus on what they do best, which is coding, he said.
February 28, 2023
DevOps Use Cases for AI-Assisted Kubernetes

As indicated in my prior blogs Optimizing Cloud Costs for DevOps With AI-Assisted Kubernetes and Optimizing Cloud Costs for DevOps With AI-Assisted Orchestration, an AI-assisted Kubernetes orchestrator is needed to optimize cloud costs for DevOps, DevSecOps and SRE. This blog describes DevOps-specific use cases for an AI-assisted Kubernetes orchestrator. The blog also describes a roadmap for implementing an AI-assisted Kubernetes orchestrator and the benefits of a solution using an AI-assisted Kubernetes orchestrator for DevOps.

Use Cases for AI-Assisted Kubernetes

An AI-assisted Kubernetes orchestrator can offer many benefits to organizations running containerized applications on Kubernetes clusters. Here are some use cases for an AI-assisted Kubernetes orchestrator:

1. Auto-scaling: An AI-assisted Kubernetes orchestrator can help automate the process of scaling up or down the number of pods based on the traffic or usage patterns of the application. The AI can analyze the performance metrics of the application and determine the optimal number of replicas needed for the application to function efficiently.
2. Load Balancing: Load balancing is a critical component of any Kubernetes cluster. An AI-assisted Kubernetes orchestrator can optimize the load balancing by analyzing the network traffic and determining the best way to route traffic to the different pods in the cluster.
3. Predictive Maintenance: An AI-assisted Kubernetes orchestrator can help identify and diagnose issues before they become critical. The AI can analyze the logs and performance metrics of the applications to identify patterns and anomalies. Based on this analysis, the AI can predict potential issues and notify the operations team.
4. Optimization: An AI-assisted Kubernetes orchestrator can optimize the resource allocation of the Kubernetes cluster by analyzing the usage patterns of the application. The AI can identify the optimal amount of resources required for each pod and allocate them accordingly.
5. Self-Healing: An AI-assisted Kubernetes orchestrator can automatically detect and recover from failures within the Kubernetes cluster. The AI can analyze the logs and performance metrics of the pods and take corrective actions to ensure that the applications continue to function properly.

DevOps-Specific Use Cases for AI-Assisted Kubernetes

An AI-assisted Kubernetes orchestrator can provide several use cases specifically for DevOps teams. Here are some examples:

1. Automated deployment: DevOps teams can leverage an AI-assisted Kubernetes orchestrator to automate the deployment process of their applications. The AI can analyze the code and configuration changes, build container images and deploy them to the Kubernetes cluster without any human intervention.
2. Continuous integration and continuous deployment (CI/CD): An AI-assisted Kubernetes orchestrator can facilitate the CI/CD process by automatically building, testing and deploying new code changes. The AI can monitor the code repository for new commits and trigger the necessary actions to build and deploy the changes.
3. Security and compliance: DevOps teams can use an AI-assisted Kubernetes orchestrator to ensure that their applications comply with security and compliance standards. The AI can monitor the security posture of the cluster and identify vulnerabilities or non-compliant configurations. The AI can also implement remediation actions to address the issues.
4. Disaster recovery: An AI-assisted Kubernetes orchestrator can help DevOps teams to recover from disasters quickly. The AI can analyze the backup data and restore the data to the Kubernetes cluster. The AI can also create a disaster recovery plan and test it periodically to ensure that the recovery process is reliable and efficient.
5. Resource optimization: DevOps teams can use an AI-assisted Kubernetes orchestrator to optimize the resource usage of the Kubernetes cluster. The AI can analyze the performance metrics of the applications and identify resource-intensive workloads. The AI can then allocate resources more efficiently to ensure that the applications run smoothly and cost-effectively.

Roadmap

Implementing an AI-assisted Kubernetes orchestrator for DevOps involves a series of steps to ensure a successful deployment. Here’s a feasible implementation roadmap for an AI-assisted Kubernetes orchestrator for DevOps:

1. Define your use cases: The first step is to define the use cases for the AI-assisted Kubernetes orchestrator that align with your DevOps team’s goals. Identify which areas of your DevOps process can benefit from automation and optimization using AI.
2. Choose a platform: There are several AI-assisted Kubernetes orchestrator platforms available. Research the options and choose a platform that aligns with your use cases and technical requirements. For example, CAST AI provides an AI-Assist for Kubernetes solution that is specially targeted to reduce cloud costs by as much as 60%
3. Plan your deployment: Plan your deployment by defining the architecture, infrastructure and resource requirements. You need to ensure that you have the necessary resources, such as compute and storage, to support your deployment.
4. Prepare your data: The success of your AI-assisted Kubernetes orchestrator depends on the quality and accuracy of the data. Gather and prepare the data needed for your AI algorithms to train and make decisions.
5. Train your AI models: Train your AI models based on the data you’ve collected. Ensure that you test and validate the accuracy of the models to avoid introducing errors in your deployment.
6. Integrate with your DevOps toolchain: Integrate the AI-assisted Kubernetes orchestrator with your existing DevOps toolchain, such as CI/CD pipelines, monitoring and logging tools and configuration management tools.
7. Test your deployment: Test your deployment in a staging environment to ensure that it meets your requirements and use cases. Run a series of tests to verify that the AI-assisted Kubernetes orchestrator is working as expected.
8. Deploy to production: After testing, deploy the AI-assisted Kubernetes orchestrator to your production environment. Monitor the performance and health of the AI-assisted Kubernetes orchestrator and make any necessary adjustments.
9. Continuous improvement: Once deployed, continue monitoring and optimizing the performance of the AI-assisted Kubernetes orchestrator. Collect feedback from your DevOps team and improve the AI models, infrastructure and deployment processes.

Benefits

There are many possible benefits of an AI-assisted orchestrator for DevOps, including:

1. Increased efficiency: An AI-assisted orchestrator can automate many of the repetitive and time-consuming test environment and deployment orchestration tasks in the DevOps process, allowing teams to focus on more critical and complex tasks.
2. Improved accuracy: AI algorithms can analyze data and make decisions more accurately than humans. By using an AI-assisted orchestrator, DevOps teams can reduce the risk of human error in their provisioning and re-provisioning processes.
3. Faster innovation: With an AI-assisted orchestrator, DevOps teams can automate many of the deployment processes, resulting in faster and more reliable deployments. This can help organizations to deliver new features and updates to their customers more quickly.
4. Better resource utilization: An AI-assisted orchestrator can analyze resource usage and optimize resource allocation, resulting in more efficient resource utilization. This can help to reduce costs and improve performance.
5. Enhanced security and compliance: An AI-assisted orchestrator can monitor security and compliance issues in the DevOps infrastructure orchestration process, identify vulnerabilities and implement remediation actions. This can help to ensure that applications are secure and compliant with relevant regulations.
6. Improved scalability: An AI-assisted orchestrator can help to manage and scale Kubernetes clusters more effectively, ensuring that applications are available and performant as demand increases.
7. Predictive analytics: An AI-assisted orchestrator can use predictive analytics to identify potential infrastructure issues and recommend solutions before they become critical. This can help DevOps teams proactively address issues and avoid downtime or other service disruptions.

What this means

As indicated in my prior blogs Optimizing Cloud Costs for DevOps With AI-Assisted Kubernetes and Optimizing Cloud Costs for DevOps With AI-Assisted Orchestration, an AI-assisted Kubernetes orchestrator is needed to optimize cloud costs for DevOps, DevSecOps and SRE. This article explained DevOps use cases for an AI-assisted orchestrator like CAST AI. Such an AI-assisted Kubernetes orchestrator can help DevOps teams to streamline their operations, reduce costs and improve the quality and reliability of their infrastructures and applications for testing and deployment. By leveraging AI technology, DevOps teams can focus on innovation and delivering value to their customers.

February 24, 2023
Managing Risk

We have built some beautiful toolchains that crank out a finished product on the fly without needing anything close to the level of intervention that was historically required. The most advanced organizations on an automation journey could change a line of code and then wait for the new version to hit production without doing a thing. I say “could” because I don’t know of a single org that does this; there are, of course, checks in place and it’s not single-line changes but projects that release. But the infrastructure is there that would allow them to do this. It’s a high probability that one day we all will.

The problem is that most organizations only look at risk when building these toolchains and don’t bother with contingency plans. But over the long life of any given toolchain—toolchains that are made up of many moving parts—there will inevitably be the need for contingency plans.

And it’s certainly not only toolchains—remember when popular libraries were purposely corrupted by their author to make a point? And unless you’ve done your best to ignore it, you’re also aware of faked versions of popular libraries that insert code before continuing on to the original…

Most organizations do not consider a scenario in which critical libraries are suddenly unavailable or a scenario where a tool is crippled by movement in the market—a vendor making harsh licensing changes, the underlying tech making changes that impact the tool, etc. While it is comforting to think that open source protects us from many changes (and it forking does), that is not complete coverage that entirely protects our systems and toolchain. We need to know what is absolutely critical and how the organization will keep delivering in case that absolutely critical element goes away.

When I have done disaster planning—of which this is an adjunct—I have done so with a vague generality in mind. We have plenty of work to do, so all I ever wanted was a list of critical parts and an idea of how we would adapt without them. I worked at a place that kept a backup data center in case of total regional disaster—the backup data center was intentionally located on the other side of the country—but most of us don’t need that kind of DR. Certainly not if we can go, “This OSS library is used in 60% of our apps; if we have to, we could adapt this other one,” instead of building infrastructure.

For OSS dependencies, we have two great options that work together. Generate SBoMs, then correlate usage from those across the entire app portfolio. Then set up and maintain a local repository with point-in-time copies of those libraries that are deemed critical.

A similar, “Our CI tool is critical, but we could easily port processes and YAML to this other one,” is enough, too—it is absolutely not going to be easy if you ever have to do it, but “We looked at this six months ago and we know what to do,” is far better than “Oh crap, now what?”

You built it, you operate it, you are rocking it. Just take some time to protect it and you are covered.

February 22, 2023
Dev of core-js Will Flip Table ¦ Another 451 PyPI Maldeps

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: Denis Pushkarev is fed up with core-js freeloaders, and hundreds more malicious packages found at PyPI. (more…)

February 16, 2023