Would it be good or bad if your deployment pipeline had a mind of its own, where your infrastructure adapted to changes before you even noticed them?
This will happen in DevOps by introducing AI agents into the ecosystem. An AI agent is an AI that can take action. Think about a traditional LLM like OpenAI’s ChatGPT or Anthropic’s Claude. These AIs process input and generate output, but it is up to the user to act on the information. ChatGPT can create code and manifests capable of running a container, but it has no agency — it cannot act on what it knows.
AI agents are different. They have three key capabilities:
- Perception: Agents gather and interpret information from their environment. This could involve reading logs, monitoring metrics or analyzing reports. Perception allows the agent to understand the context in which it operates.
- Decision-Making: Based on perception and programmed objectives, agents evaluate situations and determine the best course of action. This might involve using ML/LLM models, rule-based systems or a combination of both to make informed choices.
- Action-Taking: Unlike traditional AI models, agents can execute actions in their environment. In a DevOps context, this could mean deploying code, adjusting configs or initiating automation. The ability to act autonomously is what sets agents apart from passive AI systems.
These three capabilities work together in a continuous loop, allowing AI agents to interact with and influence their environment in real-time. This makes them potent tools in the DevOps ecosystem.
How AI Agents Can Work in DevOps
How might this look in reality?
Like this:
Just like any deployment cycle, it starts with a developer pushing code changes to a repo. In current deployment pipelines, CI/CD automation kicks in at this point to test and deploy the code.
The agent sits at this point and monitors the repo for new commits. Upon detecting new code, the AI agent analyzes the changes and determines the optimal deployment strategy. Only then does the AI agent trigger the CI/CD pipeline. This pipeline is adjusted by the agent depending on the code, running different types of tests or adjusting the testing queue given preset priorities.
Then, the AI agent deploys the application and configures/provisions infrastructure as needed (e.g., scaling servers and allocating storage). Post-deployment, the AI agent continuously monitors performance and infrastructure health, collecting metrics, logs and events for constant analysis.
The agency comes from moving through the deployment pipeline and adjusting the pipeline on the fly. An AI agent can act to resolve issues as they arise. It will analyze the data in real-time and implement fixes from its knowledge base, such as restarting services, rolling back deployments, adjusting configurations or scaling resources. If it can’t fix the issue, only then the fault needs to be escalated to a human engineer.
Within all this, the agent is also learning. After any resolution, the AI agent updates its models based on new data and human feedback. It refines its algorithms for deployment strategies, anomaly detection and issue resolution, so the AI Agent becomes more proficient over time, reducing the need for human intervention.
We can quickly see the huge benefits of this within DevOps:
- Increased Efficiency and Speed: AI agents can operate 24/7 without fatigue, making decisions and acting much faster than human operators. This leads to quicker deployments, faster issue resolution and overall improved operational efficiency.
- Proactive Problem Solving: With their ability to continuously monitor and analyze system data, AI agents can often predict and prevent issues before they impact users. This shift from reactive to proactive management can significantly improve system uptime and user experience.
- Scalability and Consistency: AI agents can manage increasingly complex systems and larger infrastructures without a proportional increase in human resources. They also ensure consistent application of best practices across all operations, regardless of scale.
- Reduced Operational Costs: AI agents can significantly reduce operational costs by automating routine tasks, optimizing resource allocation and minimizing downtime. They can also make more efficient use of cloud resources, reduce the need for large on-call teams and minimize the financial impact of outages or performance issues.
These benefits demonstrate how AI agents can transform DevOps practices, leading to more robust, efficient and cost-effective operations while freeing human engineers to focus on higher-level strategic tasks and innovation.
What Agents Mean for Engineers
So, what of those human engineers? Is this the end of the DevOps role, with machines running the machines?
Yes, and no. Yes, in that the DevOps role will change. But no, in that the DevOps role won’t disappear. Instead, DevOps engineers will be freed to contribute more strategically within an organization. We see three possibilities for human input in the system.
First, the escalatory role. In this role, DevOps engineers become the ultimate problem solvers, handling issues that AI agents can’t resolve on their own. This role requires a combination of broad technical knowledge and creative problem-solving skills, as engineers will be tackling the most challenging and unique issues in the system.
Next, is the management role. DevOps engineers in this role will oversee and manage the AI agents, much like they might manage a team of human engineers today. They will be responsible for setting goals, defining policies and ensuring that the AI agents operate efficiently and ethically. This role involves a shift from hands-on technical work to more strategic thinking, requiring skills in project management, system architecture and even AI ethics.
Finally, the agenticops role. This emerging role focuses on the intersection of AI and operations. AgenticOps engineers will specialize in designing, implementing and optimizing AI agent systems for DevOps. They will need to understand both the intricacies of DevOps practices and the capabilities and limitations of AI technologies, acting as a bridge between traditional IT operations and cutting-edge AI implementations.
The machines aren’t here to replace us; they are here to elevate us.
To learn more about Kubernetes and the cloud native ecosystem, join us at KubeCon + CloudNativeCon North America, in Salt Lake City, Utah, on November 12-15, 2024.