devopsy.kinsta.cloud

Tag: devops tools

xAI Enters the Coding Agent Race With Grok Build

The AI coding agent market just got a new competitor. Elon Musk’s xAI is rolling out its first coding agent, called Grok Build, in a bid to challenge Anthropic’s Claude and other established tools in the developer workflow space. For DevOps teams already sorting through a crowded field of AI-assisted coding tools, the arrival of Grok Build adds another option — and a few fresh ideas worth paying attention to.

What Grok Build Is

The agent, currently in early testing and available only to paying subscribers, can complete complex coding tasks based on user commands. But the more interesting part isn’t what it does — it’s how it does it.

Grok Build runs up to eight parallel AI agents simultaneously, each working through a three-stage workflow: plan, search, and build. What sets it apart from other tools is Arena Mode, an automated evaluation layer that scores and ranks competing outputs before a developer ever reviews them. Instead of manually comparing multiple code solutions, developers see a ranked list of options. That’s a practical time-saver on complex tasks.

The tool is also local-first, meaning no source code is transmitted to xAI’s servers. For teams working with proprietary codebases or in regulated industries, that’s a meaningful design choice. Installation follows a standard npm workflow, and the CLI includes an optional web UI for visual monitoring.

The underlying model, grok-code-fast-1, was built from scratch — separate from the Grok 4 lineage — with a training corpus heavy on programming content and post-training focused on real-world pull requests and coding tasks. It scores 70.8% on SWE-Bench Verified and is priced at $0.20 per million input tokens. That’s a notably competitive price point compared to what developers are paying for Claude Code or Codex CLI today.

A Crowded and Competitive Market

The AI coding agent landscape in 2026 has become a three-way race between Anthropic’s Claude Code, OpenAI’s Codex CLI, and now xAI’s Grok Build. Both Claude Code and Codex CLI have significant head starts. Codex CLI surpassed one million developers in its first month, and Claude Code alone has reportedly driven Anthropic to $14 billion in annual recurring revenue, with coding agents identified as the company’s primary growth vector.

Grok Build has some catching up to do on the ecosystem side. Claude Code and Codex CLI have tighter IDE integrations, more third-party extensions, and longer production histories. And there’s the context window gap to consider: grok-code-fast-1’s 256K context window trails Claude Opus and GPT-5.4, both of which offer 1 million tokens or more — a meaningful difference for developers loading large codebases in a single pass.

According to Mitch Ashley, VP and practice lead, software lifecycle engineering, The Futurum Group, “Coding agents are becoming the procurement front where AI labs compete to own the developer workflow. Multi-agent parallelism with built-in evaluation, paired with local-first execution, reflects vendors racing to differentiate on orchestration architecture and execution environment guarantees.”

Ashley continues, “DevOps teams now weigh orchestration patterns, evaluation pipelines, and execution location alongside model performance and ecosystem maturity. Per-token economics and execution sovereignty become procurement criteria that compound at scale, and benchmark-leading models without credible answers on both will lose enterprise footholds.”

Why xAI Needs This to Work

Grok’s growth has slowed in both consumer and enterprise markets. Research from Enterprise Technology Research found that corporate use of Anthropic’s Claude and Google Gemini is climbing sharply this year, while Grok has struggled to keep pace.

While xAI’s Grok models briefly surpassed rivals on certain benchmarks last year, competitors’ newer updates have since reclaimed the lead. The company is particularly focused on improving its coding capabilities, where Grok currently trails some competing tools.

The backdrop makes the Grok Build launch feel less like a product announcement and more like a strategic statement. xAI needs a credible foothold in enterprise developer workflows, and coding agents are one of the clearest paths to get there. With 90% of developers now using at least one AI tool at work, according to a JetBrains survey from January 2026, xAI’s timing puts it squarely in the middle of a fierce land grab.

What DevOps Teams Should Watch

For now, Grok Build is still on a waitlist. Musk set a “next week” launch timeline in mid-April, but by late April the tool had not shipped. Bloomberg’s report confirms that early testing is now underway, suggesting broader availability may be imminent.

For DevOps teams evaluating options, the practical calculus is fairly straightforward. If you need a production-ready coding agent today, Claude Code and Codex CLI are the proven choices. But if Grok Build delivers on its multi-agent architecture — and its local-first privacy design — it could carve out a real niche, especially for teams doing high-volume agentic coding where per-token cost matters.

Arena Mode is the feature to watch most closely. The idea of having AI agents compete and self-rank before human review isn’t just a clever marketing hook. If it works consistently in practice, it could meaningfully reduce code review overhead. That’s the kind of workflow improvement DevOps teams are actually looking for.

The coding agent market is still being defined. xAI is late to it, but Grok Build brings a few genuinely interesting ideas to the table. Whether that’s enough to close the gap with Anthropic and OpenAI remains to be seen.

May 15, 2026
GitHub to Leverage User Code for AI Model Training, Allows Opt-Out

GitHub is preparing a significant change to how it trains the AI models behind its Copilot coding assistant. Beginning April 24, the Microsoft-owned platform will collect user interaction data by default to improve its AI systems, unless users actively disable the setting.

The update applies to individuals using Copilot Free, Pro, and Pro+ tiers. Enterprise and business customers are excluded, based on contractual protections often negotiated by larger organizations. For millions of individual developers, however, the shift introduces a new baseline: participation in AI training is automatic unless explicitly declined.

“If you choose to help us improve our models with your interaction data, thank you,” wrote Mario Rodriguez, CPO of the GitHub product team. “If you prefer not to participate, that’s fine too—you will still be able to take full advantage of the AI features you know and love.”

As AI becomes embedded in development platforms, policies like GitHub’s will likely become more common. The question facing developers is not only how these tools perform, but also how much of their own work they are willing to contribute to a platform’s growth.

Collecting Most Everything

By analyzing how developers interact with Copilot, what suggestions they accept, modify, or reject, the system can refine its understanding of real-world programming workflows. The company argues that this leads to better bug detection and more accurate code suggestions.

In terms of what’s collected, the answer is pretty much everything: user prompts, generated outputs, code snippets, surrounding context within files, comments, and even repository structures. Feedback mechanisms, such as rating responses, may also be incorporated.

This approach is part of a larger trend across the AI industry, where companies increasingly rely on live user data to improve model performance. Early versions of tools like Copilot were trained largely on public code and curated datasets. Now, developers themselves are becoming a continuous source of training input. GitHub maintains that the policy aligns with prevailing industry standards.

Some Pushback

GitHub’s decision has drawn scrutiny, especially among developers concerned about privacy and control over proprietary code.

One potential conflict involves private repositories. GitHub states that stored code in private repos is not used for training unless it is actively processed through Copilot. However, once a developer engages Copilot within that environment, portions of that interaction may be captured and used to refine models. Critics argue this blurs the traditional understanding of what private means on the platform.

The company emphasizes that users retain control. A setting within account preferences allows developers to disable data sharing for AI training. Existing privacy choices will also carry forward, meaning users who previously opted out of data collection remain excluded under the new system.

Informed Consent

The burden now shifts to users to take action. Those who do nothing will be enrolled by default, a design choice that has prompted debate about informed consent. Some developers have expressed dissatisfaction in community forums, noting that opt-out systems can obscure the implications of participation.

GitHub’s leadership frames the change as necessary for progress. Internal testing using employee interaction data reportedly improved suggestion acceptance rates, indicating that real-world usage can materially enhance model performance. Expanding that dataset to the broader user base is expected to further speed up those gains.

The move highlights a core tension in today’s software development: the trade-off between convenience and control. AI-powered tools streamline coding tasks and boost productivity, even as they increasingly depend on access to the very work they aim to assist.

March 26, 2026
How Open Source Tools Simplify DevOps for Startups

In recent years, software development has made big strides. Tools like GitHub Copilot, ChatGPT, and other AI helpers have given developers the power to churn out code, tests, and documentation faster than ever. However, for solo founders and small startup teams, getting things written is only half the battle. Real challenges lie in getting it deployed, keeping it running, and scaling up all without it falling over. This is where the right open source tools can meaningfully simplify day-to-day operations.

A 2025 study on the adoption of DevSecOps in small and medium-sized enterprises (SMEs) found that while over two-thirds had begun or were using DevSecOps practices, 35% still cited resource constraints and 41% cited technical complexity as key barriers. It’s not just about missing out on the right tools. The real issues stem from how the environment gets in the way. Configuration drift means deployments become inconsistent, and the whole CI/CD pipeline gets fragmented. Next thing you know, you’ve got failures, security gaps, and just a whole load of unnecessary effort going to waste. What these startups really need are open source tools that can make the day-to-day admin easier, reduce cognitive overload, and make DevOps less of a nightmare.

This article digs into the DevOps bottleneck for companies that are resource-constrained, looks at the frameworks that are trying to ease the pain, and shows how adopting a more streamlined approach to operations is the key to getting creativity flowing without burning out.

The DevOps Bottleneck for Resource-Constrained Startups

For most startups, building software isn’t the hard part anymore – running it efficiently is. The DevOps ecosystem is mature but fragmented. Tools like Terraform, Ansible, and Pulumi help automate infrastructure, but assume a working knowledge of scripting and cloud configuration. Jenkins, GitLab CI, and CircleCI offer reliable CI/CD pipelines but come with the cost of continuous maintenance and complex setup. Even the more simplified platforms like Heroku, Render, and Vercel trade long-term flexibility for convenience and limit how much you can customize your environment.

For resource-constrained teams, this fragmentation becomes a bottleneck. A solo developer or two-person team ends up spending more time troubleshooting YAML files and deployment scripts than improving the product itself. Add configuration drift to the mix, where environments slowly get out of sync over time, and the operational debt compounds fast.

This growing pain has led to the rise of lightweight, open source frameworks that prioritize simplicity over scale. SoloOps consolidates deployment, monitoring, and pipeline automation into a single command line workflow so founders can manage infrastructure without juggling multiple tools or complex configurations. Coolify offers a self-hosted alternative to Heroku with built-in monitoring and deployment capabilities, and CapRover provides Docker-based application deployment with minimal setup overhead.

These tools share a common philosophy, reduce context switching, and operational complexity for small teams. They’re not trying to replace enterprise-grade solutions like Kubernetes or compete with full-featured PaaS platforms. They offer a pragmatic middle ground for early-stage startups that need production-ready infrastructure without dedicating engineering resources to DevOps toolchain maintenance.

Open-Source Frameworks Simplifying DevOps

The real proof-of-concept for any DevOps framework isn’t some fancy list of features. It’s about how easily a small team can go from code to production without burning out in the process. Let’s take a look at what happens when you’re trying to deploy a web app with a database backend.

The Traditional Multi-Tool Approach

Conventional DevOps tooling can be a real nightmare for solo founders or small teams. You’ve got to orchestrate all sorts of systems. Terraform to provision cloud infrastructure, Jenkins or GitLab CI to set up a CI/CD pipeline, and on top of that, a separate tool for monitoring, like Prometheus or Datadog, just to name a few. Then, there is log aggregation, alerting rules – it’s a whole zoo of different tools to keep track of. And just when you think you’ve got it all sorted out, things start to drift out of sync, and you’ve got to go back and fight through the whole mess all over again.

This approach works fine when you’ve got a big team and a dedicated DevOps engineer, but for a two-person startup, it means you’re constantly switching between five or six different tools, each with its own quirks, and it just becomes complicated. A simple database migration might require updates across Terraform state files, CI/CD pipelines, and monitoring configs.

The Streamlined Alternative

Modern lightweight frameworks try to make all that complexity disappear. Tools like SoloOps, Coolify, and CapRover all take a similar approach. They consolidate deployment, monitoring, and pipeline automation into a single, easy-to-use interface. No more juggling multiple toolchains or wrestling with different syntax and dashboards. Instead, you can define your infrastructure and deployment logic in one place and get built-in monitoring without having to set up a whole separate observability platform.

Coolify, for example, is a self-hosted platform that comes with web-based management and automatic SSL provisioning. CapRover lets you do Docker-based deployments with one-click app installations and built-in load balancing. And SoloOps is all about command-line workflows for teams that are comfortable with terminal operations, consolidating the deployment pipeline into a bunch of straightforward commands.

“Every extra tool represents a cognitive overhead and maintenance burden for resource-constrained teams” – O. Desmond, Lead Solutions Architect with experience building DevOps workflows for early-stage companies. The goal isn’t to have more features, it’s to cut out the friction between idea and production.

These frameworks take that thinking a step further. Instead of trying to detect when the configuration has gotten out of sync, they keep everything up to date through their unified interfaces. Then, when changes are made manually, they make it clear what’s happening, which reduces all the operational debt that just accumulates when you’re working with a bunch of separate tools.

The practical impact becomes clear when you consider deployment velocity. What might take hours of juggling multiple tools suddenly reduces to minutes with these consolidated workflows. For early-stage startups where speed and focus matter more than enterprise-grade scalability, this difference is transformative.

Conclusion

The DevOps landscape has evolved to the point where resource-constrained startups no longer have to choose between operational excellence and product development focus. Lightweight, open source tools like SoloOps, Coolify, and CapRover show that simplification doesn’t mean compromise on reliability. By consolidating fragmented toolchains into unified workflows, these tools let small teams ship faster without accumulating technical debt.

For solo founders and small teams drowning in YAML configurations and context switching, the path forward is clear. Start with tools designed for your size, prove your product market fit, and graduate to more complex infrastructure when your business requires the operational overhead. DevOps should enable creativity, not constrain it.

November 26, 2025
Nominations Are Open: DevOps Dozen 2025
Once again, it’s that time of year when our community steps into the spotlight. The DevOps Dozen is back — and nominations are officially open. If you’ve shipped something meaningful, mentored someone, elevated a team, or built the next great platform, this is your moment to share it with the world.

Over the years, these awards have grown into a signal we can trust — honoring the people and products shaping how modern software gets built and run. This year’s program reflects where the market is headed, with deeper attention on AI, platform engineering, and the practical realities of secure software supply chains. There are 24 total awards, split between Community and Tools & Services — a balance that celebrates both the humans who lead and the technologies that make them faster, safer and better.

What’s New, What Matters

A few things to note up front. Community nominations remain open and accessible. For Tools & Services, there’s a $300 fee — not to ring a cash register, but to keep spam out and ensure we’re evaluating serious entries. If a commercial submission doesn’t make the finalist cut, the fee is refunded. We also cap finalists at 10 per commercial category to keep the field competitive and focused. And as always, final selection blends public voting (40%) with judges’ evaluation (60%) — a formula that balances community energy with experienced scrutiny.

Note: Tools/Services vendors aren’t eligible for three of the community awards — check each description for the fine print.

The Categories (All 24)

Community Awards (9):
- Top DevOps Community Evangelist
- Most Innovative DevOps Open Source Project
- DevOps Enterprise Leader of the Year
- Best devopsy.kinsta.cloud Article of the Year
- Best DevOps-Related Video or Audio Podcast
- Best DevOps Survey/ Research/Report
- Best DevOps Event of the Year
- Best DevOps Book of the Year
- Best Application of AI in Enterprise DevOps Practices
Tools & Services Awards (15):
- Best Application of Agentic AI in a DevOps Tool/Platform
- Best New DevOps Service Provider
- Best New DevOps Tool
- Best DevOps for DataOps/Database Solution
- Best DevOps for Mainframe Solution
- Best Testing Service/Tool
- Best Platform Engineering Solution
- Best DevSecOps Solution
- Best Observability Solution
- Best Supply Chain Security Solution
- Top DevOps Vendor Evangelist
- Best Application of Generative AI in a DevOps Tool/Platform
- Best End-to-End DevOps Platform
- DevOps Industry Leader of the Year
- Best DevOps Industry Implementation
If you’re scanning for what’s “new,” you’re not imagining it: Agentic AI and Generative AI each get their own category on the tooling side, and Platform Engineering has a dedicated slot as this practice matures from trend to table stakes. Likewise, Supply Chain Security Solution earns standalone recognition — reflecting how software integrity, attestations and provenance moved from “nice to have” to “non-negotiable.”

How it Works (and Why it’s Fair)

The process is transparent and deliberate:
- Nominations: Open to the public. For Tools & Services, the $300 nomination ensures signal over noise. Community nominations remain free.
- Finalists: We limit commercial finalists to 10 per category; if you don’t make it, your fee is refunded. Once finalists are set, public voting opens across all 24 categories.
- Winners: Determined by a 40/60 blend—community vote (40%) and judges’ scores (60%). Finalists receive badges and recognition across the Techstrong network; winners receive trophies and the full spotlight they’ve earned.
The Judges

Every year, we assemble a panel that’s both broad in experience and sharp in judgment. This year’s Honorable Judges:
- Alan Shimel — CEO, Founder and Editor-in-Chief, Techstrong Group, Inc.
- Mitchell Ashley — VP & Practice Lead, Software Lifecycle Engineering, Futurum
- Tracy Ragan — OpenSSF and CDF Board Member
- Garima Bajpai — Senior DevOps Leader and Advisor
- Ketan Jani — Staff Site Reliability Engineer, Apple Inc.
It’s a strong mix of industry leadership, community credibility, and day-to-day operating reality — the combination you want when the mandate is to separate sizzle from substance.

Dates You’ll Care About
- Nominations: Open now and run through mid-October (get yours in early; the best entries tell a crisp story with outcomes).
- Finalists: Announced soon after; voting begins immediately thereafter.
- Winners: Revealed at Predict 2026, our annual kickoff to the tech year — where trends and truth meet.
Why These Awards Still Matter

DevOps has always been bigger than tools — and yet tools matter. Culture and process drive outcomes — and yet champions matter. The Dozen sit at that intersection. When a community evangelist rallies practitioners around a better way to ship, that changes outcomes. When a platform reduces toil and cognitive load, that changes outcomes. When we hold ourselves to higher standards in software integrity, that — again — changes outcomes.

That’s why we do this. Not for trophies, but for momentum. Not for vanity, but for validation. Not just to say “good job,” but to say “this is the work that moved us forward.”

If you’re reading this, you likely know who deserves a nomination: the unsung maintainer who shipped the fix at 2 a.m., the team that turned a monolith into a product platform, the product that finally made security guardrails feel like guardrails—not handcuffs. Tell us about them.

Nominations are open. Show the community what great looks like. Good luck to everyone who participates — and may the best DevOps win.

Celebrating Noteworthy DevOps Dozen Winners (2020–2024)

You can learn a great deal about the industry by examining past winners. Here are some of the past winners over the last 4-5 years. It is a retrospective on standout winners who’ve shaped DevOps culture, tools, and leadership:
- Best End-to-End DevOps Tool/Service – GitLab (2020)
  GitLab’s all-in-one platform accelerated DevOps workflows with collaboration baked into CI/CD.
- Best CI/CD Tool – CircleCI (2020)
  The go-to modern CI/CD platform empowering teams with reliability and speed.
- Best Value Stream Management Tool – Digital.ai Value Stream Platform (2020)
  Bridged Agile planning, security, test automation, and AI analytics into a seamless delivery flow.
- DevOps Executive of the Year – Shlomi Ben-Haim (JFrog) (2021)
  Recognized for guiding DevOps tooling evolution during a pivotal era.
- Best DevOps Repo/GitOps Tool – Red Hat OpenShift GitOps (2021)
  Helped teams scale GitOps workflows securely across clusters.
- Best Observability Solution – Honeycomb (2021)
  Empowered teams to dig deep into performance and reliability signals.
- Best End-to-End DevOps Tool/Service – JFrog Software Supply Chain Platform (2023)
  Offered comprehensive supply-chain workflow integration, from dev to production.
- Best CI/CD Tool – CloudBees CI/CDRO (2023)
  Marked its ninth consecutive award, demonstrating leadership in enterprise CI/CD.
- Best Platform Engineering Solution – Harness Internal Developer Portal (2024)
  Recognized as the leading solution enabling self-service capabilities for engineering teams.
- Best Application of AI in DevOps Tool/Platform – GitLab Duo (2024)
  Elevated DevOps by integrating AI-driven workflows seamlessly.
Each of these winners has done more than just collect trophies — they’ve changed the fabric of the DevOps ecosystem, fostering efficiency, security and innovation.

Call to Action — Nominate Now!

Be a champion — recognize the standout DevOps talent and innovations of tomorrow. Here’s your guide to getting started:

1. Submit a Nomination – Nominations are open through mid-October. Go ahead and shine the spotlight on those who deserve it.

2. Understand the Rules –
- Community categories: Free to enter.
- Tools & Services: A $300 fee (refunded if not a finalist) to maintain high signal and quality submissions.
3. Stay Tuned for Finalists – After mid-October, we’ll unveil the finalists and open public voting. Get the word out!

4. Join the Celebration at Predict – Winners will be announced at the Predict virtual event in early January. Finalists and winners receive badges, trophies, and recognition across the Techstrong network.

Nominations are live — make sure your DevOps stories get the stage they deserve. Nominate here.
September 8, 2025
ZenCoder Update to AI Coding Platform Tightens DevOps Integration

ZenCoder has updated its artificial intelligence (AI) platform for writing and testing code to provide integration with third-party DevOps tools such as JIRA, GitHub, GitLab and Sentry in addition to tighter coupling with VS Code and JetBrains integrated development environments.

At the same time, the company has added a “Coffee Mode” capability that now allows Zebcoder AI agents to perform tasks in the background.

Finally, Zencoder reports that with this update it has, based on the SWE-Bench-Multimodal benchmark, improved overall performance by a factor of two.

ZenCoder CEO Andrew Filev said the overall goal is to accelerate application development by integrating the company’s namesake AI platform more deeply into DevOps workflows.

The Zencoder platform, in addition to generating code, is also able to repair code, create tests and optimize code in real time using built-in static analysis capabilities. It makes use of multiple proprietary and open-source large language models (LLMs) to train agents to handle each of those tasks.

Rival approaches are, as a result, more prone to hallucinate in ways that generate flawed code, noted Filev. Zencoder solves that issue by providing development teams with a set of AI agents that can apply static analysis to code as it is being developed. For example, a Repo Grokking agent analyzes the entire code repository to provide the appropriate level of context required to generate code before an Agentic Repair agent creates a pipeline that automatically analyzes, fixes and refines generated code.

Most other AI tools don’t provide that level of context, resulting in code that actually won’t run in its intended production environment, noted Filev. Application developers are also unable to debug that code simply, because they don’t have any insights into how it was created by the AI tool, he added.

It’s not clear how many organizations are relying on AI to help build applications faster. A recent Futurum Research survey finds 41% of respondents expect generative AI tools and platforms will be used to generate, review and test code, while 39% plan to make use of AI models based on machine learning algorithms.

There is at this point no shortage of AI tools and platforms for writing and testing code. The challenge now is deciding which one best suits the needs of the organization versus any given individual developer. Unfortunately, many organizations still lack a coherent strategy so, as a result, many individual developers are experimenting with different tools and approaches without any overall organizational cohesion.

It’s not so much a question of whether or not AI will be relied on to build and deploy applications so much as it is the degree. Each application development team should, at the very least, now be experimenting with these tools. Actual usage of them to write code that will actually run in a production environment, however, will still require humans to know how that code has been constructed.

There is, of course, always pressure to build and deploy software faster. The issue is that it’s one thing to make a mistake when prototyping an application. It’s quite another to discover that mistake after an application has already been deployed in a production environment.

April 11, 2025
Cribl Extends Reach of Platform for Routing DevOps Telemetry Data

Cribl this week added support for multiple additional platforms to its cloud service for collecting and routing telemetry data collected from multiple DevOps tools and platforms.

The latest addition of Cribl Stream adds support for Microsoft Azure and improves its ability to search data stored in the data lake from Snowflake.

At the same time, Cribl can now also collect data via the Distribution application programming interface (API) made available by Datadog and from the ServiceNow Observability platform the company gained when it acquired LightStep.

Finally, Cribl Edge has been updated to provide an ability to monitor the health status of nodes and fleets of data sources, while Cribl Lake has added a Hybrid Worker Group capability to better enable teams to write, replay, and mix and match data sources.

Vlad Melnik, vice president of business development and alliances for Cribl, said the latest update to Cribl Streams makes it simpler to configure the platform to route data to the Microsoft Azure cloud. Previously, the platform has only provided integrations with the Amazon Web Services (AWS) cloud.

Of course, some DevOps teams have already integrated Cribl Streams with the Microsoft Azure cloud, but now that capability is built into the platform in a way that is managed by Cribl, said Melnik. However, as Azure Event Hubs is already widely used to collect log data collected from across the Azure service, Cribl is now making it easier to collect and normalize telemetry data collected from anywhere, noted Melnik.

DevOps teams are collecting more telemetry data than ever, in part thanks to the rise of Open Telemetry, open-source agent software that reduces the total cost of collecting telemetry data generated by multiple platforms and applications. The challenge is finding a way to streamline the collection of telemetry data in a way that makes it simpler to normalize regardless of how it was generated. That’s especially critical when, for example, DevSecOps teams are trying to identify the root cause of a potential cybersecurity incident, noted Melnik.

Aggregating all that data is also taking on added urgency in the age of generative artificial intelligence (AI). As DevOps teams look to operationalize these platforms, they need to be able to expose AI models to telemetry data collected from across highly distributed computing environments.

It’s not clear how DevOps teams are being expanded to add data management expertise. In some cases, they may add data engineering expertise, but in more instances, existing DevOps engineers are likely extending their skills into the realm of data management.

One way or another, DevOps teams will need to find ways to streamline the management of telemetry data that continues to grow in volume. Each DevOps team will ultimately need to determine how long it needs to store all that data but, as always, the more regulated an industry is the more onerous the telemetry data storage requirements become.

Ultimately, any effort to truly optimize application performance starts with telemetry data. The issue is not so much finding that data but rather routing it all to a place where there are tools capable of making sense of it all.

September 13, 2024
Transforming DevOps With AI: Practical Strategies To Supercharge Your Workflows

As someone interested in DevOps, I wondered how all the AI advances could benefit my field. OpenAI used deep learning to release groundbreaking products like ChatGPT and Sora. Microsoft used similar technologies to revamp its products, notably enhancing GitHub with Copilot. Many startups have sprung up, and large tech companies have poured billions into AI research.

Ultimately, how can engineers use this technology to reduce toil, add value to the software development lifecycle (SDLC), and increase development velocity? There are a few interesting options.

Random Forest Classifier for Self-Healing Systems

Say you have a distributed systems environment with lots of pods serving different services. You also have some observability tooling, like Prometheus, to provide a stream of system metrics, such as CPU usage, memory usage, disk I/O, network statistics and even container logs.

Random Forests can help you build a self-healing system. With some creative feature engineering to get rolling averages, rates of change, and error log counts, the Random Forest Classifier can generate corrective actions. For example, it can roll back services if pods fail to start after a deployment or increase pods on a service if CPU and network queries per second (QPS) exceed a threshold.

Since a Random Forest Classifier is an ensemble of many decision trees, each trained on a subset of the features, it can effectively model non-linear data while minimizing overfitting by averaging multiple decision trees and providing multi-class outputs. That’s why I would prefer them over support vector machines (SVMs) and decision trees which don’t provide all those advantages.

LSTMs for Anomaly Detection Based Smart Alerting

Metrics such as CPU usage and network stats like QPS are continuously emitted by a service. You can naturally interpret them as time series and use a long short-term memory (LSTM) neural network to find anomalies. LSTMs are a type of recurrent neural network (RNN) that are good at “remembering” long-term events.

An LSTM trained on, say, CPU utilization data can ignore seasonal events such as nightly backups and unimportant events like minor rises during maintenance events such as host-to-host migrations. It also can alert on things like an unexplained rise during off-peak hours caused by a security breach.

For example, you could train the LSTM on historical data of overall traffic QPS. Once deployed, it references the current QPS and predicts future values that you can later compare to the real future values. If there’s too large of a difference, it sets off an alert. The detected anomalies can also be cataloged and used to fine-tune the model to increase performance over time.

Q-Learning for Autonomous System Configuration Updates

Getting a bit more advanced, Q-Learning is a type of reinforcement learning algorithm that can look at performance metrics, logs, and events such as restarts and deployments, and then update the system configuration. For example, if the metrics point to strained CPUs, the algorithm can decide whether to scale up the pods by adding additional CPUs or spin up additional pods. A reinforcement learning agent can learn from the action’s outcome and adjust its policy without human intervention.

To build this, you would have to define the states (various performance metrics), actions (scale up pods, etc.), and rewards (system performance) following an action. Then the model can be trained on historical data without much manual labeling, and it could be back-tested to ensure good performance.

LLMs for System Management With Natural Language

I bet you know by now what a large language model (LLM) is and how powerful LLMs can be. While LLM-based models can only be trained by well-resourced companies like OpenAI, the APIs for these models are available for public use, albeit at a steep price. But if you do have access to an LLM, you can build a bot that accepts natural language instructions, like “Set up a new service using X image in the development environment and write a deployment report,” after which a well-integrated LLM can carry out the actions.

If an LLM is not at hand, Bidirectional Encoder Representations from Transformer (BERT) is another model used for Sequence-to-Sequence (seq2seq) use cases. It can accomplish similar things, in a more limited but more controlled way. For example, “Restart the web server on node 5” would translate to {“action”: “restart”, “target”: “web server”, “node”: 5} which can be interpreted by a traditional program.

Looking Forward

AI is changing the face of product development and has become a common fixture of software and technology in general. While it’s exciting to use the cutting edge of AI, it’s also enlightening to learn how it can be used directly to reduce toil and improve business outcomes. Simple models can achieve this end by offering proven results for smaller investments, while larger models could revolutionize the SDLC at your company. The only wrong choice is not thinking about it at all and continuing with business as usual.

May 10, 2024
The Future of Jenkins in 2024

As software development practices evolve, the tools that support them must also adapt to stay relevant. Jenkins, the venerable automation server used by many DevOps teams, is at such a crossroads as it faces the changing landscape of cloud-native applications and Kubernetes adoption.

This article examines Jenkins’ current utility, particularly its features that have made it a staple in CI/CD practices, and addresses the question of whether it will maintain its relevance in 2024 and beyond.

How Jenkins is Used by DevOps Teams

CI and CD

Jenkins has become an integral part of many DevOps teams due to its CI/CD capabilities. Continuous integration (CI) is a software development practice where developers regularly merge their code changes into a central repository. Continuous delivery (CD) is the practice of delivering the software in short cycles to ensure that it can be reliably released at any time.

Jenkins supports the CI/CD approach in a number of ways. First, it allows developers to automate code integration. This means that whenever code changes are pushed to the repository, Jenkins can automatically build and test the project. Second, Jenkins allows for the continuous delivery of software by automating simple deployment processes (although it lacks full deployment orchestration capabilities).

Build Pipeline Visualization

Another key feature of Jenkins is its build pipeline visualization. This feature allows DevOps teams to visually represent the process of software delivery from code integration to deployment. The pipeline visualization provides a clear and intuitive representation of the software delivery process, making it easier for teams to understand and manage the process.

Build pipeline visualization in Jenkins is customizable, allowing teams to tailor it to their specific needs. For example, it can be customized to show the status of each stage of the delivery process, from code integration to testing to deployment. This allows teams to quickly identify any issues or bottlenecks in the process and address them.

Automated Testing

Automated testing is another area where Jenkins shines. With Jenkins, DevOps teams can automate the process of testing their code by running scripts and integrating with automated testing tools as part of the build process.

Jenkins supports a wide range of testing frameworks and tools, allowing teams to choose the ones that best fit their needs. Moreover, Jenkins can be configured to automatically run tests whenever changes are made to the code. This ensures that any issues or defects in the code are detected early and can be addressed promptly.

Workflow Scripting

Workflow scripting is another key feature of Jenkins in DevOps teams. With Jenkins, teams can script their workflows, automating the process of software delivery. This allows teams to standardize their workflows, ensuring that each delivery process is consistent and predictable.

Jenkins supports the Groovy scripting language for workflow scripting. This means that teams can use the power and flexibility of Groovy to script their workflows. Moreover, Jenkins provides a user-friendly interface for scripting workflows, making the process easy even for those who are not familiar with Groovy.

Current Usage and Adoption Statistics

As of 2023, Jenkins remains a leader in the CI/CD space. With its extensive plugin architecture and large user base, it is used by thousands of organizations worldwide. According to the 2022 DevOps Survey by the Cloud Native Computing Foundation (CNCF), Jenkins was used by over 50% of respondents for automation and CI/CD, maintaining a significant presence in the market.

According to the Jenkins website, there are over 1 million active Jenkins users and more than 200,000 active installations worldwide, with the platform handling 1 million+ jobs per day. It had about 44% of the global market share for CI/CD in 2023.

The growth trajectory of Jenkins, however, has shown signs of plateauing when compared to newer, more cloud-native-focused CI/CD tools. This observation is backed by Google Trends data, reflecting a steady but not sharply increasing interest in Jenkins over recent years.

Despite facing competition from emerging tools designed for cloud-native development, such as GitHub Actions, GitLab CI and CircleCI, Jenkins has responded by evolving its ecosystem. The creation of Jenkins X is a strategic move to cater to Kubernetes users and address some of the criticisms related to cloud-native development. However, as we’ll discuss, Jenkins X has not been readily adopted by the industry as of yet.

Limitations of Jenkins in Cloud-Native Environments

While Jenkins is a powerful tool for DevOps teams, it has serious limitations in cloud-native environments. These limitations stem from the fact that Jenkins was originally designed for traditional, monolithic applications, not for the dynamic and distributed nature of cloud-native applications.

Statefulness

One of the critical limitations of Jenkins in cloud-native environments is its inherent statefulness. Cloud-native architectures typically favor stateless applications, which are easier to scale and manage in dynamic environments like Kubernetes.

Jenkins, however, relies on maintaining state—it stores configuration, build history and user data, making it challenging to scale and manage efficiently in a cloud-native setup. This statefulness requires persistent storage, which complicates deployments in distributed systems where resiliency and horizontal scaling are key. Moreover, the risk of data loss or inconsistency increases in cloud-native environments, particularly during scaling operations or when recovering from failures.

Configuration Complexity

Jenkins is often criticized for its complex configuration requirements. Setting up Jenkins in a cloud environment involves intricate configurations that are not only time-consuming but also prone to errors. This complexity is exacerbated when integrating with modern cloud-native tools and services, requiring extensive customization and maintenance of plugins.

In addition, Jenkins’ traditional master-slave architecture was not designed for dynamic cloud environments, where resources need to be provisioned and deprovisioned rapidly. This leads to a significant operational overhead for DevOps teams.

Immutable Infrastructure

Adopting an immutable infrastructure approach is a cornerstone of cloud-native environments, promoting the idea of replacing or rebuilding servers rather than modifying them.

However, Jenkins’ traditional model does not align well with this principle. It typically requires modifications to its runtime environment for updates or changes, contrasting with the cloud-native approach of immutable deployments. This can make it challenging to maintain consistency and reliability in cloud-native systems where automation and minimal manual intervention are paramount.

Microservices Limitations

Jenkins was originally designed for monolithic architectures and doesn’t naturally fit into microservice-based applications, which are prevalent in cloud-native environments. In microservices architectures, applications are broken down into smaller, independently deployable services.

Jenkins, however, tends to treat applications as monolithic units, making it less efficient for handling multiple, independent microservices. This limitation affects its ability to provide fine-grained control over individual services’ build, test and deployment processes. Consequently, Jenkins may struggle with the high degree of automation, flexibility and scalability required in microservices environments.

What is Jenkins X?

Jenkins X is an open source project that provides pipeline automation, built-in GitOps, and preview environments to help teams collaborate and accelerate their software delivery at any scale. It is an improved version of the original project, designed specifically for Kubernetes, a popular container orchestration platform.

Jenkins X streamlines the technical aspects of building, testing, and deploying software. It automates the installation of popular tools like Helm and Draft, making it easier for developers to get started. It also provides pre-configured pipelines so teams can start delivering software right out of the box.

Jenkins X also introduces a new Jenkinsfile format, Jenkinsfile.yaml, which provides a more straightforward and human-readable way to describe pipelines. This new format, combined with the automation provided by Jenkins X, helps reduce the complexity of pipeline configuration and management in Kubernetes.

Limitations and Criticisms

Jenkins X has not been widely adopted by the software industry and has not achieved the respect and prominence of its predecessor, Jenkins. Let’s review the main limitations and criticisms of the tool.

Lack of Maturity

Jenkins X’s relatively recent entry into the market means it has not had the time to mature like its predecessor. This lack of maturity can manifest in several ways, including fewer features compared to established tools, fewer community-tested plugins, and potentially more bugs or unresolved issues. Due to the fast-paced evolution of cloud-native technologies, tools need time to adapt and stabilize; Jenkins X is still in this phase of its lifecycle.

Moreover, a mature tool typically has extensive documentation and a large community of users who can offer support and share best practices. While Jenkins X has seen growth in its user base, it still lags behind more mature CI/CD solutions in these areas. This can lead to challenges when teams are looking to solve complex problems or when they require advanced support.

Limited to Git Projects

Another criticism of Jenkins X is that it’s limited to Git projects. While Git is the most popular version control system, it’s not the only one. Some teams may use other systems like Mercurial or SVN, and these teams would be unable to use Jenkins X.

Even within the world of Git, Jenkins X has limitations. For example, it doesn’t support all Git hosting platforms. While it works with popular platforms like GitHub and Bitbucket, it doesn’t support others like GitLab. This can be a significant limitation for teams that use unsupported platforms.

Community Support

Finally, there are concerns about community support for Jenkins X. While the project is open source and theoretically has the potential for wide community involvement, some critics argue that it’s primary source of support is CloudBees, the company behind Jenkins.

These critics worry that if CloudBees were to withdraw its support, the project could falter. They also argue that the project’s focus on certain tools and platforms (like Kubernetes and Git) reflects CloudBees’ business interests more than the needs of the broader DevOps community.

Conclusion

In conclusion, Jenkins’ role in the CI/CD landscape has been pivotal, and its continued evolution with projects like Jenkins X demonstrates an effort to adapt to the needs of modern, cloud-native environments. While Jenkins X has yet to reach the widespread adoption and maturity of its predecessor, its specialized focus on Kubernetes and GitOps reflects the shifting priorities in software development.

Jenkins, despite showing signs of plateauing in growth, still possesses a strong user base and an established presence in the market, supported by a vast plugin ecosystem and a robust community. Its limitations in cloud-native contexts—such as managing statefulness, configuration complexity, and supporting immutable infrastructure—pose challenges that Jenkins X seeks to address.

As we approach 2024, the relevance of Jenkins will likely hinge on its ability to integrate more seamlessly with cloud-native technologies and the willingness of the community to evolve its ecosystem. The breadth of its adoption and the depth of its integration in existing development pipelines mean Jenkins will continue to be a significant player in the DevOps toolchain. However, its long-term relevance will depend on the project’s ability to innovate and remain adaptable in a landscape that is rapidly embracing cloud-native solutions.

December 15, 2023
Choosing the Right Software Deployment Tools
Imagine this scenario: After investing countless hours in developing your software, it is finally ready for deployment. Now, you must select the appropriate software deployment tools to guarantee a seamless release.

This article will guide you through the concept of software deployment and discuss various tools that cater to different needs—from dedicated deployment tools to comprehensive CI/CD pipeline solutions and GitOps approaches. By understanding these categories and their benefits, you’ll be well-prepared to make an informed decision on which tool is the best fit for your organization’s needs.

What is Software Deployment?

Software deployment plays a crucial role in the software development life cycle. It involves packaging, distributing and installing an application on target systems or environments, aiming to make your software accessible and usable for end-users seamlessly.

In today’s fast-paced world, companies must be able to deliver applications rapidly and reliably. A well-planned software deployment strategy ensures that new features reach users promptly while minimizing downtime and errors during updates.

What to Look For in a Software Deployment Tool

When selecting the ideal software deployment tool, consider the following factors:
- Automation: The tool should automate complex software deployments, making the deployment process faster and more efficient.
- Continuous deployment: Continuous deployment tools allow you to deploy software repeatedly, ensuring consistent deployments. Choose a tool that automates release management and integrates automated builds throughout the development process.
- Scalability: The tool should be capable of handling large-scale deployments and distributing software across multiple servers. Opt for a tool that enables applications to deploy to multiple servers simultaneously.
- Integration: The tool should integrate with version control and other third-party tools, making the deployment process seamless. Look for a tool that integrates with modern DevOps processes and connects automated builds to configuration management.
Full CI/CD Tools

Bamboo

Bamboo by Atlassian is a robust tool that integrates seamlessly with other Atlassian products, such as Jira and Bitbucket.

It offers first-class support for multiple languages and platforms while providing built-in deployment projects for smooth releases.

GitLab CI/CD

GitLab CI/CD, an integral part of GitLab, simplifies pipeline management using YAML configuration files stored within your project repository.

This approach enables easy collaboration among team members while ensuring consistent build environments across all stages of development.

Codefresh

Codefresh is a cloud-native solution designed specifically for Kubernetes deployments, It provides a modern interface to manage pipelines visually, along with advanced real-time debugging capabilities.

Dedicated Deployment Tools

Jenkins

Jenkins was originally conceived as a continuous integration (CI) tool, but many organizations use it mainly for deployment. It is an open source automation server that provides support for building and deploying software projects, as well as to schedule and run tasks at regular intervals. Learn more about Jenkins in this in-depth blog post.

Harness

Harness is an intelligent software delivery platform that simplifies deployments through automation and machine learning. Their continuous delivery-as-a-service (CDaaS) approach helps you deploy faster while minimizing risks and errors.

AWS CodeDeploy

If you’re using Amazon Web Services (AWS), AWS CodeDeploy might be a perfect fit for your needs. This completely managed service automates code deployments to any instance or serverless Lambda function in AWS, guaranteeing consistent results every time.

Spinnaker

An open source project backed by Netflix and Google, Spinnaker offers powerful multi-cloud continuous delivery capabilities for deploying applications at scale across various cloud providers like AWS, GCP, Azure, Kubernetes clusters, etc.

GitOps Tools

Argo CD

Argo CD is a declarative, Kubernetes-native continuous delivery tool that uses Git repositories as the source of truth for application definitions. It simplifies rollbacks, provides audit trails and enforces consistency across environments.

Flux CD

Flux CD, another Kubernetes-focused option, automates deployments by monitoring your code repository for changes and applying them to your cluster accordingly. Bonus: It supports multi-tenancy and integrates with Helm charts.

Weave GitOps Core

Lastly, Weave GitOps Core offers an open source foundation for managing Kubernetes applications using the power of GitOps workflows. This versatile tool can be extended to fit various use cases while maintaining simplicity in its core functionality.

Conclusion

There are many software deployment tools available that can help automate the software delivery process, reduce errors and improve collaboration between development and operations teams. From open source tools like Jenkins to cloud-native solutions like AWS CodeDeploy, each tool has its own strengths and features that make it suitable for different use cases.

When selecting a software deployment tool, it is important to consider factors such as compatibility, automation, scalability, security, monitoring, deployment environment compatibility, integration with existing tools and support and documentation. By carefully evaluating these factors and choosing the right tool for your specific needs, you can ensure that your software deployment process is efficient, reliable and successful.

Image Source: Freepik
June 21, 2023
The Dog Ate My DevOps Dozen

As the new Techstrong Group team member, I wasn’t familiar with the DevOps Dozen awards. The basics: It’s a cool program that recognizes excellence across a number of categories in the DevOps world. The winners will be announced during our Predict 23 conference on January 12. I have to tell you that the submission process was interesting.

It’s not surprising that many of the submissions came in on the last day. Some within the last hour of the last day. But what was surprising were some of the excuses we got when companies inevitably asked for extensions.

No one actually used the “dog ate our submission” excuse—mostly because, well, it’s an online submission process so, unless the dog swallowed a computer, that’s not really a thing anymore. It seemed that honesty prevailed this year. We got a bunch of folks that said they were working on a major launch; others said they’d been consumed with getting ready for KubeCon and asked for a little more time.

We were somewhat sympathetic since we also were getting ready for KubeCon and totally understand the all-hands-on-deck push required to get ready for a major event. We also encountered some issues with our payment processor which prevented the completion of some submissions. We’ve fixed the issue and we’ve learned from that experience, so it won’t be a problem next year.

In terms of the categories, unsurprisingly the top three that attracted the most submissions were “Best DevSecOps Solution,” “Best End-to-End DevOps Tool and Service” and “Best New DevOps Tool/Service Provider”. Given the move to integrate security into the DevOps environment and the move to integrate the variety of DevOps tools onto a platform, the popularity of those two categories was expected.

The next wave of categories included “Best Observability Solution,” “Best Cloud-Native Security Solution/Service” and “Best CI/CD Tool” to round out the top six. Again, less strategic and more tactical in this second group.

If you are looking to go where the crowds aren’t, the least-crowded three categories in terms of submissions were “Best DevOps-Related Video Series (Video),” “Best DevOps Transformation (non-vendor)” and “Best DevOps Presentation of the Year.” Maybe those aren’t the sexiest, most buzzworthy categories, but the competition is minimal.

So what did we learn from this year’s submission process? Try not to wait until the last minute and that the most competitive categories are those that deal with hot issues. Keep an eye out for the beginning of public voting, and we’ll see you at the awards ceremony during Predict 23!

November 1, 2022