devopsy.kinsta.cloud

Author: Garima Bajpai

Co-Developing an AI Native Observability Platform 

As AI capabilities continue to evolve, AI is becoming central to managing the growing complexity of distributed, hybrid enterprise environments, enabling more effective analysis, correlation, and automation across interconnected systems.

Traditional infrastructure and specifically network monitoring approaches, often built around siloed tools and static thresholds, struggle to keep pace with the scale, velocity, and interdependencies of modern systems. Further blurring the boundaries between network, application, and infrastructure domains makes it harder to isolate root causes and maintain operational resilience. In this context, AIOps platforms have emerged as one response to the growing need for integrated observability, automation, and data-driven decision-making.

At AI Field Day, Selector AI presented an AIOps platform, which can be considered a foundation for co-creating more adaptive and data-driven network operations. Rather than positioning it purely as a product choice, it embraces the SaaS approach, considering professional services as part of the offering, coupled with the product features and encouraging a co-development approach towards the platform instance for customers.

Demonstrations from the AI Field Day highlight the capability of full-stack observability with a data-centric approach, where data becomes the core part of the stack. Selector’s strength lies in its data-centric foundation, ingesting diverse, multi-domain sources metrics, logs, configs, alerts, and topology into a unified analytics layer. Unlike model-first tools, it prioritizes raw telemetry correlation via ML before layering AI, creating a “single source of truth” that slashes alert fatigue and supports hybrid/cloud environments without siloed dashboards. A unified data approach combined with the co-development of the specific platform instance, provides a more deterministic way of visualizing the problem, combined with causal analysis, which can help identify root causes more efficiently.

Another strategic feature embedded within the platform is the Network Language Model, built on a fine-tuned Model with vast networking telemetry data, which bridges natural language queries to complex ops tasks. It understands domain-specific terms (e.g., interface connectivity, routing paths) and powers Slack/Teams chats. This capability provides Selector AI advantage to advance beyond basic observability with AI agent workflows that enable autonomous, explainable network operations via its agent framework. These agents leverage Retrieval-Augmented Generation (RAG) with the Network Language Model to process unified telemetry, then trigger actions.

Striking comparison with building an in-house platform offers customization but typically requires significant investment often on the order of 18 to 24 months, multimillion-dollar budgets, and dedicated engineering teams. In addition, ongoing maintenance can increase as telemetry volumes grow and machine learning models require continuous tuning. Over time, internally developed systems may accumulate technical debt, particularly if they struggle to keep pace with evolving data and operational complexity. In contrast, purchasing a platform such as Selector, where organizations can engage in a co-development approach, may reduce initial development effort and accelerate deployment, with integrated capabilities like cross-domain correlation, incident summarization, and extensibility through a partner ecosystem. Another highlight is that the Selection instance is per customer, which means that there is no need for additional overheads.

The role of AI in operations is also evolving. Rather than optimizing capabilities like site reliability, the approach tends to shift the focus toward higher-level validation and decision-making. This creates a collaborative model where human expertise, operational data, and machine intelligence reinforce each other.

Adoption of such platforms often benefits from a phased approach. Initial efforts may focus on a limited proof of value, targeting a small number of critical services to measure improvements in alert reduction and incident response times. Subsequent phases can expand telemetry ingestion, introduce agentic workflows, and automate routine operational tasks, supported by cross-functional governance structures. Over time, organizations may extend capabilities toward predictive operations, capacity planning, and broader automation, while continuously evaluating outcomes against defined performance and cost metrics.

Lastly, co-creation allows for the customization of AI models and analytics to fit unique customer needs. This “mass customization” enables teams to create specific actionable insights rather than relying on generic, “one-size-fits-all” heuristics, according to Selector’s blog.  These elements combine for agentic, self-healing networks, aligning with AI Field Day 8 themes of production-scale inference and infrastructure evolution.

From a strategic perspective, platforms like Selector can be viewed less as a standalone product and more as enablers of operational evolution. The long-term value depends on how effectively organizations integrate them into their workflows, align them with business objectives, and build internal capabilities around them.

May 26, 2026
DevOps for GenAI Toronto Edition Hackathon: Unlocking New Opportunities
This week in Toronto, at the DevOps for GenAI Hackathon, something remarkable occurred: Industry professionals, academic leaders and learners representing top academic institutions, fintech, technology firms, and consulting companies came together in a high-intensity innovation sprint that blurred the lines between academia and enterprise.

Across four banking teams (including winners from Scotiabank), three technology and consulting companies, and several other entrants from other known tech firms like RBC, Shopify, and beyond, one theme emerged clearly: when fresh minds and experienced practitioners collaborate, they can outpace traditional enterprise development cycles by orders of magnitude.

The hackathon wasn’t about flashy demos; it was about building production-ready, DevOps-grade systems for GenAI deployment and operations. And, strikingly, the student-led and mixed university-industry teams often delivered robust solutions that created a learning curve for veteran professionals, mentors, and leaders.

Setting the Scene: The Hackathon Context

The DevOps for GenAI Hackathon, held on November 3, 2025, in Toronto, brought together experts from DevOps, platform engineering, and AI/ML, alongside bright student innovators. The challenge was clear: build real-world, enterprise-caliber systems that integrate DevOps best practices with the operational demands of generative AI.

Participants tackled problems such as:
- Securing training data within governance constraints
- Monitoring GenAI systems for hallucinations and drift
- Automating CI/CD pipelines for model deployment
- Building maintainable, auditable, and compliant AI infrastructure
This was not a “demo day”; it was a proving ground for operational-grade GenAI systems worthy of venture capitalist attention

What the Winning and Standout Teams Built

In addition to the overall winner, several teams were recognized for standout contributions across different categories.

Vulnerability Resolution Agent (Scotiabank Team)

The winning team from Scotiabank developed the Vulnerability Resolution Agent, a DevSecOps innovation that utilizes AI-assisted workflows to automatically resolve GitHub security alerts directly within the developer’s IDE.

Built in Python 3.12 with FastAPI, it acts as a real-time bridge between GitHub Dependabot alerts and the Cursor IDE via the Model Context Protocol (MCP). When a new vulnerability appears, the system triggers a webhook that streams the issue directly into the developer’s workspace where custom MCP tools, such as get_latest_vulnerability and suggest_vulnerability_fix, provide instant AI-driven remediation.

By embedding security automation into the developer workflows, the team demonstrated how context-aware AI tooling can compress remediation time from hours to seconds.

Repo: https://github.com/CanadaDevOpsCommunity2025/Vulnerability-Resolution-Agent

ParagonAI-The-Null-Pointers

This project demonstrates using multiple GenAI agents to automate the triage of customer support tickets. The system is built with a CLI that deploys, manages, and exposes each agent as an independent API endpoint.

Automatically summarize, analyze sentiment, and route support tickets to the appropriate queue using GenAI agents hosted as microservices/APIs.

Repo: https://github.com/CanadaDevOpsCommunity2025/ParagonAI-The-Null-Pointers

HemoStat – Autonomous Container Health

Third place went to HemoStat, an AI-driven, multi-agent system for real-time monitoring, diagnosis, and self-healing of Docker containers. It integrates GPT-4 and Claude via LangChain to perform root-cause analysis and trigger automated remediation through a network of intelligent agents.

Using Redis Pub/Sub for event orchestration and Prometheus + Grafana for observability, HemoStat bridges the gap between AIOps and DevOps, embodying what “autonomous infrastructure” can look like.

It’s not just about detecting anomalies; it’s about fixing them automatically.

Repo:https://github.com/CanadaDevOpsCommunity2025/HemoStat

Orange Honey Mustard – AI Observability with Speech Recognition

Awarded “Most Innovative,” the Orange Honey Mustard project combined AI observability, monitoring, and speech recognition into a unified, production-ready platform.

With a FastAPI backend, a React/Node.js frontend, and OpenAI’s Whisper API for speech-to-text, it featured full-stack observability using Prometheus and Grafana, allowing users to visualize performance, latency, and transcription accuracy in real-time.

This system exemplified how AI services can be made observable, auditable, and accessible, aligning with next-generation MLOps principles.

Repo: https://github.com/CanadaDevOpsCommunity2025/AI-Observability-Monitoring-Speech-Recognition-Orange-Honey-Mustar

Why Academia and Industry Teams Excelled Together

Several common patterns emerged across the most successful teams.

Freedom from Enterprise Baggage

Teams approached problems from first principles unbound by legacy processes, slow governance, or entrenched architectures. This fresh perspective enabled cleaner designs and faster decision-making.

Curiosity Over Conformity

By questioning “why” rather than accepting “how,” they exposed assumptions that often hold back enterprise DevOps. Their divergent thinking fostered exploration before convergence a key ingredient for breakthrough innovation.

Rapid Iteration & High Risk Tolerance

Without the fear of production downtime or compliance hurdles, these teams could experiment freely, fail fast, and iterate quickly. That pace of learning is often impossible in traditional corporate environments.

Modern Tool Fluency

Students and tech-savvy professionals alike leveraged containerization, IaC, observability frameworks, and MCP integrations practices that some enterprises still treat as “cutting edge.”

Enterprise Awareness, Student Boldness

Crucially, these teams didn’t just chase novelty; they built for governance, reproducibility, and safety. Their work proves that “fresh and savvy” thinking can coexist with enterprise-grade rigor.

Implications for the Enterprise

The following examples highlight practical lessons for organizations modernizing their DevOps practices.

Re-evaluate Assumptions

Ask: If we started from scratch, how would we build this today? Sometimes innovation begins by unlearning “how we’ve always done it.”

Empower Lightweight Teams

Small, cross-functional teams with minimal bureaucracy can produce outsized innovation, mirroring the dynamics of hackathons inside the enterprise.

Adopt Modern Tooling

Infrastructure as Code, containerized microservices, and observability-first design are no longer optional—they’re the new foundation.

Embrace a Culture of Experimentation

Prototype often. Fail fast. Reward curiosity. Enterprises can thrive when they make space for playful innovation.

Partner With Academia

Hackathons are not just recruiting events, they’re collaborative R&D labs. The synergy between students and professionals, as seen here, accelerates innovation on both sides.

Next Steps

Building on the success of the DevOps for GenAI Hackathon, the next phase is about sustainability and expansion.
- License the winning and featured projects under open collaboration frameworks to make them publicly available for continued development and community contributions.
- Launch a structured program inviting contributors from academia, startups, and industry to help mature these prototypes into reusable, production-ready innovations.
- Engage potential investors to accelerate project adoption and strengthen the ecosystem around AI-driven DevOps + GenAI.
- Develop partnerships with universities and research institutions to integrate hackathon outcomes into curricula and joint innovation labs.
Conclusion

The DevOps for GenAI Hackathon demonstrated that when students, academics, and industry experts collaborate, the results can rival those of the best enterprise teams. Scotiabank’s victory, alongside individual contributions from RBC, Shopify, and others, underscores that innovation thrives at the intersection of fresh thinking and operational excellence.

These teams didn’t just prototype, they built scalable, auditable, production-ready systems. Their success challenges enterprises to open up, experiment faster, and invite the next generation of technologists into the heart of their DevOps transformation.

The message is clear: the future of enterprise innovation isn’t locked in a corporate playbook; it’s emerging from hackathons, classrooms, and cross-industry collaboration.
December 17, 2025
DevOps for GenAI Toronto Edition Hackathon: Unlocking New AI Market Opportunities
Toronto’s vibrant tech community is abuzz as the DevOps for GenAI Hackathon returns with John Willis, this time with ambitious new themes that reflect both global trends and uniquely Canadian innovation. Building on the foundational insights from the first Hackathon held in Ottawa organizers are expanding the event to spotlight end-to-end lifecycle automation for generative AI, advanced observability, agent scalability, and the critical dimensions of AI security and governance.

This progression to “DevOps for GenAI Hackathon Days” signifies a bold step forward in purpose and scope. The hackathon not only honors the collaborative foundation of DevOps but accelerates it toward a future of trustworthy, ethical, and scalable artificial intelligence in the software lifecycle, uniting communities globally around common challenges and innovation goals.

Expanding the Hackathon Theme

The Toronto edition introduces targeted tracks that tackle the most pressing issues for GenAI in production:
- MLOps Pipelines: Teams will design fully automated pipelines for LLMs and diffusion models, focusing on robust data management, fine-tuning, CI/CD, and drift detection using battle-tested tools (MLflow, Kubeflow, Argo Workflows).
- AI Observability & Monitoring: The hackathon emphasizes building centralized platforms for measuring latency, token usage, and output quality, as well as integrating safety detectors for hallucinations and toxic content.
- OpenTelemetry, Prometheus, and Grafana play leading roles, building on winning projects like InnerAI and InsightAI_Minions from the Ottawa edition.
- Agent Deployment at Scale: Participants will architect systems to containerize, orchestrate, and scale multiple AI agents in Kubernetes clusters—applying DevOps best practices for reliability, scalability, and observability.
Innovation Meets Real-World DevOps

What sets the Toronto hackathon apart is its strategic alignment with enterprise challenges. Unlike many theoretical or demo-driven AI events, the Toronto edition emphasizes hands-on solutions to core business obstacles faced when deploying GenAI systems at scale.

By structuring challenges and judging through an enterprise lens, Toronto’s hackathon delivers practical, production-ready GenAI approaches poised for immediate adoption as organizations behind them scale AI across their operations.

Security and governance are central to the Toronto DevOps for GenAI hackathon, recognizing today’s enterprise-scale risks around generative AI adoption. The event calls upon security firms and open-source security researchers to rigorously review, test, and contribute to the official repository at https://github.com/CanadaDevOpsCommunity2025

Security companies, open-source defenders, and researchers are encouraged to:
- Fork, audit, and contribute to the repository,
- Submit issues or pull requests for any discovered vulnerabilities,
- Help harden IaC, agentic workflows, and GenAI application layers by integrating advanced governance tools, such as policy-as-code engines and explainable AI dashboards.
This approach not only facilitates continuous improvement and defense-in-depth for community-driven projects but also sets an example for responsible and transparent AI innovation helping protect users, enterprises, and the broader tech ecosystem.

Accelerating Community and Collaboration

Toronto’s edition continues the hackathon’s tradition of uniting developers, data scientists, and platform engineers, but extends collaboration across industry, academia, and open-source communities. Global tech leaders such as John Willis (co-author of The DevOps Handbook) advocate for solution designs that not only prototype novel ideas but actively bridge theory with scalable practice, helping participants prepare for the “complexities of modern professional environments”.

Addressing Strategic Gaps

While past hackathons delivered creative dashboards, automated benchmarking, and middleware solutions, Toronto organizers have identified strategic gaps such as:
- Incomplete CI/CD integration and lack of deep automation.
- Insufficient multi-cloud scalability.
- Unaddressed data privacy and compliance requirements.
Teams are now challenged to harden their prototypes, embrace production-ready architectures, and pioneer standardized approaches to observability and evaluation, setting the stage for broader enterprise adoption.

The Path Forward

In summary, the Toronto edition positions itself as a launchpad for next-generation GenAI and DevOps convergence, pledging to mature open-source solutions into production-grade platforms. By blending technical rigor with collaborative energy, this hackathon not only energizes the local tech community but promises to shape the future direction of AI development in enterprise and beyond.

Hackathon participants are encouraged to build, test, and iterate on ideas with the knowledge that their innovations today will be the foundations of tomorrow’s trustworthy, scalable, and ethical GenAI platforms.

The Toronto DevOps for GenAI Hackathon is setting the pace for AI innovation, and its path forward will see its unique blend of DevOps and generative AI scaling across new geographies and themes in 2026–27. Plans are already in motion to expand to key innovation hubs in North America, EMEA, and APAC, each bringing new production-grade challenges, vertical themes like AI security, sustainability, and agentic orchestration, and expanded community participation.

By keeping sponsorship, contribution, and mentorship onboarding adaptable and inclusive, the GenAI hackathon ecosystem will foster sustainable growth, deeper industry ties, and richer participant experiences as it scales geographically and thematically.

Join us at the Hackathon

https://www.eventbrite.com/e/devops-for-genai-hackathon-tickets1407877793379?aff=oddtdtcreator
November 17, 2025
Accelerating GenAI Innovation with DevOps Hackathons—Hackathon Insights, Strategic Gaps and Future Directions
Importance of Hackathon

Hackathons have become a vital platform for driving innovation, collaboration, and skill development across diverse industries. These intense, time-bound events bring together programmers, designers, and problem solvers to rapidly prototype new ideas and tackle real-world challenges. Beyond fostering creativity, hackathons enable participants to expand their technical skills, network with like-minded professionals, and gain industry recognition.

The “DevOps for GenAI” hackathon, recently held in Ottawa, showcased 15 competitive projects at the intersection of DevOps and Generative AI. The event highlighted the growing convergence of AI observability and prompt engineering, with projects split between these two domains. This white paper outlines the intent, technical gaps, strengths, future perspectives, and collaboration requirements of the presented projects, drawing from the available repositories and event summaries.
- AI Observability: Roughly half the projects focused on monitoring, debugging, and optimizing AI systems. These solutions aimed to provide real-time insights into model performance, resource utilization, and anomaly detection, reflecting the industry’s need for robust AI system management.
- Prompt Evaluation: The remaining teams targeted prompt engineering and evaluation tools for GenAI systems, aiming to improve prompt quality, automate testing, and provide feedback loops for continuous improvement.
The rapid adoption of large language models (LLMs) and generative AI (GenAI) systems in enterprise and research settings has elevated the need for advanced observability, monitoring, and accessibility solutions. This white paper explores the combined strengths and future potential of open-source projects produced during the DevOps for GenAI Hackathon, Ottawa edition. It particularly explores three projects in the AI Observability space —InsightAI_Minions, AI Observability Monitoring InnerAI, and GenA11yHelper—to deliver a unified, extensible, and accessible framework for GenAI deployment and operation.

AI Observability & Monitoring (Inner AI)

Presented by James Drayton Beninger, Amogh Thoutu, Farhan Mahamud, Shubhangi Singh, Bardia Azami, Inner AI Group, Canada DevOps Community 2025

AI Observability & Monitoring (Inner AI) middleware solution, designed to enhance transparency, reliability, and accountability in text-based generative AI (GenAI) applications. The solution integrates seamlessly into developer workflows, providing real-time monitoring, metrics, and testing capabilities for large language model (LLM) interactions. We discuss the system’s core strengths, outline future directions for research and development, and identify opportunities for collaboration within the broader AI and DevOps communities. The goal is to foster robust, trustworthy AI deployments across industries by advancing observability standards and practices. The Inner AI solution comprises four main modules:

• API Wrapper: Intercepts and augments LLM prompt/response cycles.
• Metrics: Collects detailed telemetry on LLM performance and behavior.
• Dashboard: Visualizes real-time and historical metrics for developers.
• Testing Module: Simulates end-user interactions to validate system robustness.

Key Strengths

• Seamless Integration: Designed as middleware, the system requires minimal changes to existing application code, supporting rapid adoption.
• Comprehensive Monitoring: Captures both quantitative (latency, error rates) and qualitative (response quality, prompt drift) metrics.
• Automated Testing: Simulates diverse user scenarios, enabling proactive detection of issues such as hallucinations or degraded model performance.
• Extensibility: Built using widely adopted frameworks (e.g., FastAPI, Prometheus, LangChain), facilitating customization and scalability.
• Developer-Centric Design: Provides actionable insights through an intuitive dashboard, accelerating debugging and model iteration cycles.

InsightAI_Minions: Enabling Observability and KPI Monitoring for LLM Deployment

By Denis Shleifman, Morteza Mirzaei, Dorian Gerdes, Manav Isrrani, Sachin Kumar, Canada DevOps Community 2025

InsightAI_Minions, an open-source framework for deploying, monitoring, and benchmarking large language models (LLMs) using DevOps best practices. The system provides automated KPI measurement, advanced alerting, and extensible observability for LLM deployments. We discuss the strengths of the platform, future development directions, and the need for collaborative efforts across the AI and DevOps communities to advance the state of LLM observability and reliability.

The rapid adoption of LLMs in production environments requires robust tools for deployment, monitoring, and performance evaluation. InsightAI_Minions addresses this gap by providing a comprehensive solution for automated LLM deployment and observability, leveraging modern DevOps pipelines and open-source monitoring stacks.

Strengths of InsightAI_Minions

• Automated Deployment: Utilizes GitHub Actions and Docker Compose for seamless LLM deployment, supporting continuous integration and delivery workflows.
• Comprehensive KPI Measurement: Simulates diverse test prompts (from a set of 50,000) to measure latency, token throughput, and success rates, providing actionable insights into LLM performance.
• Advanced Observability: Integrates OpenTelemetry (OTEL) for metrics collection and VictoriaMetrics for time-series storage, with visualization via Grafana dashboards.
• Proactive Alerting: Implements advanced alerting (warning and critical) with configurable Slack notifications, enabling rapid incident response.
• User-Defined KPIs: Allows users to define custom metrics and dashboards, enhancing flexibility for varied deployment scenarios.

InsightAI_Minions represents a significant step forward in operationalizing LLM deployments with robust observability, benchmarking, and alerting capabilities. Continued collaboration across industry, academia, and standards organizations is essential to address evolving challenges and ensure reliable, scalable, and transparent AI systems.

Future Perspectives

• Expanded Integration: Support for additional deployment platforms (e.g., Kubernetes, cloud-native environments) and CI/CD tools beyond GitHub Actions.
• Enhanced Metric Coverage: Inclusion of more granular performance and reliability metrics, such as GPU utilization, memory footprint, and prompt-specific analytics.
• AI-Driven Anomaly Detection: Incorporation of machine learning models for predictive analytics and automated anomaly detection in LLM behavior.
• Scalability Improvements: Optimization for large-scale, multi-model benchmarking and distributed deployments.

GenA11yHelper: DevOps-Powered Prompt Testing for Smarter GenAI Applications

By Yassine Sami, Mihir Soni, Dondre Samuels, Vaishali Jaiswal, Prompt Pilot Group, Canada DevOps Community 2025

GenA11yHelper is a lightweight, DevOps-driven tool designed to streamline prompt experimentation and optimization for large language models (LLMs). By integrating user feedback, automated prompt promotion, and robust versioning within a cloud-native workflow, GenA11yHelper empowers teams to iteratively enhance GenAI applications. This white paper highlights the project’s core strengths, outlines future development directions, and identifies key areas for community and industry collaboration to maximize the impact of accessible, effective GenAI systems.

• Integrated DevOps Workflow: Utilizes Terraform, Docker, and GitHub Actions to automate deployment, prompt versioning, and CI/CD, ensuring repeatability and scalability.
• User-Centric Feedback Loop: Incorporates real-time user ratings to identify and promote the most effective prompts, directly aligning model outputs with end-user needs.
• Cloud-Native and Lightweight: Deployed on AWS using EC2 and S3, with a minimal resource footprint, making it accessible for a wide range of teams and organizations.
• Open and Modular Architecture: Built with Streamlit, LangChain, and OpenAI API, enabling rapid prototyping and easy integration with other GenAI tools and workflows.

Future Perspectives

• Enhanced Feedback Analytics: Planned integration with Weights & Biases will provide advanced visualization and tracking of prompt performance over time, enabling data-driven optimization.
• Expanded Accessibility Features: Future releases aim to incorporate accessibility testing for prompts and outputs, ensuring GenAI solutions are inclusive for users with diverse needs.
• Support for Multiple LLM Providers: Extending compatibility beyond OpenAI APIs to include other LLM backends, fostering broader experimentation and resilience.
• Community-Driven Prompt Libraries: Establishing shared repositories of high-quality, versioned prompts to accelerate GenAI adoption and best practices across domains.

GenA11yHelper demonstrates a practical, DevOps-powered approach to prompt engineering for GenAI applications, with strong potential for future growth and impact. Continued collaboration across open source, industry, and academia will be essential to realize its vision of accessible, effective, and user-driven GenAI systems.

Strengths of Combined Approach

The integration of InsightAI_Minions, AIObservability-Monitoring_InnerAI, and GenA11yHelper represents a significant advancement in the operationalization of GenAI systems. By uniting performance monitoring, real-time observability, and accessibility, this approach lays the foundation for robust, transparent, and inclusive AI deployments—meeting the demands of modern enterprises and diverse user communities.

Combining InsightAI_Minions’ KPI benchmarking with InnerAI’s real-time middleware monitoring provides both macro (system-level) and micro (transaction-level) insights into LLM performance. Integration enables unified dashboards (Grafana, Prometheus) and customizable alerts, facilitating rapid incident response and proactive maintenance.

Unified AI Operations Platform

• Single pane of glass: The integration can evolve into a centralized AI operations platform, offering unified monitoring, alerting, and accessibility management for all GenAI assets.
• Self-healing AI systems: With comprehensive observability and alerting, automated remediation (e.g., rollback, scaling) becomes feasible, reducing downtime and manual intervention.

Overall DevOps for GenAI Hackathon Strengths

• Innovation: Teams demonstrated creative approaches to AI observability and prompt evaluation, including novel dashboards, feedback tools, and automated benchmarking.
• Community Collaboration: The hackathon fostered cross-disciplinary teamwork, with developers, data scientists, and DevOps engineers collaborating closely.
• Rapid Prototyping: Projects showcased the ability to rapidly prototype and iterate on ideas, leveraging open-source tools and cloud-native technologies.

Strategic Gaps in Coding and Further Considerations

Despite promising prototypes, several strategic coding gaps were observed:

• Integration Depth: Many solutions demonstrated basic integrations between DevOps pipelines and AI models but lacked end-to-end automation or seamless CI/CD integration.
• Scalability: Most projects were at the proof-of-concept stage, with limited attention to scalability, multi-cloud deployment, or enterprise-grade robustness.
• Security & Compliance: Few projects addressed data privacy, secure model deployment, or compliance with industry standards—critical for production AI systems.
• Testing Coverage: Automated testing, especially for AI-specific edge cases and prompt variability, was often incomplete or absent.

Future Perspective

• Production-Ready Solutions: Moving forward, projects should focus on hardening prototypes for real-world deployment, emphasizing scalability, security, and maintainability.
• Standardization: There is a need for standardized interfaces and APIs for AI observability and prompt evaluation tools, facilitating broader adoption within enterprise DevOps workflows.
• Continuous Learning: Integrating feedback loops to enable models and prompts to evolve based on real-world usage and monitoring data will be crucial.
• AI Ethics & Trust: As GenAI systems become more pervasive, embedding ethical guidelines and transparency into observability and evaluation tools will be essential.

Collaboration Required

• Industry Partnerships: Collaboration with cloud providers, AI platform vendors, and enterprises will help transition prototypes to production environments.
• Open-Source Community: Continued engagement with the open-source community can accelerate development, testing, and adoption of these tools.
• Academic Involvement: Partnerships with universities can drive research into advanced observability metrics, prompt evaluation methodologies, and AI system reliability.

Importance of Community Building

Beyond enhancing individual skills, hackathons foster strong community building by bringing together diverse talents and perspectives, which boosts networking, teamwork, and knowledge sharing. This sense of community not only accelerates professional development but also inspires collective innovation, creating environments where participants support each other and co-create impactful solutions that might not emerge from solitary efforts.

Hackathons as a Way to Bridge Academia and Students

Hackathons act as a key educational strategy linking academic theory with practical, industry-relevant experience. They cultivate a culture of creativity, collaboration, and innovation that benefits students by preparing them for the complexities of modern professional environments while also enriching academic programs through close ties with industry.

Conclusion

The DevOps for GenAI hackathon has energized the local tech community and highlighted both the promise and challenges of integrating GenAI into DevOps workflows. Addressing the identified coding gaps and fostering broader collaboration will be essential for maturing these innovations into impactful, production-ready solutions.
July 28, 2025
SRE in the Age of AI

Site reliability engineering (SRE) is a concept introduced by Google in 2004 and since then it has been adopted by various leading software organizations. In its purest form SRE is what you get when you treat operations like it is a software problem.

Industry-leading reports point out the strategic value that SRE offers to the software-centric community. Key takeaways from the sixth edition of SRE Report 2024 reflect the importance of SRE’s foundational role, specifically operationalizing cloud-native distributed software systems at scale. Another significant development is the introduction of the fifth DORA metric in 2021 — reliability, which clearly outlined the importance of reliability and the SRE practices.

With the seamless incorporation of software in every sphere of life, we must tightly integrate reliability practices pragmatically. The operational performance of our dynamic and complex software ecosystem is vital, and site reliability engineering practices come to our rescue.

With the next wave of more complex integration and convergence of software into more disruptive fields, such as material science, pharmaceuticals, health science, security forces and space technologies, it is evident that we cannot overlook operational performance indicators and software assurance. Evolving the SRE practices side by side, creating innovative assets for our SREs and finding new ways to operate a software-centric ecosystem in a reliable way are priorities for the leaders.

Furthermore, significant transformations are happening in the process of software development. Starting from the exponential rise of open source, requiring SRE to rethink and be open to new ways of collaboration to meeting the reliability expectation. Recovery from operational incidents or even updates or correction require new knowledge, new partners and new ways to operationalize reliability.

Moving to the next, immediate challenge is the deployment of software applications in a hybrid way cloud with on-premises. The new deployment models bring a fresh set of reliability challenges and introduce new forms of risks to deal with during runtime. Some areas of reference that can be affected are failover strategy, performance tuning and disaster recovery & rollback. SRE practices such as observability play a crucial role in the evolution.

Another major interruption is through generative AI, coding assistants and in general adoption of AI at scale. Lastly, the introduction of new regulations for data, a secure supply chain and rising demand for sustainable and green software contribute to the evolution of SRE at its core.

We discussed the key triggers of why SRE practices will evolve above and beyond the status quo. Let’s explore more about how SRE prepares themselves to respond to the next-generation user experience and customer demand with a suite of new capabilities. In the next section, we will further explore SRE-driven software operations at scale and associate key trends to the evolution of the practices.

Leading SRE Practice in the Age of AI

SRE practices are key to creating a resilient software ecosystem. Organizations will start seeing tangible results once the SRE focuses on incorporating new technology and ways and collaborating with new partners in the ecosystem. Further in this article, we will explore the approach toward value realization of the injection of AI into SRE evolution.

Tackling Open Source — SRE in Action with AI-Powered Assets

Let’s dive deep into the key areas described above, starting from open source. One of the main challenges for SREs is runtime maintenance of open source. It is estimated that 96% of codebases contain some open source that can result in operational overhead if organizations are not prepared enough. One of the examples is Log4j, where various organizations were affected by the transitive dependency, and it was reported that hundreds of hours were wasted in the process of dealing with it.

With the rise of open-source integration into mainstream applications, SREs are tasked to monitor and manage open-source vulnerabilities in real-time knowingly or unknowingly. As the complexity of the ecosystem increases, it becomes difficult without a pragmatic approach. Starting from ensuring that the software bill of material (SBOM) is in place and smart integration of SBOM into the runtime environment to quickly understand and react to the incidents, use of tools to quickly detect such vulnerabilities & react to them without creating additional toil.

Another possible approach for SREs is to proactively develop more collaboration with the open-source program office (OSPO) to create SRE assets such as playbooks, checklists, tools and its integration with operations such as SBOM and SW inventory management, including open-source libraries for better operational performance. The OSPO, along with SRE, can trigger AI-supported audits time to time to track the operational risks and mitigation strategies.

SRE Unwraps AI’s Potential for Next-Generation Reliability

Soon SREs will have to deal with machine-written, AI-generated code. It poses security and compliance challenges at a different scale. SREs must be prepared to tackle new types of vulnerabilities introduced by these coding assistants. Another area of consideration is bad actors can enhance their capabilities and inject malicious code by exploiting coding assistants. SREs continue to evolve their knowledge and assets with new tools that can defend against these threats introduced through AI assistants.

Advancing to take a deeper dive into the generative AI technology, most companies use open-source LLM models due to cost constraints. The usage of open-source LLM requires customization to adapt to workflows and integrate it with proprietary data to enhance the value. SREs can integrate with the data science community to provide early feedback. This feedback would help reduce hallucination, which is when AI models make up stuff. SREs can also measure the hallucination quotient, through metrics to monitor the performance of the models.

Looking forward to another capability is the use AI agents. There is substantial excitement about pairing AI agents with SRE. SREs will play a pivotal role in improving the output of AI agents by human preference and making it more useful with time.

There are various other possibilities for incorporating generative AI into SRE workflows. Generative AI is new, but its adoption is growing at a massive scale. By implementing a proactive approach, SREs can remain ahead of the curve and minimize the associated risks.

The Strategic Importance of AI-Enhanced SRE for Cloud-Agnostic Posture

Moving on to the transformation to a cloud-agnostic posture of leading organizations is an opportunity for SRE to step up. Some of the core operational challenges of onboarding to the cloud include cost-sprawl, increased clutter and security & compliance risks are highlighted in my previous article.

SRE can play a vital role in performance tuning of cloud deployment in runtime, enhancing rollback & recovery posture in the cloud. AI-powered backup can support SREs to dynamically adjust the regular backup schedules and recommend backup strategies for applications based on usage and criticality. Lastly, rollback planners intelligently model rollbacks by defining checkpoints, analyzing logs, etc. SRE can leverage AI capabilities for resource optimization and predictive maintenance based on historic data.

SRE practices such as observability, when powered by AI capabilities, can support SREs in this transformative journey. From monitoring operational KPIs and dynamically adjusting thresholds to exponential ideas of self-healing capabilities, next-generation observability tools are evolving in multiple dimensions.

Instead of the traditional fragmented ecosystem of observability, if we can standardize the interfaces to democratize the development of troubleshooting, predictive maintenance and insights of applications on top of observability tools, then SRE has the potential to create more assets for the community to simplify operations for complex cloud-native applications. AI-powered automation applications can produce key business insights, developer feedback dashboards, troubleshooting apps and much more. The possibilities with AI are unlimited if we standardize the ecosystem in some way.

Another area is the mitigation of security incidents. With generative AI tools, SRE can intelligently scan applications for runtime vulnerabilities and recommend corrective action, and AI agents can accompany SREs to help navigate the security challenge. When a security incident occurs, AI-powered security assessment and audits and proactive defense through pattern recognition and predictive system behavior provide SREs practical insights for better resilience.

Leveraging New Skills

The sixth edition of the SRE Report 2024 indicates that 53% of professionals consider AI will be valued. Technology can improve the efficiency and effectiveness of SREs and eventually enable them to prepare and respond to challenging technology-powered reliability incidents quickly. In developing organizational-level talent strategies for operations, the “Ops” side of DevOps is essential. Often reskilling and upskilling of “Ops” staff is not on the priority list of large organizations, which creates a strategic gap for reliability. The value of SREs leveraging these new skills will likely be seen over time, mostly in the last mile of introducing technologies in the development flow. The key challenge for SRE is to figure out where to start harnessing the power of new technologies. SRE communities will be uniquely valuable for embracing the evolving roadmap of SRE.

Conclusion

In today’s time, SREs are likely to move forward and regularly enhance their capabilities. To instill trust in the digital ecosystem, it is important for organizations to focus on the “Ops” side of DevOps. SREs in the long term can devise a resilient future by ensuring smooth operationalization of software combined with new technology. The best-case scenario is to have the right balance of investment into developer productivity with regard to investing into resiliency and operational efficiency with new assets, tools and skills for SRE.

June 28, 2024
Cloud-First, Cloud-Smart to Cloud-Minimalism – A Strategic Shift

Before we discuss the strategic shift in cloud computing adoption and various connected developments, let’s revisit the basics of cloud computing.

According to NIST, the essential characteristics of cloud computing are:

On-demand, with self-service capability: The consumer can provision the capacity without human intervention when required.

Board network access: The capability is available over the network and can be accessed easily through a standard mechanism.

The consumer benefits from resource pooling: The resources are dynamically assigned and reassigned. Capabilities are scalable as per consumer needs. The service should be measurable.

According to a report by Forrester, The State Of Cloud In The U.S., 2022, most U.S. enterprise infrastructure decision-makers are using at least one type of cloud deployment. Forrester released similar reports for Asia, covering Australia and New Zealand, China, India, Japan, Malaysia and Indonesia. Asia is witnessing targeted growth for cloud-based deployment for applications and infrastructure portfolios.

With more and more companies realizing the benefits of cloud adoption, it is fast becoming a battleground, where companies aggressively pursue their plans to quickly onboard cloud at a near-to-limitless scale. These massive plans come with the “cloud-first” strategy.

Cloud-First Strategy — An Overview

Many organizations have attempted to define and refine the “cloud-first” strategy. Let’s highlight some common principles that can be the baseline for further analysis. Cloud-first strategy is often described by the following:

1. Using cloud services as the default, considering the five characteristics of cloud computing.
2. Designing for the cloud, architect cloud-native and limit customization.
3. Adopting cloud service functionality, automation practices, cloud-based security and other features.
4. Monitoring and measuring cloud services in real-time.

Prioritizing cloud services has led to short-term gains, providing the edge on agility and scalability. However, it also poses some challenges, and cloud-first is not a “one size fits all solution.” To top that, many companies have indicated cost sprawl and overrun, security & compliance concerns. Recognizing these challenges in the “Report to the President on Federal IT Modernization,” released in 2017 by Executive Order 13800,1 the Office of Management and Budget (OMB) pledged to update the Government’s legacy Federal Cloud Computing Strategy (“Cloud First”).

Cloud-First to Cloud-Smart Strategy

Cloud smart seems like the next step in the adoption of cloud computing. Gartner estimates that 60% of organizations that have adopted a cloud-first approach will switch to a cloud-smart approach by 2023.

Cloud smart addresses the practical implementation challenges. It shares a more comprehensive outlook, providing critical guidance toward security, procurement and workforce alignment.

Finally, it provides more mid-term gains for organizations, rather than focusing on “the here and now” problems.

The CIO Council created a list of actions as concrete steps to be cloud-smart. It outlines several topics, including developing methods of optimizing cloud usage, security architecture enhancement and implementation of security control at scale and collaborative processes to develop innovative models, considering sustainability, vendor lock-in and workforce development. To maximize the value of cloud computing, the cloud-smart approach enables organizations to assess, plan and execute the right mix of private, public, edge and on-premises needs without sprawling into the cloud providers. The top considerations are:

1. Setting realistic goals aligned with the key business outcome.
2. A selective investment strategy toward cloud computing, tools and services.
3. A roadmap toward cloud optimization keeping sustainability goals in mind.
4. Tight control of security and compliance.
5. Cloud exit strategy.

But how does cloud adoption maturity exploit cloud computing? One of the possible ways is to transition into more cloud-minimalism, which is decluttering, refining the value dimension of cloud computing to a more meaningful level of adoption keeping sustainability in mind.

Cloud Smart to Cloud Minimalism

Cloud waste rose at an average of 30% of cloud budgets in 2021. In 2022, it was 32%, according to Flexera. Stormforge reported that 75% of organizations suspected cloud waste. According to a report by PWC, about 53% of enterprises were yet to see “substantial value” from their investment.

Organizations must look at decluttering and refining the value dimension of the cloud. There is a significant focus on security and data protection. Mixing clouds with on-premises cloud computing capabilities is on the cards.

Organizations are also willing to take the next steps toward sustainability. Cloud minimalism focuses on a few optimized activities rather than a big-bang approach toward onboarding. One of the main drivers to investigate the next-generation cloud computing strategy is due to competing visions, goals and future scenarios triggered by complex cloud-based supply-chain models, a substantial shift in security and privacy, regulations and adoption of machine learning and data-driven models.

Cloud minimalism does not mean rejecting or following a specific big-bang way but consciously selecting and intentionally moving workloads to the cloud effortlessly.

Some factors influencing the choice for long-term strategic evolution are:

1. Sustainability Goals Take Center Stage: Sustainability is not a checkbox item. For example, Amazon targets to power 100% of its data centers with renewable energy by 2025. Microsoft aims to be “carbon negative” by 2030. However, operationalizing sustainability is more than just looking at these CSP targets, it is about sustainable engineering practices, controlling and managing waste and underutilization.

2. Security and Privacy Reimagined: Mega cloud security to micro cloud security, the future is about everything. Specifically regulatory and compliance-related guidelines will refine geographical boundaries for hosting applications, and categorization of sensitive information will lead to new security controls, and in some cases, limitations around cloud onboarding. Moreover, the explosion of data-centric applications will need additional controls seen as overheads for cloud providers to host such applications.

3. From Cloud-Native to Edge-Native: With the development, we will seek on-premises, cloud and edge hosting models as a mix for cloud-native to edge-native applications. Architectural evolution and modernization will help to optimally define boundaries.

4. AI Goes Mainstream: The AI era demands more systematic handling of data resources. It is critically important to define ownership, rights and governance posture. In the context of the cloud, it becomes more important to be selective and pragmatic: Where who and how the data is handled. Cloud and data minimalism are disruptive ideas to start but can reap long-term benefits. Developing and evolving AI applications toward processing clean, optimized and precise data with a mix of cloud and on-premise capabilities.

Applied Cloud Minimalism

Recently, we have experienced an explosion of tools and services in the cloud. Cloud computing strategy requires careful due diligence, moving forward. It is time to think “lightweight cloud playbook”. Applied cloud minimalism focuses on the existing workload, decluttering and simplifying the architectural posture, meter usage and optimizing the footprint. Applied minimalism will guide redundant workloads, lift and shift-based workloads and outdated or legacy tools and services.

Applying cloud minimization principles is not a “one size fits all solution”. However, mindful onboarding can bring new opportunities to cloud technology, for example, cloud management tools, responsible sharing and data optimization algorithms, etc.

Conclusion

It is important to look at the strategic goals and outline the short-to-long-term needs of the organization for cloud adoption. It might seem lucrative to embrace the cloud in its entirety in the short term, but it can bring some bottlenecks in the mid-to-long term. Short-term benefits around wholesale cloud onboarding can quickly vanish if there is limited pragmatic thinking and planning. Selectivity in investing in the cloud can bring more benefits in the mid-term to long term. With a more visionary approach toward selecting use cases and determining architectural choices, measuring and metering the impact to further optimize the footprint can be one option.

This article presents a disruptive approach to clouds. We do not know if cloud minimalism is the only answer to cloud complexity and clutter, but it provides some more pathways to meaningfully onboard and explore cloud computing.

June 26, 2024
404: Role Not Found – Generative AI’s Impact on DevOps Jobs
Many DevOps professionals are wondering if generative AI will be a friend or foe, even as DevOps remains at the forefront when it comes to onboarding new technologies. Our DevOps communities have been enabling and fostering the collaborative cultural shift and evolving roles and responsibilities of software professionals. In this article, we explore the impact of generative AI in key DevOps roles and explain how to prepare ourselves for the shift.

Let us start by highlighting the basic roles and their broad areas of responsibility so we begin with a common vocabulary.

Overview of DevOps Roles

DevOps roles are evolving quickly, but PagerDuty outlined six main DevOps roles for modern organizations.
1. The DevOps Evangelist: Responsible for delivering changes, collaborating with dev and ops teams and working toward optimizing flow, enabling feedback and fostering experimentation.
2. The Code Release Manager: Responsible for creating cadence between delivery and deployment of code. This role often requires deep technical expertise to run and maintain the releases.
3. The Automation Architect: Responsible for designing and implementing automated pipelines and reducing manual toil.
4. The Experience Assurance Expert: Responsible for user experience, enabling feedback loops and ensuring superior experience, assessing feature performance and improving end-product usability.
5. The Software Developer & Tester: Developers and testers are responsible for the design and development of the software product. Primary responsibilities are writing code, testing and fixing bugs.
6. The Security and Compliance Engineer: Responsible for the overall security and compliance of the software product. Collaborating with all the above roles to ensure standards and regulations are implemented.
All roles above are expected to work across the value chain to maximize their impact. Note that there are many other roles, including leadership and executive roles, which we have not covered in this article. Next, let’s assess the impact of generative AI on some of these roles.

Adding Generative AI to Every Role

Before diving into the business value of adding generative AI to job roles, let’s build a common vocabulary, starting with defining generative AI. According to Gartner, generative AI can learn from existing artifacts to generate new, realistic artifacts (at scale) that reflect the characteristics of the training data but don’t repeat it. It can produce novel content such as images, video, music, speech, text, software code and product designs.

Automated code generation: AI-augmented code generation tools serve as companions for developers and testers. As an example, GitHub Copilot, developed by GitHub in collaboration with OpenAI, primarily offers predictive code generation and supports multiple languages. Tabnine offers deep learning capabilities and supports around 20 programming languages.

As these and other tools evolve, software developers’ time may be freed up to focus on higher-level, more creative, and innovative thinking and business value-driven development. Some pundits say these code generation tools may be a stopgap in the race to hire developers. It may also result in generative AI-based moderators, a new job role entirely. These moderators would help with exporting or reusing existing code rather than trying to reinvent it.

AI-based release management: Generative AI can also help with release management; from a simplified approach for generating release notes to boosting the productivity of the feature release to setting a precise release cadence, generative AI-based tools are coming to our rescue. For example, Digital.ai introduced predictive decision-making, threat insights and many more features to orchestrate releases. Another example is K8sGPT, a tool for scanning your Kubernetes clusters, diagnosing and triaging issues in simple English. More integration of generative AI is on the way for DevOps tools. Release managers’ roles are at a pivotal point and will evolve with time and new skills to manage real-time releases, integration with several channels of special edition software, classic upgrades and core software releases. They will need to address different customer segments with managed subscription-based, open source versions and take a human-centric approach to software releases.

Security and compliance engineers: DevSecOps firm Jfrog is integrating generative AI capabilities into its Xray and Artifactory solutions. It will automatically detect security flaws, license violations and more early and often. Snyk surveyed more than 400 IT professionals and found the use of artificial intelligence (AI) to write code is creating a security paradox. Generative AI also can be used to simulate malware attacks and identify vulnerabilities. However, this is a double-edged sword as it also can be used to enhance the portfolio of bad actors. That means that security and compliance engineers must increase their focus on regulatory, assessment, audits and validation of generative AI outcomes. Specialist knowledge, decision-making, and ethical evaluation are needed for a strategic approach toward next-generation cybersecurity capabilities.

The automation architect: An automation architect is responsible for decomposing complex tasks and automating them through reusable code. These tasks become more sustainable, with less rework and delays. As an automation architect, generative AI is another tool in the toolset that helps improve efficiency and productivity. Generative AI can help automation architects to scale and allow self-service options for DevOps teams and automate redundant tasks.

Generative AI’s Impact on Key Job Roles

As discussed in the previous section, there are many jobs that will be transformed by generative AI’s influence. Many companies will view generative AI-enhanced tools as a pot of gold where most of the work that requires building templates, infrastructure provisioning, log analysis, reuse or refactoring code can be done with the help of generative AI-enabled tools.
- Short-term skills gap: Software-related skills gaps are everywhere, and organizations need to go beyond reskilling to solve the problem. The role of leaders is key for successfully transitioning into the new era of workforce modeling. DevOps professionals should also look at strategies to improve our productivity by investing in capabilities and tools to fill the skills gap.
- Mid-term talent augmentation: Role disruption is most likely to happen due to generative AI; research and a more structured approach are needed to address the upcoming change. DevOps teams must focus on communication patterns, onboarding of human-machine collaborative models and ensuring performance of the new work models. Copyright, intellectual property (IP), licensing and the use of open source technology will be essential ingredients of mid-term talent augmentation with generative AI.
- Long-term, flexible workforce: Widespread and massive onboarding of generative AI in the long term will require a pragmatic approach to flexible workforce modeling. Task-based frameworks will emerge to guide the flexible orchestration of workforce, roles or even jobs or tasks. Task-based frameworks may mature into capability-driven roles that evolve into a flexible workforce where humans and machines together will deliver the desired business outcomes.
With rapid advancement in generative AI technology, companies will be forced to design new ways of working to keep pace with these changing dynamics. DevOps professionals should see this as an opportunity to get ahead of the curve, assess how generative AI technologies can boost productivity and enhance it with new tools and capabilities.
February 26, 2024