devopsy.kinsta.cloud

Tag: microservices architecture

When the Message Broker Becomes the Bottleneck: Scaling Push and SMS With MongoDB Polling
A back-end team extracted notification delivery from a large Ruby monolith into a centralized Go service, only to discover that their shared RabbitMQ cluster could not reliably support the new workload. Here’s how they solved the problem using MongoDB as a polling queue and the trade-offs that came with it.

The Scaling Problem Inside the Monolith

Back in 2019, the engineering team at Wheely, a popular ride-hailing platform, had a coordination problem that emerged with the product. User-facing notifications such as push alerts, SMS confirmations and status updates were scattered around the large Ruby monolith. Various parts of the codebase made direct attempts to send messages without a shared logic, unified retry strategy and consistent way to track delivery outcomes. As the system continued to grow, new back-end microservices were introduced, with each one requiring its own path to the user’s device.

Push campaigns made things worse. Sending thousands of notifications at once exposed limits that routine traffic had never revealed. The monolith wasn’t designed as a notification delivery system, and scaling it that way implied additional risks to the core business logic already implemented there.

The team needed a dedicated, centralized service that could serve as a single point of entry for all outbound communications, both for the monolith and future microservices alike.

Why the Existing Message Broker Wasn’t Enough

The natural architecture for this kind of service involves a message broker. The monolith and microservices publish notification events, workers consume them and delivery happens asynchronously. The team’s infrastructure already had RabbitMQ in place, so the initial design assumed it would handle this workload.

In practice, the shared RabbitMQ cluster proved unreliable under the notification service’s workload. Unstable behavior of the cluster during large push campaigns caused several delivery failures, which were difficult to diagnose, hard to retry correctly and disruptive to other services that used the same broker.

RabbitMQ Internals

To understand why RabbitMQ turned out to be a bottleneck, it’s worth taking a quick look at its internal architecture.

RabbitMQ is built on top of the Erlang virtual machine: BEAM, where almost every broker component runs as a separate Erlang process. Queues, connections, channels and consumers are independent lightweight processes, communicating with each other via messaging.

This model provides strong fault tolerance and scalability in various scenarios. However, under a very heavy load, the overhead of managing these processes can become noticeable. Moreover, RabbitMQ is also sensitive to the type of load it receives.
- The first factor is the number of queues. Each queue is a separate Erlang process with its own internal data structures. As the number of queues increases, more memory is used, the Erlang VM scheduler starts working harder and there is an additional operational overhead from operating the broker.
- Second, the length of the queue matters. RabbitMQ was designed as a system to process continuous message flow, not to store very large backlogs for long periods. If producers write faster than consumers can read, the queue starts growing quickly. It results in higher memory usage, puts additional pressure on the disk for persistent messages and significantly increases delivery delay.
- Finally, RabbitMQ clusters are sensitive to network conditions. Replication between nodes, synchronization of queues after recovery and network latency can all significantly impact the system’s performance and cluster recovery time after failures.
These characteristics are not flaws in RabbitMQ; they simply define the type of loads it is best suited to handle. When designing a high-load system, it is important to consider not only the broker’s capabilities but also the workload profile it will need to support.

Why RabbitMQ Was No Longer the Right Fit

RabbitMQ had handled its role well for a long period. The problem was not with the broker itself, but the change in workload that took place over time.
- First, the system was increasingly dealing with large-scale push and SMS campaigns. A single campaign generated tens of thousands of individual messages, all arriving in RabbitMQ practically at the same time. This created sharp load spikes, made queues grow and increased delivery delays.
- Second, some consumers were still running inside the Ruby monolith. Compared to the newer Go services, they were slower at processing messages, allowing queues to accumulate much faster than they could be drained. Simply increasing the number of consumers wasn’t a practical option. Performance of the monolith had already become a limiting factor, and scaling it would have been complicated due to the current system’s architecture.
- Finally, RabbitMQ served as a shared infrastructure component for various services. Every new communication-related feature added load to the same cluster, making it an increasingly critical point of failure.
Extracting a Centralized Go Notification Service

The new service was implemented as a single, stateless API responsible for accepting send requests and handling everything else: Routing to the target provider, retrying failed deliveries and recording the outcome. The monolith and other microservices sent requests to a single endpoint without having to manage the delivery process themselves.

Go was chosen for the implementation of the new service. It was containerized and deployed using Docker on the existing AWS infrastructure, consistent with the rest of the microservices on the platform.

On the mobile side, the service integrates with the Apple Push Notification service (APNs) and Firebase Cloud Messaging (FCM). SMS delivery was handled via integration with one or more third-party providers.

Using MongoDB as a Polling Queue

Since it was not possible to rely on any broker, it was necessary to find some alternative way of buffering delivery tasks and processing them. The alternative chosen by the team was the usage of MongoDB as a polling queue. MongoDB was one of the tools used in the current stack.

When a notification request arrives, the service writes a document to a dedicated MongoDB collection. Each document represented a single delivery task and contained all the information needed to be processed independently: The recipient, the message payload, the channel, the current status and metadata to retry tracking.

A simplified document life cycle looked like this:

pending → processing → delivered

↘ retry → (back to pending, up to N attempts)

↘ failed (dead-letter)

Workers polled the collection at short intervals, querying for documents in the pending state. To process two workers trying to process the same task, MongoDB’s atomic findOneAndUpdate operation was used to transition a document from pending to processing in a single step. Only one worker succeeded in claiming any given record.

Retry policy was embedded in the document itself. Each failed attempt incremented a counter, as well as updated a retry_after timestamp using an exponential backoff strategy. Workers excluded documents whose retry_after time had not yet passed. Documents that reached the maximum retry count were moved to the failed state rather than being tried to be processed again.

Idempotency was maintained by creating a stable identifier for each delivery task when it was created. In case the same request arrived twice due to the retry on the caller’s side, the service could detect the duplicate before writing a second document.

Processed and failed records were either archived into another collection or dropped after the retention window.

Index design was critical. The collection required compound indexes covering status, channel and scheduling fields for efficient polling queries. Without the right indexes, polling at scale would have caused unacceptable read load on MongoDB.

Optimizing MongoDB for a Write-Heavy Workload

Using MongoDB as a persistent queue required careful data model design.
- First, we separated the queue into a write-heavy collection, fully isolated from the rest of the application data. This way, it was possible to optimize it independently by choosing its own indexes, configuring a dedicated retention policy and ensuring that intensive writes did not affect the rest of the system.
- At the same time, we attempted to implement an append-heavy data model. Messages were created once and then updated just a few times while moving through the delivery pipeline before being either archived or deleted. This life cycle aligned well with MongoDB’s strengths and helped maintain stable write performance.
- Queue documents were intentionally kept compact. They included only information needed for routing, delivery and status tracking. This reduced the amount of data transferred, lowered disk and replication load and allowed more of the working set to fit in memory.
- Indexes required particular attention. We deliberately limited them to those needed for efficient polling and status-based lookups. Any additional index increases the write cost, as MongoDB must update it on every insert or update. For write-heavy workloads, keeping the number of indexes minimal becomes more important than maximizing read performance.
- Messages were polled in batches instead of one at a time. During each poll cycle, the worker would claim only a certain number of tasks, which significantly reduces the number of database operations, thereby making the load more predictable in case of many incoming messages.
- Finally, to prevent the collection from growing indefinitely, we either periodically archived processed messages or automatically removed them using TTL indexes. This kept both the collection and its indexes relatively small, improving write speed and the efficiency of polling queries.
It was the combination of these decisions that made MongoDB an efficient and predictable persistent queue for this workload. Observability relied on tooling already in use at Wheely: The ELK stack for log aggregation and Datadog for metrics and alerting.

The Trade-Offs We Accepted

MongoDB was not built to be a message broker, and using it as one comes at a cost.

During each poll cycle, database read operations were generated regardless of whether the queue was empty. At scale, it added measurable load to MongoDB. This load was then associated directly with notifications throughput instead of being isolated in a dedicated broker. Despite a careful index design and a sensible polling interval, it was still possible to reduce but not eliminate this overhead.

Polling introduces inherent latency. A message written to the queue would not be processed until the next poll cycle. It is a reasonable decision for most notification use cases, but it was a deliberate compromise compared to the near-immediate delivery that one can expect from a properly functioning broker.

The team also had to implement, test and maintain functionality that mature message brokers typically provide out of the box: Atomic claiming, retry scheduling, dead-letter handling, duplicate suppression and observability. Each of these capabilities added complexity to the service itself.

There was also an organizational risk. Solutions described as temporary often become permanent. Teams adopting this pattern should clearly define under what conditions that would justify migrating to a dedicated broker, and they should build sufficient observability to detect when those conditions are met.

A dedicated message broker continues to be the right choice in the case of a stable infrastructure, when the team has the operational expertise to operate it and the delivery latency requirements are strict. This architectural approach was chosen based on specific sets of constraints, not as a general recommendation.

Operational Results and Lessons

After the migration, the service handled notification volumes several times higher than the previous peak without delivery failures caused by broker instability. The service achieved a much better delivery success rate and now serves as the shared communication layer for multiple back-end services across the platform.

Three lessons worth carrying forward:
- First, architecture must reflect the maturity of the infrastructure it runs on. A theoretically correct design built on an operationally unreliable dependency can perform worse than a pragmatic design built on infrastructure the team already operates well.
- Second, database polling can be a viable delivery mechanism, but it requires a thorough implementation. Idempotence, atomic claiming, backoff, dead-letter handling and observability are not optional additions; but are the key to making the pattern safe. Skipping even one of them leads to failure modes that are difficult to diagnose under load.
- Third, extracting a shared capability from a monolith is an operational change, not just a code refactoring exercise. The migration required a careful cutover strategy to avoid interrupting live delivery. The organizational and operational work was at least as significant as the engineering work.
Architecture Diagram

[Monolith/Microservices]

│ send request

▼

[Notification Service API]

│ write task

▼

[MongoDB — pending queue]

│ poll + claim (findOneAndUpdate)

▼

[Worker pool]

│ │

▼ ▼

[APNs/FCM] [SMS Provider(s)]

│ update status

▼

[MongoDB — delivered/retry/failed]
July 24, 2026
Why Your AI Agent Strategy is Failing (and How to Fix It): The Microservices Playbook for AI Agents
Despite billions in AI investment and countless vendor promises, most enterprises are still treating AI agents like glorified copilots rather than autonomous systems. After working with numerous enterprise customers implementing AI agents across various industries, a pattern has emerged: The companies finding real success aren’t the ones building the biggest, most ambitious agents — they’re the ones treating agents as microservices.

As of October 2025, this pattern has only intensified. The sobering statistics tell the story: A full 42% of companies are abandoning their AI initiatives in 2025, up from just 17% in 2024, while MIT reports that 95% of generative AI pilots fail to achieve rapid revenue acceleration. Yet, simultaneously, McKinsey has found that 80% of organizations with formal AI strategies succeed, compared with only 37% without. The difference between winners and losers? It all comes down to their architectural approach.

The Monolithic Agent Trap

Walk into any Fortune 500 company today, and you’ll likely find teams attempting to build what industry experts now call ‘monster agents’ (i.e., single AI systems designed to handle multiple complex workflows across different departments). The logic seems sound: Why deploy 10 specialized agents when one super-agent can do everything?

The reality is far messier. Every enterprise customer I’ve spoken with who has attempted the monolithic approach has run into the same problems — complexity explosion becomes unmanageable, debugging turns into a nightmare and outputs become increasingly unpredictable as the agent tries to juggle too many responsibilities.

This isn’t just anecdotal. A 2025 CIO analysis explicitly warns: “Most companies get this wrong…they try building monolithic agents — jacks-of-all-trades. But they often become haunted by hallucinations — the stronger they are, the harder they fall.” Recent research has also documented the shift from monolithic entities to compound systems with factored agent architectures that separate high-level planning from lower-level tool execution.

The software architecture parallel is striking. We learned decades ago that monolithic applications don’t scale well. The same principles apply to AI agents, but somehow, in our rush to implement AI, we’ve forgotten these hard-won lessons in an attempt to be efficient or exercise centralized control.

The Advantages of Taking a Microservices Lens When Implementing AI Agents

The companies seeing real ROI from AI agents are taking a different approach. They’re building specialized agents focused on very specific tasks with constrained toolsets. Instead of creating one agent to handle customer service, inventory management and financial reporting, they’re deploying three separate agents, each optimized for its specific domain.

This microservices approach to AI agents offers several key advantages:
- More Predictable Inputs and Outputs: When an agent has a narrow focus, it’s much easier to constrain and predict what you’ll get out based on what you put in. This predictability is essential for building enterprise trust in AI systems.
- Easier Debugging and Testing: If something goes wrong with your customer service agent, you’re not trying to debug a system that also handles inventory and finance. You can isolate, test and fix issues without worrying about cascading effects across unrelated workflows.
- Independent Scaling: Different business functions have different scaling requirements. Your financial reporting agent might need to handle month-end spikes, while your customer service agent faces different demand patterns.
- Loose Coupling: Specialized agents can share information without being tightly integrated. This flexibility becomes crucial as your AI infrastructure evolves and you need to swap out or upgrade individual components.
Anthropic’s research team published guidance emphasizing this approach: Start with simple prompts and single-agent solutions, only adding complexity when measurements prove it necessary. The principle is clear: Specialized beats generalized in production environments.

How Microservices Solve the AI Agent Trust Problem

The microservices approach to AI agents offers another critical advantage that’s often overlooked: It makes trust and transparency truly achievable. Companies that successfully navigate AI agent implementation face data trust and privacy concerns. Despite vendor assurances about data protection, many enterprises remain skeptical about sending sensitive information to LLM providers.

This skepticism isn’t unfounded. Privacy and security concerns remain the primary barrier to LLM adoption, cited by 44% of enterprises as their top concern. The trust challenge creates a fascinating paradox for AI providers: They need greater adoption to improve their models through training, but if customers don’t trust them to safeguard information, adoption remains limited.

The contradiction plays out in enterprise policies in revealing ways. Companies may ban employees from using one LLM directly and then purchase an AI system that uses another’s APIs in the background. This pattern persists but has evolved into more sophisticated governance: Organizations ban the public LLM interface due to lack of control over data usage while permitting another’s enterprise version API with proper governance, BAAs and DPAs.

Rather than AI rejection, this represents strategic risk management — enterprises embrace the technology while maintaining control and compliance. The approach has matured from blanket bans to controlled internal solutions. Amazon developed CodeWhisperer, Commonwealth Bank built CommBank Gen.ai Studio and many companies have implemented enterprise-specific solutions using foundation model APIs with proper governance frameworks.

Building AI Agent Trust Through Transparency

The enterprises finding success with AI agents share a common approach: They prioritize transparency and auditability alongside functionality. They want to see exactly what their agents are doing, why they’re making specific decisions and how to trace problems back to their sources.

This is where the concept of ‘agentic memory’ becomes critical. You need a way to capture not just the inputs and outputs of your AI systems, but also all the intermediate steps and decision points. When something goes wrong, you need to be able to travel back in time and understand exactly what happened.

Auditability features include tamper-evident logs recording all significant operations, tracking changes, entity identity, timestamps and cryptographic linkages. Memory must be auditable under GDPR, HIPAA and CCPA, with users able to inspect and delete stored memories.

I see this need often. Customers want their AI systems to work with rich, contextual data, but they also want full correlation between that data and the actions their agents take. Events provide this context naturally because they tell you what changed, when it changed and specifically why it changed. This event-driven approach gives agents the information they need while maintaining a complete audit trail.

Here’s where the microservices approach proves its worth again. When an agent has a narrow, well-defined scope, these audit trails become manageable and meaningful. You’re not trying to untangle the decision-making of a system juggling multiple unrelated responsibilities. You can trace exactly what your customer service agent did without wading through irrelevant financial calculations or inventory updates.

How to Measure the True ROI of Enterprise AI Investments

When I talk to enterprise customers about their AI investments, the ROI discussion often focuses on cost savings, specifically around reducing headcount or automating manual processes. While these benefits are real, they’re not the primary value drivers I see in successful implementations.

Customers getting the best ROI from AI agents focus on three key areas:
- Faster time to value because they can quickly debug and course-correct their specialized agents. Instead of spending months trying to fix a complex multi-purpose agent, they can identify and resolve issues in days or weeks.
- Enhanced decision-making due to better data inputs and full visibility into agent reasoning. Their agents are providing insights that were not previously available.
- Greater vendor independence by avoiding platform lock-in and building on vendor-agnostic infrastructure. This allows customers to maintain flexibility as the AI landscape evolves, a crucial aspect for long-term strategic positioning.
A Pragmatic Implementation Framework

Based on what I’ve seen work across multiple enterprise deployments, here’s the framework I recommend:
- Start Small and Specific: Pick one well-defined use case where you can clearly measure success. Don’t try to solve multiple business problems with your first AI agent implementation. High performers target core business areas where value is generated.
- Build Infrastructure First: Before you deploy your first agent, make sure you have the infrastructure for visibility, audit trails and debugging. These capabilities are much harder to add after the fact. Implement observability from day one.
- Prioritize Context Over Complexity: Invest in systems that provide rich, contextual data to your agents rather than trying to make your agents smarter through more complex reasoning. Allocate 50–70% of the timeline and budget to data preparation.
- Plan for Independence: Avoid vendor lock-in from the beginning. Build on architectures that allow you to swap out components as better options become available. Design for composability using multiple models strategically based on the use case.
- Test Extensively: Small, specialized agents are much easier to test thoroughly. Take advantage of this by building comprehensive test suites before moving to production.
- Secure Executive Sponsorship: McKinsey found CEO oversight of AI governance most correlated with higher EBIT impact, sharing that organizations with a formal AI strategy achieve 80% success.
- Redesign Workflows: This has the biggest effect on EBIT impact. Don’t just automate existing processes — reimagine them for AI-native operation.
We are at an inflection point for enterprise AI adoption. The technology has moved decisively from experimentation to production deployment. The consolidation we’re seeing (i.e., Microsoft retiring AutoGen in favor of a unified Agent Framework, LangGraph reaching 1.0, standardization through MCP and A2A protocols) signals genuine maturation.

The companies that will succeed aren’t necessarily the ones with the biggest AI budgets or the most sophisticated technology. They’re the ones taking a systematic, pragmatic approach to implementation by treating AI agents as specialized tools rather than magic solutions. They understand that organizational challenges now exceed technical barriers.

The principles we’ve learned from decades of software architecture apply directly to AI agents. Organizations that embrace these principles, establish formal AI strategies with CEO oversight, fundamentally redesign workflows, and track well-defined KPIs will establish sustainable competitive advantages.

The hype around AI agents will eventually fade, but the underlying technology will become as vital to enterprise operations as databases and web servers are today. The question becomes whether you’ll implement AI agents thoughtfully or get caught up in the current wave of unrealistic expectations and disappointing results.
December 16, 2025
Technical Leadership in Modern Software Infrastructure: Architecting for Performance and Scale

Between drivers, riders, restaurants and delivery services, Uber produces petabytes of data each day. In order for the company to succeed, real-time data processing stands as a vital requirement for functions that include customer incentives, fraud detection and machine learning model predictions.

The engineering leadership team set out to design a system that would fulfill their specific as they grew to more territories and category offerings. The team built a complete real-time data infrastructure using open-source technologies, which enables real-time analytics and decision-making throughout the organization.

The system processes enormous data streams in real time, resulting in better user experiences and faster system responses, enabling Uber to scale its services throughout different markets and user groups across the globe.

Employment website Indeed faced different, but no less extensive problems, because of its rapid growth. The company’s expansion into a worldwide platform required a complete transformation of its backend infrastructure to serve millions of users and thousands of employers. Engineers moved their monolithic architecture into a Kubernetes-based system that used a microservices architecture. The transition enabled separate teams to operate independently while speeding up the rapid deployment of changes. According to Indeed’s engineers, service deployment time dropped by 80% while system uptime achieved 99.99%

These two examples offer just a glimpse into the importance of proactive technical leadership when it comes to architecting at scale. Leaders don’t just react, they create the frameworks and guardrails that empower engineers to deliver efficiently at scale.

Technical leadership begins with grasping both overall system understanding and detailed components. Infrastructure exists in a constant state of change. A skilled technical leader recognizes this principle by creating infrastructure that enables teams to move fast through the implementation of CI/CD (continuous integration/continuous delivery) pipelines and fault-tolerant system design, allowing for experimentation without destructive outcomes.

When I was the tech lead at a small, bootstrapped startup, we were under pressure to rapidly deliver proof-of-concept features to validate new business opportunities, but we also needed to manage infrastructure costs carefully. We chose to leverage a serverless architecture using AWS Lambda functions for a new set of services we needed to build quickly. I recommended and led the adoption of serverless functions because they enabled extremely fast deployment cycles, minimized operational overhead, and allowed developers to focus on building functionality rather than maintaining servers. The lightweight, event-driven nature of Lambdas fits our needs well, especially when paired with simple, well-defined APIs for integration. We could deploy new features in hours or days instead of weeks, iterate quickly based on feedback, and scale seamlessly as demand grew – all while keeping costs tightly aligned with usage. This technology choice directly contributed to faster go-to-market timelines for several new product features, helping the business validate ideas faster and ultimately accelerate revenue growth.

The process of designing contemporary software infrastructure requires leadership abilities beyond technical expertise. Strategic leadership is essential because it prevents even top engineering teams from developing inflexible systems that cannot scale.

Good technical leadership requires equal attention to human systems as it does to technical systems. Leaders establish team culture standards, create engineering best practices and make documentation and onboarding essential so the system becomes scalable for both performance and human resources.

For tech leads, balancing short-term needs with long-term scalability is a daily challenge. In one role, I partnered closely with the product team to translate technical issues into real financial and operational impacts, like showing how much engineering time we were losing each week because of certain tech debt hotspots. We made intentional short-term compromises, but stayed disciplined about paying down debt as part of feature development. That approach helped us deliver quickly without piling up scalability risks, and it made the service much more resilient as usage and complexity grew.

Companies that achieve large-scale success – whether they operate as global cloud providers or ride-sharing giants – succeed because their technical leaders make smart architectural and budgetary decisions, empower their teams and anticipate growth before it arrives.

Ultimately, technical leadership is what transforms an average system into an exceptional one. It’s what transforms scattered efforts into a cohesive strategy. And it’s what ensures that infrastructure doesn’t just support the business today, but accelerates its ability to adapt, compete, and ensure the company is prepared to respond to future challenges.

June 26, 2025
How Microservices Guardrails Help Teams Move Faster
The evolution of enterprise software architecture has brought microservices to the forefront of modern development practices. Although a microservices architecture offers agility and scalability, it often introduces new complexity that can become overwhelming if left unchecked.

In speaking with customers about their distributed architectures, I’ve seen complexity emerge in various ways as their distributed systems have evolved over the years. This complexity naturally builds up in homegrown applications – even those with only 20-30 services. Another common challenge I’ve encountered is when organizations acquire systems with microservices-based architectures. Teams often lack clarity on service interactions and dependencies, especially when key employees have departed and technical documentation is missing. This leads to slow deployments and significant troubleshooting difficulties across hundreds of microservices.

The Mounting Challenges of Unmanaged Microservices

Although there’s a clear trend toward microservices adoption, merely implementing microservices doesn’t guarantee speed and agility. Without proper management, microservices can proliferate rapidly, creating numerous challenges for development teams. Individual teams may create unexpected cross-dependencies and redundant services, gradually drifting from the original architectural vision. The rise of AI-powered development tools and the corresponding proliferation of code compounds this challenge.

The Impact of Insufficient Governance

When microservices grow without oversight and proper documentation, often resulting in unneeded dependencies and service duplication, organizations face diminished system performance and reduced development velocity, which can result in increased operational complexity and costs. Teams spend more time debugging complex interactions between services and less time delivering new features. They face challenges such as identifying unnecessary circular dependencies, hard-to-trace multi-hop flows and difficulty visualizing and documenting microservices in real time. For large teams, onboarding new team members becomes increasingly cumbersome as the system’s complexity grows.

The long-term impact of this complexity on not only system health but on engineering velocity and team morale puts organizations at risk of lagging innovation. Without intervention, teams must either increase their time spent on maintenance and fixes or face mounting technical debt.

Creating Effective Guardrails

Well-implemented governance enables organizations to support the architectural lifecycle – a continuous process that ensures a system’s architecture remains aligned with its design and evolving requirements. It allows teams to move quickly while maintaining the overall system’s integrity and resilience. These guardrails should define clear boundaries for service interaction, data access patterns and technological choices while still allowing teams to innovate within their domains. Clear guidance helps teams make more consistent decisions, which leads to better system reliability, easier maintenance and faster development cycles.

By enforcing best practices, organizations can avoid suboptimal design decisions that can impact system efficiency and reliability. Moreover, a well-governed architecture typically requires less manual intervention and troubleshooting, reducing the operational burden on IT teams. Governance also permits systems to more efficiently handle increased loads, allowing businesses to grow without the constant need for modernization.

Best Practices for Implementing Effective Microservices Governance

Successful microservices governance requires a balance between control and flexibility. Organizations should establish clear architectural principles that guide team decisions without overly restricting their ability to solve problems. These principles should include:
- Automated checks for architectural violations.
- Clear guidelines for application boundaries.
- Established patterns for common integration scenarios.
- Defined workflows for cross-team collaboration and service changes.
Fostering a culture of architectural responsibility across teams is also important. Organizations must:
- Establish clear guidelines and standards for architectural decisions.
- Integrate architectural considerations into the development process from the beginning.
- Promote the benefits of continuous architectural improvement.
- Enable teams to automatically visualize architecture in real time as it evolves.
Additionally, adopting smaller, autonomous teams can significantly improve collaboration and accountability in microservices development. Following Amazon’s “two-pizza team” principle, the idea is to keep teams small enough that they can be fed two pizzas. This helps ensure streamlined communication and efficient coordination.

Modern Tools For Architectural Governance

Although enterprise architecture has long been a core practice in most organizations, modern architectural governance relies increasingly on automated tools that can detect and prevent architectural drift before it becomes problematic. When architectural observability—or the ability to analyze software architecture, observe drift and identify issues—is combined with AI and automation capabilities, teams can implement and act on the software architecture life cycle in real time. When complemented with application performance management (APM) tools, architectural observability accelerates incident response and reduces mean time to repair (MTTR) through enhanced system visibility and a contextual view of system flows.

Taming Complexity: Examples From the Field

With the right tools, organizations can work their way back to a stable microservices architecture. For example, a global travel services provider acquired a microservices-based system from a smaller organization. At that point, key employees had left the company and minimal documentation made it difficult to see and understand the interactions across hundreds of microservices in production. In this scenario, architectural mapping and sequence diagrams of production flows would provide the necessary visibility into their architecture to regain clarity and control of the distributed system.

In the same context, ensuring adherence to existing architectural rules of the parent company is important for alignment. By implementing architectural observability and governance, the organization can align its microservices-based system with broader portfolio protocols, improving production efficiency, accelerating deployments and simplifying troubleshooting.

This challenge isn’t unique to a single organization. Even industry leaders face governance obstacles when managing complex microservices architectures at scale. Netflix’s transition from a monolithic architecture to microservices has been widely recognized as a success in enabling scalability and innovation. However, as its system grew, the company faced governance challenges in managing service complexity, ensuring stability and maintaining rapid feature development. A prime example is Netflix’s video processing pipeline, which initially ran on a monolithic system called Reloaded but became difficult to scale and maintain as video formats and streaming demands evolved.

To address these challenges, Netflix rebuilt its video pipeline on a microservices-based platform called Cosmos, introducing clear service boundaries and workflow orchestration. This shift to microservices improved observability, accelerated feature development and increased system flexibility. By implementing structured governance and automation, Netflix transformed its video processing pipeline into a more scalable, efficient and adaptable system, ensuring continued innovation while minimizing operational complexity.

The Path To Sustainable Microservices

Microservices architecture offers benefits in terms of scalability and agility. However, its success is contingent on establishing flexible guardrails through effective governance and documentation. By implementing clear principles, automated checks, proper tooling and accurate documentation, organizations can enable their teams to move quickly while maintaining a cohesive, maintainable system architecture. This balanced approach helps ensure that teams can rapidly develop and deploy applications while maintaining the microservices promise of agility and speed.
May 22, 2025
Best of 2023: Microservices Sucks — Amazon Goes Back to Basics

As we close out 2023, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2023.

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: Amazon Prime Video has ditched its use of microservices-cum-serverless, reverting to a traditional, monolithic architecture. It vastly improved the workload’s cost and scalability. (more…)

December 28, 2023
Saying Goodbye to Legacy Systems
Are you still using a rotary phone, tube television or Rolodex? Probably not. But if you were, you’d still be using newer technology than what many enterprises still rely on every day to drive mission-critical applications that we use every day when shopping, banking or hundreds of other normal tasks.

At least two-thirds of large enterprises are running mainframe-based apps that date back 10 to 20 years and more than 25% of businesses feature applications that are between 20 or 30 years old. Today, businesses are often running and maintaining legacy applications built on traditional databases while simultaneously building microservices-based apps to accelerate new services and innovations. The mantra “If it ain’t broke, don’t fix it” certainly applies here; and while there are many reasons organizations continue to rely on mainframes and legacy systems, there are also many organizations that want to begin modernization initiatives but don’t know where to start.

However, the past few years saw a wave of new digital-first companies in all industries putting pressure on legacy systems-based businesses—from retail to finance to manufacturing to technology. The COVID-19 pandemic accelerated many organizations’ modernization plans and made clear that for those that wanted or needed to modernize to stay competitive, avoiding the problem just wasn’t an option. Companies found that if they weren’t proactive about modernizing, the cost of being reactive when they needed to update and evolve was exorbitant.

Accelerating IT modernization initiatives can be a major challenge: Safely modernizing those legacy apps and mainframe systems and the critical data in them can seem impossible, but it can be done. Companies like Kroger, for example, have already made that jump and there’s a lot to learn from them. Here are three things that can help any enterprise on their journey to say goodbye to mainframes and legacy systems:

You Need a Frictionless Path for Legacy Apps

Many modern solutions are only focused on greenfield applications and helping newer born-in-the-cloud companies and projects; however, the reality is most Fortune 1000 companies have a lot of existing applications they can’t afford to ignore or completely rebuild all at once. Supporting existing applications with standard interfaces like Postgres while delivering the scalability and resiliency to support modern applications is critical to providing a bridge from where companies have been to where they are trying to go. Distributed SQL databases provide a seamless path forward to modernize existing apps while supporting new and current initiatives.

Embrace Automation

It’s time to stop relying on complex code or fragile systems with bolted-on third-party tools. Being able to rely on intelligence and automation in the data layer for distribution, resiliency and availability is critical to minimizing changes and impacts on current applications. Doing so will reduce risk, offload legacy app tasks and ensure systems have the intelligence to work properly. With a distributed SQL database, you can deliver a single logical database that appears to applications like a monolithic, single-server database but with the underlying intelligence to allow it to stretch across hybrid, private and public clouds.

Finding Flexibility and Becoming Future-Proof

To avoid being tied to specific systems or architectures, it’s time to adopt a future-proof mindset. Businesses can achieve this through:
- Multi-cloud/hybrid cloud: With a distributed SQL database, you can deliver a single logical database that looks like a monolithic, single-server database but with the intelligence to stretch across hybrid, private and public clouds—providing the flexibility to be deployed wherever the application is needed or is best served.
- Any hardware: No two data centers look alike and there are a wide variety of hardware needs, from bare metal to VMs and containers. The right modern distributed database provides ultimate flexibility in where the data layer can be installed to match whatever hardware and configurations are already in use.
- Any deployment: Existing organizations have a wide variety of environments and deployment types, from single data centers to two or more private data centers to more and more hybrid environments that span on-premises and public clouds. A modern distributed database provides the freedom to support a wide range of deployment options, including synchronous replication, geo-partitioning and cross-cluster asynchronous replication.
- Open source freedom: While many databases provide some open source components, there are still a few 100% open source solutions committed to the Apache 2.0 license. Users can benefit from the full set of enterprise features and capabilities in these open source solutions as well as benefit from the security and transparency that comes with a 100%-open source solution.
It’s time to update the old adage, “If it ain’t broke, don’t fix it” to “If it ain’t fixed, it could be broke,” with regard to modernizing legacy systems. If you’re considering starting a modernization journey, save time, resources and money now to make upgrades that might later be costly, time-consuming and overwhelming for your internal teams.
June 6, 2022
Best Practices for Modernizing App Modernization

Modernizing legacy systems to embrace a loosely-coupled microservices architecture is the biggest predictor of success for continuous delivery, according to Google Cloud’s 2021 Accelerate State of DevOps report. Studies continue to show that the most elite DevOps performers enjoy more frequent code deployments, faster lead times to deploy, low change failure rates and faster time-to-recovery.

Leading enterprises like Netflix, Tesla and Starbucks proved that a microservices architecture minimizes fine-grained dependencies between services, which can then be independently tested, deployed and scaled. Combined with DevOps best practices, this reduces technical debt and allows for modernization based on a factory model approach that incorporates automated analysis and deep observability.

That said, application modernization to date has been a manual process requiring multi-year projects that suffer a high failure rate. How can we apply modern tools, known DevOps best practices and desired business benefits to improve and accelerate a company’s modernization efforts? In other words, how do we make modernization more modern?

By using automation, AI and data science, building repeatable pipelines and shifting architecture left, architects can get in front of technical debt before it becomes unmanageable. In this article, we’ll explore these practices in more detail.

Intelligently Automate Everything Possible

DevOps methodologies have taken manual steps and applied automation to create repeatable processes that scale. Modernization teams today are in dire need of these methodologies, as they are stuck with manual approaches that don’t scale across an entire application estate. Thus, applying intelligent automation, AI and data science to modernization—and creating a repeatable pipeline for refactoring—is a natural extension of DevOps successes over the last decade.

Continuous Integration, Continuous Deployment, and Continuous Modernization

CI/CD methodologies are a core best practice for DevOps teams and are already familiar patterns that development organizations are using for their application life cycles. But, there is no continuous delivery without continuous modernization. Elite teams continuously monitor technical debt accumulation and address it at every application release cycle, thereby implementing continuous modernization in their application life cycles.

Deep Application Observability

Observability, defined as monitoring, logging and tracing, is a foundation for cloud-native architectures. Modernizing an existing monolith requires deep observability beyond simple static code analysis to include deeper domain-driven behavior analysis and dynamically tracking dependency interactions. This is also the secret to uncovering dead code that is no longer used.

Ensure Architectural Parity

You must understand your infrastructure architecture to inform your application architecture and your modernization options. Lift-and-shift causes headaches for many organizations because the traditional three-tier architectures of monolithic applications have a completely different topology than cloud-native applications, which are designed from scratch to leverage the efficiencies of the cloud.

Whether it’s a hybrid approach (part on-premises, part cloud), public or private cloud, it’s important to ensure that your modernization strategy ensures architectural parity between the target application and its ultimate destination in the cloud. Also, it’s important to understand your modernization options: Rehost, replatform, refactor, rewrite, etc. What can you lift-and-shift and what needs more attention?

To find the answer, incorporating data science-driven analysis, automated service extraction and, more importantly, people, to help you understand the possibilities is critical.

Get Executive Support for Your Modernization Project

This isn’t about simply churning through your backlog; your executive sponsors for a project of this scope need to understand, in business terms, why this is crucial for the organization. There are plenty of metrics showing the financial and opportunity costs of maintaining mission-critical applications that are aging and problematic.

To get executive buy-in for your modernization initiatives, you need to formulate a business case that communicates the cost of maintaining the legacy apps, both in time and resources to maintain, but also the negative impact on innovation, customer acquisition and retention and speed to market.

DevOps practices emerged to improve how IT organizations reliably and efficiently deliver software from development to operations and out to customers. Applying these emerging technologies of automation, pipelines and deep observability to application modernization initiatives will enable teams to accelerate how they deliver application transformation projects. By investing in training application teams in these methods and providing modern tooling to support that training, organizations more reliably and more efficiently move both people and projects forward into the next generation of cloud applications.

March 2, 2022
Step Aside, Monoliths: Meet the Composable Enterprise
A composable enterprise, defined by Gartner as “an organization that delivers business outcomes and adapts to the pace of business change”, relies on the assembly of interchangeable application building blocks. This architectural overhaul has largely been driven by a demand for more configurable application experiences, and the need to evolve existing application portfolios that are often too risky and costly to replace.

New business opportunities require agility from application portfolios; however, many enterprises are still limited in their ability to adapt. Why? They rely on monolithic ERP systems and cumbersome legacy applications with static processes and haphazard structures. A modular setup can enable a business to rearrange as required depending on external or internal factors, such as shifts in consumer attitudes or sudden supply chain disruptions. Organizations are experiencing these shifts now and require a new approach to enterprise applications to be able to adapt.

Bridge the CX Gap with Greater Composability

A composability approach is the best way to capture all the advantages of modern enterprise software. According to a recent Boomi report, by 2023 organizations that have adopted a composable approach will outpace the competition by 80% in the speed of new feature implementation. This shift requires enterprises to rethink how they architect their operations, harness a combination of packaged functions and technologies and successfully deliver seamless moments of service to their customers.

There are three key ways that businesses can make the composability shift:
- Adopting a service mindset
- Scaling the delivery of microservices
- Packaging business capabilities using Application Programming Interfaces (APIs).
1. Designing for Service: Compete on Outcomes, not on Products

A composable enterprise whose portfolio is made up of heterogeneous applications from a palette of best-of-breed solutions allows organizations to address key inflection points throughout every customer, product or service lifecycle.

As more consumers demand continuous value and reliability throughout an asset’s lifetime, businesses must shift to selling outcomes and experiences instead of products to meet these new expectations for quality service. This requires each part of an operation to align, not around immediate sales or revenue, but around delivering a quality Moment of Service™ — the inflection point where everything comes together to create better value and outcomes for customers.

Moving to a servitized model requires a composable stack to deliver services to order. At a systems level, this transformation requires organizations to adapt applications dynamically and deliver positive customer experiences with effective quality management, customer support, and access to complete information about the service offering. Unlike traditional enterprise software, this involves connecting data and applications that have often sat in separate silos. A composable enterprise can provide a service-orientated architecture, that enables businesses to become outcome-based and ready to quickly adapt to future disruptions.

2. Scale Component-based Architecture with Microservices

If applications are built as loosely coupled services, then companies can employ a composable architecture to capitalize on independently deployable modules that are organized around business capabilities. This allows organizations to swap modules in and out, to suit emergent needs and build a well-structured best-fit solution for their unique business.

In contrast to a monolithic architecture, businesses can easily add resources to the most needed microservice rather than having to scale the entire application as demand for an application increases. In practice, this allows businesses to simplify customizable workflows and optimize business processes while leveraging and applying tools such as robotic process automation, artificial intelligence, or the plethora of hyper-automation capabilities available today.

3. Embrace Open-ended APIs to Maximize Data-sharing Capabilities

Packaged business capabilities (PBC) assembled using APIs are the foundation of every composable enterprise. They are used by businesses to secure data across cloud services, business systems and mobile applications. Historically, APIs have been used in monolithic applications to exchange data between the entire application and external applications and services. In composable software, APIs exchange data from individual modules to external applications and within the application — from module to module. This has significant implications for how systems are designed and built.

APIs can provide a controlled and consumable method for connecting and sharing consumer and business data by creating experiences tailored to individual needs. For instance, an API-centric model can secure and manage data access to help businesses with faster decision-making and deliver relevant new services adaptable to market changes. In the future, there will be more automated continuous process improvements as machine learning models recommend, or even proactively make, process changes to improve outcomes for the business and end customer.

Opt-in to the Composability Evolution

To overcome the limitations of monolithic applications, businesses must rethink their approach to enterprise applications — starting with the business architecture and technology stack. A composable software architecture enables organizations to address the internal and external pressures that send shockwaves throughout the value chain.

As a composable enterprise, organizations can re-engineer their businesses to ensure customer touchpoints and stages come together for better moments of service, but companies must be certain that processes are optimized across each of these inflection points to mitigate issues and fuel growth. An IT architecture built on a foundation of composability will be essential to the successful delivery of a software-powered business development strategy that can provide continual value to customers and the business itself.
January 11, 2022
Best of 2021 – Transform Legacy Java Apps to Microservices

As we close out 2021, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the sixteenth in our series of the Best of 2021.

Velocity is one of the major tenets of DevOps, and anything that maintains or improves velocity benefits the overall process. Yet, there are still many speed bumps in the DevOps process that threaten to derail progress and slow DevOps processes to a crawl. One of the biggest speed bumps arises when transforming legacy monolithic applications into something more cloud-friendly—namely, microservices.

Microservices are clearly the future of applications developed for the cloud; research giant IDC predicts that 90% of all new apps deployed by 2022 will feature microservices architectures. IDC also notes the primary benefits of microservices are the improvements to developers’ ability to design, debug, update and leverage third-party code. Although DevOps tends to be about developing internal code, the benefits offered by a microservices architecture remain much the same.

Microservices: Getting There

Moving toward a microservices architecture can be straightforward or complex, depending on the path an organization is taking and where they begin. For example, if the goal is to transform legacy applications into microservices, the process can be complex and tedious. Transformation of legacy applications often means manually disassembling legacy code and breaking it up into standalone code elements that will run as microservices. That greatly complicates things for the DevOps pipeline, where fast iteration of existing code is the norm, not the exception.

However, there are tools coming to market that can ease the process. One such tool is vFunction, which recently came out of stealth with the mission to transform monolithic Java apps into microservices.

vFunction has an interesting approach to that process. The vFunction platform is deployed locally, where the vFunction Java virtual machine (JVM) agent examines legacy applications and gathers data which is then collected on the vFunction server. From there, software architects can further investigate the collected information about the legacy applications, and start designing a microservices architecture to modernize the codebase. “Once developers are satisfied with the design, vFunction generates a service specification file, which contains all the information required to create the microservice,” said vFunction CEO and co-founder Moti Rafalin. “That file is used by our automation engine, which scans the original code and copies the relevant artifacts to create the new service.”

Rafalin said that vFunction leverages supervised learning techniques, graph theory and clustering algorithms to identify dead code and code anomalies that could prevent a clean breakdown, or “decomposition,” of the legacy application. Ultimately, vFunction has the potential to automate many of the manual steps that DevOps engineers used to perform to break monolithic applications into microservices. According to Rafalin, vFunction makes the process as much as 15 times faster and can achieve somewhere between 80% to 95% automation. Currently, vFunction is focused solely on Java, which it deems the biggest opportunity today. However, the company has plans to support applications written in other languages in the future.

Beyond newcomer vFunction, there are other ways to deal with moving legacy Java applications into the cloud. One example is Anthos, which is a managed application platform that extends Google Cloud services into DevOps environments. Another is IBM’s Mono2Micro, which is an AI-based, semi-automated toolset to assist with refactoring monolithic Java applications.

For those looking to build microservices-based applications from scratch, numerous tools exist. Most interesting are the development environments that use rapid application development (RAD) techniques and leverage low-code models. There are dozens of RAD platforms available; prime examples being WaveMaker , OutSystems and AgilePoint.

In most cases, those moving legacy applications to a microservices model will usually select numerous tools, based on the particular use case. For some, it may be easier to rewrite smaller legacy applications; for others, refactoring may be the only way to capture the intellectual property in legacy applications. Either way, carefully selecting the appropriate tools goes a long way towards maintaining velocity in the DevOps realm. Maintaining that velocity proves critical for building cloud-friendly applications on budget and on time.

“Unless one modernizes those monolithic applications to be cloud-native, they end up requiring very large machines and often end up paying more to the cloud providers,” Rafalin said. “Organizations now understand that, if they want the true benefits of the cloud, they need to modernize these apps. And this starts with adopting an architecture that includes microservices, APIs, and modern design principles.”

The lesson is to choose wisely, and select toolsets that support scale as well as growth, and that can support the requirements of microservices.

December 27, 2021
Driving E-Commerce Transformation with Microservices

It’s no surprise that COVID-19 has accelerated digital transformation, but just how fast these changes have occurred is incredible. According to recent research, companies have accelerated their share of digital or digitally enabled products by up to seven years.

These changes are impacting every sector in every region around the world, but they are being felt acutely by companies that sell their solutions online—which I imagine includes your company. To thrive in the new normal of digital commerce, businesses need to be supported by a fast, flexible technology architecture that can adapt to new products and go-to-market strategies and can be spun up on the fly. The best—and really, the only—way to do that is with a microservices architecture.

Why Monolithic Architectures are Uniquely Bad for Commerce

While monolithic architectures can be simple to develop, test, and deploy, their limitations, especially in the age of digital, quickly become apparent. Software becomes too large and complex to make changes quickly, or correctly. Bugs can be a disaster, and monolithic architectures are often a barrier to adopting new technologies since any change requires significant time and monetary investment to update the entire application.

These limitations are even more glaring in the age of subscriptions. In today’s market, if you are selling products or services online, you are likely offering subscriptions. In fact, 95% of companies offer at least one subscription-based product. For subscription commerce businesses, the need for microservices architecture is even more critical. A recurring revenue strategy requires frequent adjustments and easy integration with other systems, requirements that a monolithic architecture often can’t accommodate.

Scale Is the Critical Factor

One of the greatest benefits of a microservices architecture is the ability to scale quickly. Monolithic architectures are much more rigid because everything has to be scaled at the same time. In contrast, a microservices architecture allows organizations to scale individual components of an application to meet niche use cases. Moreover, because microservices give you the freedom to update individual components, organizations can make changes without any interruption to broader systems. In other words, with a microservices architecture, developers are able to upgrade the airplane while it’s in flight.

There’s internal scale, but external scale through ecosystems is also an important consideration. Recent research found that 56% of executives believe engaging in ecosystems is the most effective way to access new markets, while 55% believe that the new capabilities gained from partnering are essential to building new capabilities. Ecosystem participation—whether as an operator or a participant—requires timely integration and development, a reality that monolithic architectures just cannot support.

Microservices as a Recruitment Tool

Innovation and digital transformation aren’t just business needs. They can also be one of the most powerful recruiting tools available to draw in the best talent. The job market is rife with opportunity for prospective employees—especially now, during the Great Resignation, where a significant number of employees are looking to change jobs—and businesses need to embrace every advantage they can to attract top talent.

Legacy technology can have its place, but it can also be a red flag for job candidates. Most developers want to join an organization that values innovation and gives them the freedom to build unique products and solve meaningful business problems, which microservices development offers. A microservices architecture can also make onboarding new talent easier, since developers aren’t obligated to learn every facet of the architecture. It’s a win for new employees and existing organizations alike.

Driving Transformation Through Microservices

The needs of any given business are always changing, but digital and subscription commerce businesses have proven to be extraordinarily dynamic. Monolithic architectures of the past simply aren’t equipped to power digital transformation or support commerce businesses when they emerge on the other side. Microservices may come in a smaller “package,” but they will deliver much bigger benefits to your business in the long run.

September 9, 2021