As we close out 2023, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2023.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Amazon Prime Video has ditched its use of microservices-cum-serverless, reverting to a traditional, monolithic architecture. It vastly improved the workload’s cost and scalability. (more…)
Is the DevOps world slowly backing away from microservices and to a monolithic renaissance? After years of proselytization about the benefits (Flexibility! Resilience! Support faster release speeds!) of microservices and serverless architecture, there are signs of a potential backlash against the singularly fragmented microservices architectural approach.
The argument in favor of rolling back to monolithic architectures feels similar to the cloud repatriation debate. Just as with a lot of cloud-first transformations—often in lockstep with them, actually—many organizations moved headlong toward microservices without a case-by-case business or technical rationalization for the change. As the shiny newness of microservices has dulled under that approach’s daily exposure to real-world deployment and business challenges, the monolith isn’t looking so bad in a lot of situations.
The monolithic versus microservices debate boiled over the last few weeks in the wake of the publication of a case study earlier this month from the Amazon Prime Video team. The analysis from Marcin Kolny, senior software development engineer for Prime Video, reported that his team was experiencing scaling bottlenecks in its audio/video quality inspection tooling, which was architected as a distributed system using serverless components. By rearchitecting the tool as a monolith, they were not only able to increase scaling capabilities but drastically reduced the infrastructure costs around that tool.
“Moving our service to a monolith reduced our infrastructure cost by over 90%,” Kolny wrote.
The announcement elicited a fair degree of hyperbole on the topic, but it also spurred a very organic airing of the pros and cons of monolithic, microservices and serverless architectures.
The following are 10 different perspectives spurred on by this online debate from a range of CTO and engineering experts. Some of them are diametrically opposed to one another, many are overlapping in their opinions. But as a body, they point to the general (and very pragmatic) consensus that there simply will never be a one-size-fits-all approach to software architectures.
“It’s clear that in practice, microservices pose perhaps the biggest siren song for needlessly complicating your system. And serverless only makes it worse. Replacing method calls and module separations with network invocations and service partitioning within a single, coherent team and application is madness in almost all cases.”
—David Heinemeier Hansson, creator of Ruby on Rails and CTO at 37signals (maker of Basecamp and HEY)
“Application architectures are neither good nor bad, nor are they appropriate for every use case. Organizations need to step back and consider carefully what it is they’re trying to achieve and which architecture might best serve that need instead of choosing the most “modern” architecture because, well, it’s in vogue.”
—Lori MacVittie, distinguished engineer in office of CTO at F5Networks
“In the ongoing debate between monolithic and serverless architectures, it is important to acknowledge that both approaches have their merits and considerations. Monolithic architectures have been prevalent in large human-centric organizations due to their management advantages and historical use cases. However, as the world continues its rapid progression towards a more AI-driven future, the landscape is shifting towards architectures that embrace fluidity, autonomy, and distribution.”
— Reuven Cohen, veteran CTO and entrepreneur, mentor for TechStars
“Over the last 15 years, everyone has wanted to move to serverless microservices-based horizontally-scaled architecture – but it is not ideal for every application’s specific needs. As with any choice in life and in business, there are always tradeoffs; you must evaluate and choose the appropriate architecture based on your business and technology needs. Another important dimension to this is workforce skills for application architecture.
—Hemang Davé, client technical leader and CTO at Kyndryl
“The problem is that they called this refactoring a microservice to monolith transition when it’s clearly a microservice refactoring step, and it is exactly what I recommend people do in my talks about serverless first. I don’t advocate “serverless only”, and I recommended that if you need sustained high traffic, low latency and higher efficiency, then you should re-implement your rapid prototype as a continuously running autoscaled container as part of a larger serverless event-driven architecture, which is what they did. If you built it as a microservice to start with, it would probably take longer (especially as you have to make lots of decisions about how to build and run it) and be less able to iterate as you figure out exactly what you are trying to build.”
—Adrian Cockcroft, longtime DevOps advocate and veteran architect with stints at eBay, Netflix, Amazon, and Battery Ventures, among others
“Microservices, while duplicative and inefficient from a compute standpoint, allow you to release stuff really quickly, which is a tradeoff that most organizations at scale are willing to make.
Microservices exist to reduce communication pathways—not to solve technical scaling challenges. At a certain point, teams stop functioning due to the communication overhead. A seven-person team has 21 communication pathways, whereas a 100-person team has 4,950. I work with large retailers selling billions of dollars a year online, and putting everything into a single monolith just doesn’t work. Could the entirety of Amazon.com be deployed as a single monolith? No way.”
—Kelly Goetsch, chief strategy officer at commercetools
“Speed to market and speed to develop does not always equate to long-term scalability and maintainability. You must actively balance your investments across these two critical pillars to build a viable product. Active rebalancing and reconsideration of the stack and technology mix for your FinOps and BizOps context is both art and science.
Cloud-native distributed systems paradigms are here to stay. The Prime Video folks have mistakenly labeled a properly factored, highly coupled data-intensive processing step a monolith—it’s at best a rocky outcropping in their distributed microservices forest. “
—Ajish George, managing director of cyber data & analytics for State Street
The Shiny Thing is Not Always the Best Thing
“The news that AWS discovered that serverless and microservices aren’t for everything is surprising. Not because it didn’t work out for that use case; but because the wrong use cases have been so well-known for the serverless/microservices pattern and should have been obvious to Prime Video. It’s worth remembering: That shiny new thing isn’t necessarily a better way of trapping mice. Right tool for the right job.”
—Doug Vanderweide, director of cloud architecture for Avanade
Microservices are all the rage. A Forrester study found that 76% of enterprises were rearchitecting applications for microservices.
At the same time, microservices are definitely not a silver bullet. Among those already using microservices in production, one study found that 59% said each microservice added increased operational challenges such as data management. Even more concerning, the same report revealed that when asked to compare the difficulty of troubleshooting different environments, 73% said microservices were more difficult, and only 21% said they were easier than monoliths.
But how did we end up like this? According to Matthew Sinclair, VP of engineering at BCG Digital Ventures, it is because that, as technologists, “we want to use the most modern tech, and sometimes we even want to do it with tech we’ve got no experience in. We see this with engineers, not because it’s the right thing to do but because they want to learn something new.”
Sinclair, who was speaking as a part of an Indorse Engineering Leaders Online Panel, went on to point out how the problem becomes self-perpetuating: Customers hear about new tech—in this instance microservices—that will fix the problems they have with monoliths. They rave about microservices to software engineers, who also get excited about it and then go off and start adding microservices in software engineering projects.
The problem is, there is just as much chance of getting into a tangle with microservices as there is with monoliths. That’s because people often start breaking things up into microservices too early—they’ve heard it will be a good idea and they’ve seen some of the benefits, so they try to apply it to everything. In doing so, they create what some might call a “micromess.”
So, how do you avoid ending up in a micromess?
You need to go back to the fundamentals. One way to look at it is understanding that microservices are distributed systems, something many people will have experience with. They should also be familiar with what another panelist, Oswaldo Hernandez Perez, engineering manager at Improbable, called the first law of distribution: “If you don’t need to do it, don’t do it.”
So that means focusing on why you are building what you are building. What are you trying to achieve? This is a fundamental question that’s applicable to businesses of all sizes. What problem are you trying to solve, and how will your solution remove friction from its users’ lives?
That’s what people care about. Even if you’re developing a niche app for a highly technical audience, they are unlikely to care too much about how it got to them, only that it did and it is fixing a problem for them.
If the only way to achieve that is with microservices, then yes, you should definitely use them. If there is an alternative, then consider using that. Do not simply start breaking everything up into microservices just because that is what everyone is currently talking about.
Ultimately, microservices are an architectural pattern to reduce complexity. It does this, but it also adds complexity elsewhere. If used in isolation, then you’ll fix your complexity in one dimension and have it proliferate elsewhere. So, it is vital that you use the many other tools out there to minimize complexity across the entire organization.
It goes back to the question: What problem are you trying to fix? The majority of businesses can do that with boring tech. Granted, the likes of Amazon and Google, which are cloud-native to their core, can use microservices, but even large enterprises should be asking themselves if it’s really the right choice for them.
Established businesses are likely to have a significant legacy codebase, which needs integrating with. Again, microservices might not be appropriate in this situation—it might be worth saving them for cloud-native projects, and use boring tech to resolve the challenges within legacy IT.
Time and time again, we come back to the fundamental question: What are you trying to fix with this product and/or technology? Not how, but what? What friction will you banish from the user’s life if they engage with your product? Only once you answer that question should you consider whether microservices is for you. Chances are, you may find you don’t need them by that point.
As opposed to a traditional monolith, microservices architecture can theoretically bring many benefits. Microservices decouple software elements, enable reusable components and allow independent development cycles. However, in practice, microservices are prone to many issues.
For example, teams accustomed to a traditional monolithic build process may find it difficult to reshape their established practices. Furthermore, as the number of microservices increases into the hundreds, managing separate CI/CD pipelines for each service becomes untenable, especially as DevOps components aren’t always optimized to fit every pipeline precisely.
These are some of the real-world problems that Dan Garfield, chief technology evangelist at Codefresh, brought up in episode five of the DevNetwork Dev Professional Series. In this article, we follow his talk to examine the benefits of microservices, their hangups in a real-world CI/CD case study and solutions for scaling delivery as complexity and numbers increase.
To get a better idea of why we need microservices, Garfield uses an analogy from space—the Apollo 13 mission. Apollo 13 was made of two ships: the lunar module and the command module. In software terms, these were essentially two giant monoliths.
When the command module experienced an oxygen failure, its crew needed to repurpose oxygen filters from the lunar module. However, the lunar module engineers designed their oxygen filters differently, causing a major headache to reassemble and retrofit these parts.
This analogy goes to show how monolithic-style development can create tunnel vision. When teams adopt guidelines that work only for the monolith, they are usually incompatible externally. This stifles the possibility of reusable components, thus decreasing code value and extending overall development effort.
“Being able to reuse components and scale independently is the benefit of microservices,” said Garfield.
To see real-world microservices architecture in action, take Expedia. Previously, Expedia services had existed within monoliths, which meant much duplication. Services were detrimentally tethered, meaning if a service broke you had to fix the entire monolith.
Garfield describes how Expedia isolated its architecture into individual microservices to represent abilities such as car search, car sorting, bookings, airline search and others. While this aided elasticity and usability, the microservices journey came with issues.
Expedia ran into multiple problems within their microservices journey. Though Expedia had valid business reasons to adopt microservices, “they didn’t have a plan for how to scale their CI/CD that was realistic,” expressed Garfield.
Foremost, as the number of microservices expanded into the hundreds and thousands, standardizing CI/CD became quite difficult. The organization was using one CI/CD pipeline per service, meaning they now had to support thousands of pipelines, each connected to an individual Git repository. A microservice may also have different pipelines for production, internal use or testing environments, leading to even more duplication.
When you scale microservices, you also encounter a new set of roadblocks related to network and storage. This became extremely evident when Expedia realized the sheer network bandwidth and related costs from running validation and integration testing on thousands of separate CI/CD pipelines.
“The scale of automation itself is where most people have the biggest bottlenecks,” said Garfield.
Consolidating codebases with plugins and shared libraries became a common issue. Expedia developers experienced issues with Jenkins and were forced to programmatically or manually edit each pipeline, copy and pasting code from one to the next. In short, the flow was not easy, and differing approaches between teams led to shadow IT.
To help ease large-scale microservice environments and CI/CD pipelines, Garfield recommended taking a few approaches:
There are many resources on Docker Hub to ease common CI/CD steps such as building a Docker image, git clone, running unit tests, linting projects and security scanning. Using these images can help avoid wasted development effort.
In this setup, Triggers carry context, or metadata, allowing a pipeline to change its behavior accordingly. These Triggers could be time-based or centered around actions like a repo Git commit, or pushing a new image. When a trigger is initiated, the trigger brings in relevant dependencies to perform an action, such as cloning a repository or cloning a codebase. Further steps can then consume these unique criteria.
Garfield demonstrated how this is accomplished in practice within the Codefresh CI platform with native Kubernetes integrations. The platform also enables you to filter and view by certain criteria; a window into commits and changes.
Garfield acknowledged this overall strategy does mean making microservices a bit more uniform, but perhaps the ease of deployment could outweigh the development constraints.
To solve this issue, teams can adopt a canary release strategy. Canary release is a practice in which new software is first released and tested among a small subgroup of users. It’s a good way to “limit the blast radius of a change that goes wrong,” said Garfield.
Adopting a single container-based CI pipeline could help resolve the expanding set of issues with large-scale microservices testing and deployment. By templatizing pipelines, many microservices could abstract complexity and reuse the same modular components. Essentially, make CI/CD reusable in the same way microservices are.
Service Oriented Architecture (SOA) was the great hope of organizations decades ago when they sought to advance legacy system integration, reduce and bypass layers, and rapidly access the system of record. At the time, the existing solution was point-to-point integration, creating a brittle “spaghetti” middle layer that was hard to manage. This was replaced by SOA, which was later augmented with ESB. This fixed the spaghetti mess by creating an intermediate set of layers, which added complexity to the integration process.
Unfortunately, most IT departments with legacy systems and SOA still struggle to be as agile as needed in this ever-increasing global and digitized world.
Today, upward of 90% of the world’s enterprise applications are monolithic. When they were created, monolithic design was the best approach available. However, as business needs evolved and the demand for greater agility intensified, developers and IT teams are considering breaking free of these monolithic architectures.
SOAs have become a hindrance, not the solution they once were.
The promise of SOA initiatives was extending the reach of core business functions while reducing the internal expenses and complexity that grew alongside monoliths. SOAs achieved these goals by breaking the core functions of a monolith into web services using protocols such as simple object access protocol (SOAP) and eXtensible markup language (XML).
SOAP was built for universal application communications. Because it’s based on XML, SOAs designed with SOAP could, in theory, be used to create an agnostic integration layer. Rather than struggling to piece together various proprietary systems, these protocols would open monoliths on different operating systems so they could work together.
This agnostic integration layer would let system administrators connect pieces of a monolith to an enterprise service bus (ESB) to achieve an agile, plug-and-play SOA. In some cases, this approach succeeded.
A great deal of “legacy SOA” still provides value. In these cases, the value the SOA provides is the behind-the-scenes server-to-server communication that mostly aids developers. When it comes to serving internal and external customers–and their rapidly changing demands–SOA isn’t always up to the task.
To achieve service integration, the ESB must oversee the messages from start to destination. This communication isn’t as simple as the SOA vendors may have promised. Consider an SOA integration within a core banking application. The message must go from the core application on the mainframe to a branch office server. If the business logic states a message is only relevant for one business day, administrators must decide whether to move it to a queue, log or disregard when the day passes. This is a common scenario for core banking applications in an SOA, but how well does it work?
In this example–and others like it–the ESB must know whether a business day has passed to make the right decision about where to send the message. Therefore, the integration requires an algorithm–meaning that ESB cannot simply rely on a universal rule for sending messages between the monolith and external integration.
When administrators introduce new business logic into the monolith integration, it turns what was supposed to be an agnostic service layer into a new application layer, increasing complexity. Instead of simply integrating the monolith into a new digital service, the enterprise ends up with a larger monolith–one that includes the mainframe and all the new integration stacks.
Despite the promises of SOA, the integration problems result in increased maintenance time, greater complexity of code and software and the continued growth (not elimination) of monolithic applications.
The first rule of effective integration is “smart end-points and dumb pipes.” Building logic and layers into the service layer breaks that rule, increases the overall complexity and adds another legacy application to your portfolio.
Bottom line: Trying to optimize the SOA approach will only result in larger monoliths.
Using microservice architectures helps realize the original promise of SOAs, delivering the benefits of SOAs while removing many disadvantages.
For decades, CIOs have tried to transition away from monoliths by taking traditional approaches to integration, only to find they’ve doubled down on legacy investments. All these integration stacks end up coupled to legacy systems and only result in more work for the IT team–and a less agile organization.
The integration step of SOA was never intended to include business logic. Trying to force business logic into this approach leads to workarounds and extra effort just to achieve a less-than-ideal result.
The key factor is how to incorporate microservices without adverse effects. Luckily, microservices architecture can be forged from SOAs by introducing these new principles:
Ultimately, the goal should be to refactor monoliths in a way that shifts the IT stack toward microservices. By taking the right approach, you can make the most of both microservices and APIs within your SOA. You enhance your digital journey, leading you to IT efficiency, faster cycles, greater scalability and competitive differentiation.
In designing and organizing a two-day workshop on microservices, I’ve been thinking a lot about how to explain monolith application decomposition and what a transition to microservices might look like. This is a small subset of that material, but I wanted to share it to get feedback (in the workshop we go into more detail about whether you should even break up your monolith). This is based on my own tried-and-true real-life experience, as well as my work with the many Red Hat customers I’ve met in North America over the last few years. Part I explores the architecture, while the second part (to be released shortly) will cover some technology that can greatly help in this area.
Before we dig in, let’s set up some assumptions:
If you dig into the conference/blog posts this topic, often times you’ll find it offers these words of advice:
I’m afraid this advice is quite useless.
The more useful material discusses an approach that at times look something like this:
Let’s take a closer look:
You start off with some nasty monolith. In the image above I’ve simplified this to denote the different modules and database tables that may be involved here. We identify which modules we wish to break out of the monolith and figure out which tables are involved and then go from there. Of course, the reality is monoliths are far more entangled with modules conflated with each other (if any modules). More on that in a bit.
The next step is to identify which tables are used by the Foo module and break those out into its own service. This service is now the only thing that can access those Foo tables. No more sharing tables! This is a good thing. And anything that once did refer to Foo must now go through the API of the newly created service. In the above image, we update the Bar and Cheese service to now refer to Foo service whenever it needs Foo things.
The last step is to repeat this effort until you’re left with no more monolith. In the above image, we’ve done the same thing for the Bar service and you can see that we’re moving to an architecture in which services own their data and expose APIs—which sounds similar to what we hear about microservices.
While this approach is generally a decent set of guidelines, it misses a lot of the fidelity that we really need when going down this path. For example, this approach glosses over the fact that we cannot just stop the world to remove tables from databases. Also:
– Etc., etc.
Let’s take a look at a concrete example and what the approach/pattern would look like and the options we may have as we go.
This example comes from the aforementioned workshop. I’ll be adding some color around the splitting of the services, but there are more detailed guidelines around Domain Driven Design, coupling models and physical/logical architecture that we go into for the workshop that we’ll leave out for now. On the surface, this approach appears to deal with decomposition of existing monolith functionality but applies equally well to adding new functionality around the monolith; this may be the more likely case, since making changes to the monolith can be quite risky.
This is the Monolith that we’ll be exploring. It’s based on the TicketMonster tutorial from developers.redhat.com. That tutorial explores building a typical Java EE application and ends up being a good example—it’s not overly complicated but has enough meat to it we can use it to illustrate key points. In part two of this blog post we’ll actually go deeper into the technology frameworks/platforms.
From this image, we’re illustrating that the Monolith has all modules/components/UI co-deployed talking to a single monolithic database. Whenever we need to make a change, we need to deploy all of it. Imagine the application is 10+ years old and very difficult to change (for all the technical reasons, but also for team/organization structure reasons). We would like to break out the UI and key services that will allow the business to make changes faster and independently from each other to experiment with delivering new customer and/or business value.
Considerations
In this step, we’re going to decouple the UI from the Monolith. Actually in this architecture, we don’t actually remove anything from the Monolith. We start to reduce risk just by adding a new deployment that contains the UI. The new UI component in this architecture should be very close (exactly?) to the same UI that’s in the Monolith and just calls back to the Monolith’s REST API. Of course, this implies the monolith has a reasonable API that an external UI could use. We may find this is not the case; typically, this type of API may resemble more of an “internal” API, at which point we’d need to think about some integration between the separate UI component and the backend monolith and what a more consumable public-facing API might look like.
We can deploy this new UI component into our architecture and use our platform to slowly route traffic to it while still routing to the old monolith—this way we can introduce it without taking downtime. Again, in the second part of the blog post we’ll look at this in more detail; however, the concept of dark launch/canary release/rolling release are all very important here (and for subsequent steps).
Considerations
In the previous step, we introduced a UI and slowly rolled over traffic to the new UI (which communicated directly with the Monolith). In this step we do a similar deployment strategy, except now we slowly release a new deployment of the monolith with the UI removed. We can bleed traffic over and stop/rollback if we see issues. Once we have all traffic going to our Monolith without the UI (termed Backend from here on) we can remove the Monolith deployment completely. By separating out the UI, we’ve now made a small decomposition to our monolith and reduced our risk by relying on dark launches/canaries/rolling releases.
Considerations
In the next step, and skipping the detail of coupling, Domain Driven Design, etc., we’re introducing a new service: the Orders service. This is a critical service that the business wants to change more frequently than the rest of the application and has a fairly complicated write model. We may also explore architectural patterns such as CQRS with this model, but I digress.
We want to focus on the boundary and API of the Orders service in terms of the existing implementation in the Backend. In reality, this implementation is more likely to be a rewrite than a port of existing code, but the idea/approach here is the same either way. Notice in this architecture, the Orders service has its own database. This is good; we’re shooting for a complete decoupling. However, we’re not there yet. There are a few more steps we need to consider/undertake.
This step would also be a good time to consider how this service plays in the overall service architecture by focusing on the Events it may emit or consume. Now is a great time to perform an activity such as Event Storming and think through the events we should publish as we start to take transactional workloads. These events will come in handy when we try to integrate with other systems and even as we evolve the monolith.
At this point, we should be continuing to evolve the service’s API and domain model and how it’s implemented in code. This service will store any new transactional workloads into its own database and keep it separate from any other services. Any services needing access to this data would have to go through an API.
One thing we will not be able to ignore: Our new service and its data is intimately related (if not exactly the same in some areas) to the data in the monolith. This is highly inconvenient, actually: As we start to build the new service, we’ll need existing data from the Backend service’s database. This could get really tricky because of the normalization/FK constraints/relationships in the data model. Reusing an existing API on the monolith/backend may be too coarse-grained and we’d have to reinvent a lot of gymnastics to get the data in the shape we want.
What we want to do is get access to the data from the Backend in read-only mode through a lower-level API and have a way to shape the data/data model into the model that fits better with the domain model in our new service. In this architecture, we’re going to temporarily connect to the Backend database directly and query the data directly as we need. For this step, we need a consistency model that reflects a direct database access.
Some of you may cringe initially at this approach. And you should. However, the plain fact is this approach is totally valid and has been used successfully in highly critical systems—and more importantly, is not the end-state architecture (if you think this may end up being end-state architecture, stop and do not do this). You also may point out that connecting up to the Backend database, querying data and massaging it into the right shape we need for our new service’s domain model involves a lot of nasty integration/boilerplate. I’d argue that since this is temporary in our evolution of the monolith, it’s probably OK: ie, use technical debt to your advantage and then quickly pay it down. However, there is a better way. I’ll discuss this in part 2 of this blog post.
Alternatively, some of you might be saying, “Well, just stand up a REST API in front of the Backend database that gives lower-level data access and have the new service call that.” This is also a viable approach, but not without its own drawbacks. Again, will discuss this in more detail in part 2.
Considerations
In our next step of this approach/pattern, we need a way to actually direct traffic to our new service. However, we don’t want to do a big-bang release. We don’t want to just throw this into our production traffic (especially considering that this example uses an “order” service that takes orders! We don’t want to introduce any issues with taking orders!). Although we cannot easily change the underlying monolith database, if there’s hope, you may be able to make changes to the monolith application carefully to call our new orders service. I highly recommend Michael Feather’s “Working Effectively with Legacy Code” if you’re unsure of how best to do this. There are patterns such as the Sprout Method/Class and/or Wrap Method/Class that could help.
When we make a change to our monolith/backend, we don’t want to replace the old code path. We want to put in just enough code to allow both the old or the new code path to run—potentially even in parallel—as deemed fit. Ideally, the new version of the monolith with this change will allow us to, at run time, control whether we’re sending to the new Orders service, the old monolith code path or both. In any combination of call paths, we want to instrument heavily to understand any potential deviations between the old and new execution path.
Another thing to note: If we enable the monolith to send the execution to the old code path as well as calling our new service, we need a way to flag this transaction/call to the new service as a “synthetic” call. If your new service is less critical than in this example, an order service, and you can deal with duplicates, then maybe this synthetic-request identification is less important. If your new service tends to serve more read-only traffic, again, you may not worry as much about identifying synthetic transactions. In the case of a synthentic transaction, however, you want to be able to run the entire service end to end including storing into the database. You may take options here to either store the data with a “synthetic” flag or just roll back the transaction if your data store supports that.
The last thing to note here is when we make our change to the monolith/backend, we again want to use a dark launch/canary/rolling release approach. We will need our infrastructure to support this. We will look closer at this in the second part.
At this point, we’re forcing the traffic back through the monolith. We’re trying not to perturb the main call flow as much as possible so we can quickly roll back in the event a canary doesn’t go well. On the other hand, it may be useful to deploy a gateway/control component that can more finely control the call to the new service instead of forcing to the monolith. In this case, the gateway would have the control logic whether to send the transaction to the monolith, to the new service or both.
Considerations
If we find the previous step does not introduce negative impacts to our transaction path and we have high confidence through our testing and initial experimentation in production with shadowing the traffic, we can configure the monolith to actually send traffic over to the new service. In this case, we need to be able to specify certain cohorts/groups/users to always go to the new service and we’re slowly draining the real production traffic that goes through the old code path. We can increase the rolling release of our Backend service until all of our users are on the new Orders microservice.
One point of risk we need to make concrete: Once we start rolling live traffic (non-shadow/synthetic traffic) over to the new service, we’re expecting users that match the cohort group to go to this new service always. We’re not able to switch back and forth between old and new code paths. In this case, if we do need to rollback, this will involve more coordination to move any new transactions from the new service back to a state that the old monolith could use. Hopefully this doesn’t happen, but is something to note and plan for (and test!).
Considerations
At this point, we should have our Orders microservice taking live production traffic. The Monolith or Backend is still around handling other concerns, but we have successfully migrated service functionality out of the monolith. Our immediate pressing concern now is to pay off the technical debt we’ve borrowed when we created a direct database connection between our new microservice and the Backend service. Most likely this would involve some kind of one-time ETL from the monolith database to the new service. The monolith may be required to still keep that data around in a read-only mode (think: regulatory compliance, etc.). If this is shared reference data (ie, read only) this should be okay. We must make sure that the data in the monolith and the data in the new microservice is not some shared/mutable data. If the data is shared/mutable, then we can end up in divergent data/data ownership problems.
Considerations
Once we’ve completed the previous step, we should have our new Orders microservice-independent and ready to participate in a services architecture. The steps presented here all have their own considerations, pros and cons. We should aim to satisfy all of the steps and not leave technical debt to accrue interest. Of course, there will be variations on this pattern, but the approach is sound.
In the next blog post, I’ll show how to do these steps with the example service I’ve referred to previously and dig deeper into some of the tools, frameworks and infrastructure that helps assist. We’ll look at things such as Kubernetes, Istio, feature-flag frameworks, data-view tools, and test frameworks.
Part 2 will follow in December
Follow along (@christianposta) on twitter or http://blog.christianposta.com for the latest updates and discussion.