As we close out 2023, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2023.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Amazon Prime Video has ditched its use of microservices-cum-serverless, reverting to a traditional, monolithic architecture. It vastly improved the workload’s cost and scalability. (more…)
In enterprise development and deployment, the pattern is easy to detect. A big monolithic application is targeted for re-architecting as a microservices architecture. People excited about microservices get together and break up the design to make it far less monolithic and far more loosely coupled.
If your organization is lucky, it is at this stage that those responsible realize that breaking a monolithic application down too far is bad for complexity, performance, monitoring and maintenance—pretty much everything re-architecting is supposed to fix.
Far too many organizations don’t realize this until the implementation phase, though. First, a metric ton of little tiny microservices are deployed, then the organization realizes that functionality such as database access can be applied much more efficiently through one larger service that handles all interaction with the database for the app, rather than through five or 10 smaller services that replicate code and end up calling each other anyway.
Function as a service (FaaS) has much the same issue. It is an astounding solution for some uses, but in the end, users must consider what access the function requires to other parts of the system and what task is being achieved. The early uses of FaaS are well-documented; longer running independent jobs such as image or video processing were a bit early hit, for example, particularly for services that have highly varying traffic rates. But other uses, such as handling bottlenecks in traffic spikes (hello holiday shopping season!) or out-of-band operations processing, are less documented at this point. Entire applications have been designed from the ground up in serverless, but they are relatively few in terms of overall applications out there.
FaaS is highly suited to DevOps, because upgrading the function is as simple as redeploying code. There are no concerns about OS or library version, because those are largely beyond your control. But that is as big a negative as it is a positive. There is no flexibility in OS version or library inclusion because those are beyond your control—for the most part, anyway.
As with the vast majority of new technologies, expect adoption of serverless to grow and the hype to grow more quickly. It is definitely in the market where we have seen massive shifts—virtualization and then containers really did live up to the hype in terms of usage—but it needs to have a driving reason for adoption. Look closely at what your application and its DevOps toolchain are doing before hopping into the deep end. For some uses, FaaS is the fastest/easiest/lowest-maintenance solution. But that does not make it the Swiss army knife that virtualization and containers have proven to be. Indeed, if you think persistent storage is painful under containers, try it in a FaaS environment—as just one example.
FaaS is useful in many scenarios, but just remember where/when/why you are adding FaaS into your DevOps processes, and take advantage of on-demand processing with few scaling limitations while avoiding seeing everything as a FaaS problem because FaaS is the FaD.
The financial services industry historically has been riddled with barriers to entry. Challengers found it difficult to break through low margins and tightening regulations. However, large enterprises that once dominated the market are now facing disruption from smaller, leaner fintech companies that are eating away at the value chain. These disruptors are marked by technological agility, specialization and customer-centric UX.
To remain competitive, financial services firms are reconsidering their cumbersome technical architectures and transforming them into something more adaptable. A recent survey of financial institutions found that about 85 percent consider their core technology to be too rigid and slow. Consequently, about 80 percent are expected to replace their core banking systems within the next five years.
Emerging regulations meant to address the new digital payment economy, such as PSD2 regulations in Europe, will require banks to adopt a new way to operate and deliver. Changes such as PSD2 are aimed at bringing banking into the open API economy, driving interoperability and integration through open standards. To become a first-class player in this new world of APIs, integration and open data, financial services firms will need the advantages provided by microservices.
Modern fintech requirements create challenges to the established security infrastructure. Features such as digital wallet, robo-advising and blockchain mandate the need for a new security mechanisms. Microservices follow a best practice of creating a separate identity service to address these new requirements.
Rapidly bringing new features to market is a cornerstone of successful fintech companies. Microservices make it easier for different application teams to deliver new functionality independently to meet emerging customer demands. Microservices also scale well to accommodate greater numbers of users and transactions.
The integration layer in a modern fintech solution needs a powerful set of APIs to communicate with other services, both internally and externally. This API layer is notoriously challenging to manage in a large monolithic application. Microservices make the API layer much easier to manage and secure through isolation, scalability and resilience.
In the face of rapidly changing customer, business and regulatory requirements, microservices help financial services companies quickly respond to these changes. But this doesn’t come for free; companies take on increased operational overhead during the shift to microservices. Technologies such as a service mesh can help manage that.
Service mesh provides a bundle of features around observability, security and control that are crucial to managing microservices at scale. Previously, existing solutions such as DNS and configuration management provided some capabilities such as service discovery, but didn’t provide fast retries, load balancing, tracing and health monitoring. The old approach to managing microservices requires that cobbling together several different solutions each time a problem arises, but a service mesh bundles it all together in a reusable package. While it’s possible to accomplish some of what a service mesh manages with individual tools and processes, it’s manual and time-consuming.
Competition from innovative fintech startups, along with ever-increasing customer expectations means established financial services players must change the way they deliver offerings and do business with their customers. Delivering on these new requirements is difficult with legacy systems. Financial services firms need a software architecture that’s fit for purpose—agile, adaptable, highly scalable, reliable and robust. Microservices make this possible, and a service mesh makes microservices manageable at scale.
Benchmarking microservices often falls behind on the engineering team’s to-do list. However, those that aren’t are missing out on a key component. The fundamental goal of benchmarking is to better understand the software, and test out the effects of various optimization techniques for microservices. In this post, I’ll describe an effective approach to benchmarking microservices.
First, create a spreadsheet for tracking your benchmarking. One convenient way to document a series of benchmarks is in a Google Spreadsheet. It allows collaboration and provides the necessary features to analyze and sum up your results. Structure your spreadsheet as follows:
Then, before you engage in benchmarking, clearly state (and document) your goal. Examples of goals are:
Next, pick one key metric, stating clearly which one metric you are concerned about and how the metric affects users of the system. If you choose to capture additional metrics for your test runs, ensure that the key metrics stands out.
Now, you’re going to have to think like a scientist and perform a series of experiments to better understand which inputs affect your key metric and how. Consider and document the variables you devise and create a standard control set to compare against. Design your series of experiments in a fashion that leads to the understanding in the least amount of time and effort. You’ll then need to define a methodology for running your benchmarks. It is critical your benchmarks be:
Document your methodology in detail. Also document how to re-create your environment. Include all details another person needs to know:
In most cases, to accomplish repeatable, rapid-fire experiments, you need a synthetic load generation tool. Find out whether one already exists. If not, you may need to write one. Understand that load generation tools are at best an approximation of what is going on in production. The better the approximation, the more relevant the results you’re going to obtain. If you find yourself drawing insights from benchmarks that do not translate into production, revisit your load generation tool.
It is then important to validate your benchmarking methodology, and repeat a baseline benchmark at least 10 times and calculate the standard deviation over the results. You can use the following spreadsheet formula:
=STDEV(<range>)/AVERAGE(<range>)
Format this number as a percentage, and you’ll see how big the relative variance in your result set is. Ideally, you want this value to be less than 10 percent. If your benchmarks have larger variance, revisit your methodology. You may need to tweak factors like:
Important: Determine the number of results you need to get the standard deviation below a good threshold. Run each of your actual benchmarks at least that many times. Otherwise, your results may be too random.
Now that you have developed a sound methodology, it’s time to gather data. Some tips include:
Now, you’ll need to create enough loads to be able to measure a difference. There are two different strategies for generating load:
As you execute your benchmarks and develop a better understanding of the system, you are likely to discover new factors that may impact your key metric. Add new experiments to your list and prioritize them over the previous ones if needed.
In some instances, the code may not have configuration or control knobs for the inputs you want to vary. Find the fastest way to change the input, even if it means hacking the code, commenting out sections or otherwise manipulating the code in ways that wouldn’t be “kosher” for merges into master. Remember: The goal here is to get answers as quickly as possible, not to write production-quality code—that comes later, once we have our answers.
Once you’ve completed a series of benchmarks, take a step back and think about what the data is telling you about the system you’re benchmarking. Document your insights and how the data backs them up. It may be helpful to:
Finally, you’ll need to present your insights. Here are some final tips for exhibiting to management and/or other engineering teams: