devopsy.kinsta.cloud

Author: Javier Garza

  • Building Amazing Apps, Part 3: Optimizing the Network

    Building Amazing Apps, Part 3: Optimizing the Network

    This is part three of a series of three articles on building and delivering amazing apps at the edge. Read part one here, and part two here.

    Introduction

    There are millions of apps available today, and it’s incredibly easy for any user to download and install any of them with just a few taps on a screen or click of a mouse. If you want to build and deliver an app that will amaze users, you need to be able to differentiate from the rest of the pack.

    Over the past 14 years, I’ve been working closely with the top companies delivering the most popular apps, helping them optimize their apps for speed and scale so users get a great experience. In this three-part series, we’ll explore the fundamental steps you must take to build and deliver your own amazing app at scale that will keep users coming back for more.

    We’ll break these steps down into three key areas:

    • Front end: Covers the parts of the app that end users interact with.
    • Back end: Looks at the required origin infrastructure to support the app, including databases, application, web servers and more.
    • Network: The glue that connects the front end and the back end.

    In the final article of this series, we’re going to explore four things you can do on the network to optimize your app and make it more amazing:

    • Distributed DNS
    • Protocol optimization
    • Latency optimization
    • Edge computing

    Distributed DNS

    The Domain Name System (DNS) is a critical part of making sure your clients can access your applications (in fact, it’s a very popular attack vector by hackers today because if your DNS is down, nobody can access your PPP). Distributing your DNS across many servers can help ensure that your DNS (and thus, your app) is fast and highly available, so you should use a DNS vendor that has a large DNS server network in a wide range of locations.

    Validating whether you have a properly distributed DNS infrastructure is rather easy. Just follow these steps:

    • Identify how many name servers your domain uses and their IP addresses. Ideally, you want a number between three and 12—the more the better. You can use command-line tools such as dig and nslookup to identify the name servers of a given domain.
    • Use an IP geolocation system to find out the physical location of the name servers. At this point, you can get a rough idea if those servers are close enough to your end users. Hint: Some CDNs offer diagnostic tools that can be scripted via APIs to automate IP geolocation.
    • If you want to be more thorough, you may want to run connectivity tests between the main locations of the users and the name server IP addresses to make sure the network latency—i.e., how long it takes for information to go back and forth between the client and the server—stays under 100 ms (the lower the better). Some internet providers and CDNs provide looking glass technology, which enables you to run connectivity tests that originate from servers around the world.

    Protocol Optimization

    Once your clients know where to connect, how they connect and the protocol used is something that shouldn’t be neglected. HTTP/2 is the modern, superior network protocol, as it uses a more efficient binary format, header compression, single network connection, stream multiplexing and server-push. Actually, I was very involved in the early years of HTTP/2 (the standard was approved in February 2015) and have since helped some of the largest websites on the internet implement HTTP/2 and still spend a considerable amount of my time testing the new protocol in all sorts of environments. I’ve shared my findings in talks at conferences including Velocity and in the “Learning HTTP/2” O’Reilly book, which I co-authored. In a nutshell, most sites can improve page load performance between 5% and 15% by just switching the protocol from HTTP/1.1 to HTTP/2, which is as simple as flipping a switch if your website uses HTTPS.

    To illustrate that improvement, I created a demo site that tests the download speed of many small images over HTTP/1.1 and over HTTP/2 and displays the differences.

    Figure A

    As you can see in Figure A, the content loaded eight times faster over HTTP/2 than over HTTP/1.1 on my mobile phone over an LTE connection. That’s a huge improvement considering how simple it is to implement HTTP/2.

    According to the HTTP Archive, as of June 2018, 20% of hosts support HTTP/2, but those hosts are responsible for 40% of all requests—meaning HTTP/2 powers some of the web’s most popular sites.

    However, as explained in the HTTP/2 Performance chapter of the Learning HTTP/2 book by O’Reilly, there are a lot of factors that can impact the benefits of HTTP/2. Some of them you can control, while others—such as network packet loss—are much more difficult to tackle and can impact HTTP/2 performance because HTTP/2 runs over a single TCP connection.

    Note: TCP stands for Transport Control Protocol and some parts of its implementation (e.g., slow to start and cumbersome network congestion algorithm) can impact HTTP/2 performance because all the communication with the server goes through a single TCP connection.

    The future of the web is HTTP/3, which uses HTTP/2 over QUIC (a newer and more efficient transport layer protocol). However, as of April 2019, HTTP/3 is still in draft state and hasn’t been approved as a standard.

    Ideally, your servers should monitor the network connection and use different network protocols depending on the current conditions. Certain content delivery networks (CDNs) offer this functionality. For example, to maximize throughput, a top CDN provider uses custom software on its edge servers to dynamically use different network protocols, TCP settings and congestion control algorithms, depending on the specific conditions of each individual connection the server makes with an end user. In short, using the optimal protocol makes your app faster and more amazing.

    Latency Optimization

    Latency is a physical limitation of nature that impacts the throughput on computer networks. You probably know that whenever you are outdoors and there is a storm with lightning, you can count the number of seconds until you hear the thunder to identify how close the storm is (and if you count less than a second, you better get inside!). In computer terms, latency is measured as the time it takes to send a network packet back and forth between two systems. This concept is known as round-trip time (RTT) and is typically measured in milliseconds. You can use network tools such as ping or mtr to easily measure the RTT between two systems.

    Latency has a huge impact on performance because the further away you are from the server you want to connect, the slower the communication will be. As a rule of thumb, you want to ensure your latency is well under 100 ms. The easiest way to achieve that is by leveraging the local device cache to avoid extra network connections to the server and using a CDN to put your server closer to the end user.

    Here is a graph (see Figure B) showing that latency plays a larger role on page load time than bandwidth. As you can see, doubling the bandwidth from 5Mbps to 10Mbps results in just 5% improvement in page load times, and any bandwidth increase over 10Mbps doesn’t make a big difference. Reducing the latency, however, drastically improves the page load times. That makes your app more amazing for your end users.

    Figure B

    Edge Computing

    If you are a developer or a DevOps engineer (or just love tech) chances are you are within the TOP 1% of world population in terms of mobile computing power. But things you take for granted, such as resizing images on a mobile phone, are going to take much longer for many millions of internet users who have less powerful mobile devices.

    So, what can we do if running logic at the origin causes delays because of the network latency and running logic on the client side may be too slow on less powerful devices? The solution is edge computing, which consists of moving the origin logic to a server close enough to the end users that latency does not impact performance. CDNs are in the best position to offer this proximity because of their natural distributed architecture; a CDN can set cookies, rewrite HTML and resize and deliver optimized images for the screen size of your users so those images don’t take a long time to display on low-end mobile devices with limited CPUs.

    Edge computing is the next step beyond cloud computing. You should try to use a CDN company that offers you the ability of running business logic as close as possible to where your users are, to ensure that your app is fast and can scale without impacting your infrastructure.

    Summary

    In this series of articles, we reviewed some basic things you should consider if you want to build and deliver amazing apps at scale. This is not a comprehensive list of all the things you can do but provides you with some of the most impactful areas for app improvement.

    — Javier Garza

  • Building Amazing Apps, Part 2: Optimizing the Back End

    Building Amazing Apps, Part 2: Optimizing the Back End

    This is part two of a series of three articles on building and delivering amazing apps at the edge. To read part one, click here.

    There are millions of apps available today, and it’s incredibly easy for any user to download and install any of them with just a few taps on a screen or click of a mouse. If you want to build and deliver an app that will amaze users, you need to be able to differentiate from the rest of the pack.

    Over the past 14 years, I’ve been working closely with the top companies delivering the most popular apps, helping them optimize their apps for speed and scale so users get a great experience. In this three-part series, we’ll explore the fundamental steps you must take to build and deliver your own amazing app at scale that will keep users coming back for more.

    We’ll break these steps down into three key areas:

    • Front end: Covers the parts of the app that end users interact with.
    • Back end: Looks at the required origin infrastructure to support the app, including databases, application, web servers and more.
    • Middle-mile: Explores the internet, where all the information needs to traverse back and forth between the front end and back end.

    In this article, we are going to explore five things you can do on the back end.

    Distributed Architecture

    In a nutshell, distributed architecture means having origin (back end) infrastructure that is geographically dispersed to maximize availability and performance.

    There are three main types of back end configurations:

    • On-premises: This is where the hardware and software are owned and managed by your organization. This type of back-end architecture is often more expensive and resource-intensive to maintain but offers a higher degree of security and flexibility because you have full control of the hardware and software.
    • Cloud-hosted: This is where the hardware and software are hosted on a third-party cloud provider. This type of architecture runs on dedicated or multi-tenant servers, and is usually more cost-effective to create and manage because some of the cloud providers include efficient infrastructure configuration/orchestration tools.
    • Hybrid: This is where your architecture is a mix of on-premises and cloud-hosted. This is the most flexible and common solution, since most companies that traditionally have relied upon all on-premises architecture are slowly migrating some of their back end to the cloud. For this type of solution, it’s best to use open source, multi-cloud tool solutions.

    The goal for your configuration should be having a back-end system that is geographically dispersed enough (with most locations close to your users) to ensure low network latency, while at the same time avoiding a single-point-of-failure to ensure the service runs, despite any localized outage.

    Regardless of your configuration, the general idea of distributed architecture remains the same: Using data centers where the load is distributed via a global load balancer, and where critical information is replicated across the data centers via fast, fiber-optic private links.

    Here are some concepts you can research that will help you to choose the most optimal, distributed architecture for your needs: virtual machines, containers, microservices, data replication, stateless apps, serverless, distributed processing, etc. devopsy.kinsta.cloud is an excellent resource to learn more about them.

    Generally speaking, you can simplify your origin infrastructure and maximize end-user performance by putting a content delivery network (CDN) between your end users and your back end. A CDN is a super-distributed network of reverse proxies that allows you to put your content very close to end users, minimizing network latency to achieve rapid app interactions, especially when the CDN caches your content on its edge servers.

    Auto-Scaling

    You can optimize costs and performance by choosing an architecture that auto-scales (up and down) depending on current traffic demands. To implement this effectively, you need to monitor global traffic and set triggers that launch infrastructure on-demand so it is ready when needed. Most of the largest cloud services companies support auto-scaling services, and some of them even allow you to tune your settings to favor either costs or performance. One of the easiest ways to implement effective auto-scaling is with CDNs, which can automatically scale traffic with minimal effort just by configuring a set of rules that define what content should be handled completely by the CDN (either dynamically generated at the edge or cached) and what content should be pulled from your origin infrastructure.

    Continuous Monitoring

    Continuous monitoring helps maintain the quality of service. There are many open source tools for continuous monitoring that fall mostly into the following four categories:

    • Infrastructure monitoring: This allows you to keep track of compute resources, storage, network usage, etc.
    • Cloud providers: Most of the largest cloud providers have integrated tools for monitoring your back end in each respective cloud platform.
    • Application performance monitoring (APM): To monitor your application’s framework.
    • Aggregators: Tools that allow you to aggregate and analyze data across several monitoring tools.

    In addition, it’s a good idea to complement server-side monitoring with real user monitoring (RUM) tools to identify whether a specific code deployment impacts performance.

    Performance

    A faster back end makes for a faster front end, and, as we know from part one of this series, performance is a major contributor to user experience. In addition to focusing on improving metrics including latency, throughput and errors (all of which are directly correlated with back-end performance), it’s critical to have a “culture of performance” in your company. José Pérez provides an overview of that concept in his article, “Fostering a Web Performance Culture.” This is a perfect example of how you could implement a culture of performance in your organization, by having performance tools integrated into your IDEs—this can provide instant feedback to developers on how new code performs against a defined performance baseline so that any code that might degrade performance is identified before it is deployed.

    For content that is not personalized, such as a product catalog, news or media assets, you can easily improve performance by putting a CDN in front of your back end (and caching that content on the CDN’s edge servers). Those edge servers are distributed around the world, located much closer to end users than your back-end infrastructure, to minimize network latency and maximize throughput.

    DevOps Automation

    Finally, a big part of setting up monitoring that works (and in responding to failures quickly) is DevOps automation. Building automated DevOps tooling allows you to react to changing conditions automatically, in real time.

    For example, you can use DevOps automation to alert your team via a chat system such as Slack when a deployment impacts the service, with an option to roll back easily the deployment within the chat app. Another great example of DevOps automation is Pinterest’s implementation of a “performance detective” that runs a binary search (similar to git bisect) to determine the offending commit that is degrading app performance after a release. If you combine these two automations, you could detect a performance degradation caused by a release; let the DevOps team know that it has been rolled back and track the offending code in the version control repository. That’s just one specific example among the many ways that DevOps automation can further optimize your back end.

    Coming up next in this three-part series: Everything you need to know about optimizing the middle-mile.

    — Javier Garza

  • Building Amazing Apps, Part 1: Optimizing the Front End

    Building Amazing Apps, Part 1: Optimizing the Front End

    There are millions of apps available today, and it’s incredibly easy for any user to download and install any of them with just a few taps on a screen or click of a mouse. If you want to build and deliver an app that will amaze users, you need to be able to differentiate from the rest of the pack.

    Over the past 14 years, I’ve been working closely with the top companies delivering the most popular apps, helping them optimize their apps for speed and scale so users get a great experience. In this three-part series, we’ll explore the fundamental steps you must take to build and deliver your own amazing app at scale that will keep users coming back for more.

    We’ll break these steps down into three key areas:

    • Front end: Covers the parts of the app that end users interact with.
    • Back end: Looks at the required origin infrastructure to support the app, including databases, application, web servers and more.
    • Middle-mile: Explores the internet, where all the information needs to traverse back and forth between the front end and back end.

    Let’s explore the front end, and the five things you need to take into consideration to build and deliver a world-class front end for an amazing app.

    Beautiful User Interface

    People love beautiful things. That’s why they enjoy looking at the orange colors of a sunset or why they pay exorbitant amounts of money for a work of art. Building an aesthetically pleasing interface must be a top priority during the design phase. The first sign of a good user experience is an attractive design, and users are more likely to spend more time with the app if it is appealing to the eye.

    So where do you start when designing your app? Begin by reviewing the most popular apps on the app stores or the ones you use and enjoy the most and write down what design characteristics you like from each one. You should also follow industry best practices for app design such as:

    • Pay attention to graphic design.
    • Ensure you use curated images that look great.
    • Pick beautiful fonts with curved edges, e.g., Open Sans.
    • Manage white space effectively to display as much information as possible without having crowded text.
    • Ensure your UI is intuitive and simple to reduce the learning curve.
    • Avoid information overload through short and clear wording.
    • Maintain consistency within all the screens in the UI.
    • Select a distraction-free UI that avoids blinking text and excessive use of colors.
    • Follow standards (follow UI designs already in place).

    Accessibility

    At a fundamental level, it is a social responsibility to ensure your application can be used by people with impaired mobility, vision, hearing, cognition or language understanding. In some context, it is also a legal requirement.

    An excellent accessibility reference for developers is the Web Content Accessibility Guidelines (WCAG), which is sponsored by the World Wide Web Consortium (W3C). This reference includes accommodations for blindness and low vision, deafness and hearing loss, limited movement, speech disabilities, photosensitivity and more. For example, you should provide text alternatives for non-text content such as an image or a graph using the alt text HTML tag that accessibility software can use to describe what the image or the graph is. This is as simple as <img src=”smiley.jpg” alt=”Smiley face“>.

    Accessibility is too often forgotten or left to the later stages of app design. However, it is much more difficult to implement accessibility functions later on than if you plan for it upfront.

    Speed and Responsiveness

    User patience is shrinking and first impressions matter, so your app needs to start as quickly as possible. But it’s often easier said than done to get everything ready for user interaction within 1 second flat as certain operations, such as sending data to a server over a slow mobile network, can take a long time.

    So, given the challenges, how can you overcome them to provide a fast experience in your app?

    Start with performance-oriented design in mind, building the app logic in a way your users will perceive it as being fast at all times. You can, for example, implement caching and store the information presented to the user in the local cache. You can then use the cache to restore the last status almost instantaneously during startup while the app opens the network connection to the server and requests the data. Another characteristic of performance-oriented design is performance budget and code optimization. Performance budget means you set a performance baseline for a specific piece of code or functionality and put tools in place to ensure that baseline is not exceeded when releasing new code. Code optimization means trying new approaches to make the code run faster.

    Do this in the following cycle: optimize -> deploy -> measure.

    Repeat this cycle several times until your performance gains start to flatline. After this point, you can apply two concepts from the psychology world to make your app “feel faster” (aka “perceived performance” and “active waiting”).

    Perceived performance refers to how users perceive whether something is slow or fast, while active waiting refers to the fact that users perceive time as passing more quickly when they are engaged while they wait. So, when there is an operation where the user has to wait, you can incorporate some movement on the screen to draw the user’s attention—or, better yet, get them to interact with the app by providing a suggestion for optimizing their usage of the app or a curated quote. The chat app Slack does this pretty well.

    Network Awareness 

    Your app should keep track of network status, monitoring the available bandwidth and identifying when the network is not available at all. With this information, the app should adapt its behavior to improve the overall user experience. For example, let’s say the user is trying to send some data to the server while the network is not available. One solution is to switch the user to an offline mode, and let the user know they are offline and that the operation they are trying to do will be processed as soon as they are back online. At the same time, you could offer a cached version of the information, so the user can partially interact with the app.

    Network monitoring in your app should be planned carefully to avoid consuming too much power, which may impact the device’s battery life. For example, when the network is already active, you could send a few bytes to measure the latency to the server. You should avoid a constant network keep-alive, as actively maintaining the network even when there is no data to send or receive may impact the device’s energy resources.

    There are also intelligent edge solutions you can apply to your app that automatically adapt the compression of images depending on the network conditions. This means that the edge server can deliver a smaller version of the image for devices that have a slow network connection, without having to implement any logic on the app.

    Security

    Security is not an option, but a requirement that you should keep top of mind while building your application. The last thing you want is a security breach that compromises your users’ data or the integrity of your servers.

    For starters, ensure all your network connections are encrypted end to end and use secure protocols such as TLS v1.2 at the minimum or v1.3, which offer better security and a faster connection establishment. Ensure that your certificate offers strong ciphers that implement “perfect forward secrecy” (which makes it more difficult to be caught by man-in-the-middle attacks) and implement login techniques within the app that verify the user through secure techniques such as two-factor-authentication.

    You can also enhance your application security by implementing security measures at the edge such as automated bot protection. In this instance, machine learning technology in your edge servers can use device sensors such as the accelerometer to differentiate a real user from a bot that has stolen credentials and is trying to perform a malicious action.

    Coming up next in this three-part series: Everything you need to know about optimizing the back end.

    — Javier Garza