Tag: DNS

  • Speeding Up Software Delivery Is Changing How We Debug Performance

    Speeding Up Software Delivery Is Changing How We Debug Performance

    Deployment frequency has become one of the clearest markers of a mature engineering organization. Teams that once shipped monthly now ship daily, and teams that shipped daily now ship several times a day. This shift has largely delivered on its promise. Smaller changes are easier to reason about, rollbacks are faster, and feedback loops are shorter.

    What has received less attention is the effect this same shift has had on the practice of debugging performance issues. Several assumptions that previously held, a stable release to compare against, a known change set behind a given incident, sufficient time between deployments to observe a system before the next one lands, no longer apply in the same way.

    A Moving Baseline

    Performance debugging has traditionally depended on comparison. An engineer examines current behavior against a known good state, typically the previous release, and narrows down what changed. This approach works well when releases are infrequent enough that the previous release remains a stable, well understood reference point.

    When deployments occur several times a day, that reference point moves continuously. A regression noticed in production may trace back to any one of a dozen deployments merged since the last time someone reviewed that code path closely. The regression itself is genuine, but the comparison method that once made it straightforward to isolate no longer has a single fixed point of reference.

    This does not indicate that frequent deployment introduces more performance issues. It indicates that the traditional method of identifying them, comparison against a known baseline, becomes more difficult to apply as the baseline itself becomes a moving target.

    Smaller Changes, More Difficult Attribution

    A related effect occurs at the level of individual changes. Smaller, more frequent deployments are generally easier to review and easier to roll back. However, when a performance issue surfaces days after a change was merged, rather than immediately following it, attributing the issue to a specific change becomes a process of elimination across many small deployments rather than an inspection of one larger one.

    A single large release makes attribution simpler in one way, since there are fewer candidates to consider, but harder in another, since each candidate is larger and more complex to inspect. Frequent small releases invert that trade off. Each individual change is simple to review, but the number of candidates behind a delayed symptom grows quickly, and a minor dependency update or an altered query parameter buried in a routine pull request is just as capable of causing a regression as a larger architectural change.

    Neither approach removes the underlying difficulty. It appears in a different form depending on how a given team ships.

    Where Transaction Level Visibility Applies

    This is where visibility at the transaction and code level becomes relevant, not as a substitute for deployment history, but as a means of observing what actually occurred at runtime, independent of which release introduced it. A trace showing precisely which method, query, or downstream call consumed the time for a specific slow request does not require prior knowledge of which deployment caused the issue. It identifies the issue directly, within the transaction in which it occurred, whether that turns out to be an unindexed query, a retry loop against a failing dependency, or a serialization step that grew more expensive as payload sizes increased.

    This changes the starting point of an investigation. Rather than beginning with an assessment of which among several recent deployments caused a given issue, an engineer can begin with a direct account of where a specific request spent its time, and work backward toward the change that introduced it, informed by observed behavior rather than elimination alone.

    Dependency mapping serves a similar function at a structural level. As services are deployed and redeployed independently of one another, understanding which services actually call which, based on current behavior rather than the architecture as documented at the last major review, becomes part of maintaining visibility that keeps pace with the system itself.

    Delivery Speed and Diagnostic Speed Are Separate Measures

    It is worth being precise about what deployment frequency actually measures. It measures how quickly a team can move a change into production, a genuinely useful indicator of process maturity. It does not measure how quickly that same team can diagnose a performance issue once one appears in production.

    These two measures can move independently of one another. An organization may improve deployment frequency substantially while diagnostic time remains unchanged, or even increases, if the practices used to investigate performance issues have not kept pace with how frequently the system now changes. Improving one without corresponding attention to the other tends to produce a team that ships quickly and then spends a disproportionate share of that gained time identifying what went wrong afterward, which is a cost that rarely shows up in delivery metrics themselves.

    Keeping both measures aligned generally requires treating runtime visibility as something that keeps pace with deployment frequency itself, present for every service from the point of deployment, rather than something introduced after a service has already accumulated a performance issue worth investigating.

    Where This Leads

    The practices that enable faster software delivery and the practices that make performance issues easier to diagnose are not inherently aligned. The two require deliberate coordination. This typically involves treating transaction level tracing as part of how a service is deployed, rather than as a separate concern addressed later, so that the same velocity built into shipping code remains available when investigating what that code actually does once it is running. OpManager Nexus provides a 30-day free trial for teams that want to see this kind of transaction level visibility applied to their own environment.

  • Microsoft Outage Outrage: Was it BGP or DNS?

    Microsoft Outage Outrage: Was it BGP or DNS?

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: All of Microsoft’s cloud services go down, everywhere except China. Redmond’s IaaS, PaaS and SaaS—including GitHub—were dead for several hours, and are still running unreliably—despite Microsoft saying it’s fixed the problem. (more…)

  • Get Comfortable with Shifting Right to Improve Resiliency

    Get Comfortable with Shifting Right to Improve Resiliency

    The operations team receives a page at 2:00 a.m. Something isn’t working correctly. Do they know how to fix it? Perhaps more importantly, do they know whether it matters to customers or how it impacts the business? There are different techniques that companies implement to maximize the quality of their software. We talk about shifting left or right, chaos engineering and others. While they each have their own goals to improve quality, there’s an underlying common goal as well: All three techniques increase your team’s knowledge of the system and their ability to resolve problems quickly.

    Here, we’ll walk through the software life cycle and discuss the common challenges to software quality, running complex systems and managing dependencies between teams. The common thread is a proper observability strategy which can help software quality, hand-offs between different teams and resolve problems quickly by ensuring whole-system resiliency from design to maintenance.

    Techniques for Improving Quality

    Shift left is a concept that has attracted much attention recently. The practice revolves around the idea that detecting and correcting problems sooner minimizes their impact. As the thinking goes, they’re easier to fix, they wind up costing less and there are fewer ongoing issues with bad code.

    The rise of DevOps and DevSecOps has propelled the popularity of shift left. Yet, there’s a fact that development teams often miss: no matter how early an organization spots errors, problems will inevitably remain. Early testing can’t catch everything. As a result, some organizations have begun to rebel and move to a shift-right framework. It normalizes errors and aims to better resolve issues across the development process, including post-production.

    Shift right is the perhaps less-extreme cousin to its rebellious relative, chaos engineering. With chaos engineering, team members cause problems on purpose to keep everyone on their toes and test the limits of the system. They shut down servers, disconnect wires, delete files, etc. The goal is to confirm system resilience–from a user perspective, there should be no impact even with all of this chaos. A resilient system means that teams can fix failed components at their leisure during the day. With a resilient system, critical failures are less likely to happen and it becomes second nature to fix them when the dreaded 2:00 a.m. page does occur. Unlike chaos engineering, shift right doesn’t explicitly break things–it just makes the assumption that broken things will exist so teams need to be ready to handle them.

    Both concepts (and yes, even chaos engineering) are valid—but each company needs to carefully analyze which is right for them. One variable to consider is the complexity of the system you’re building. Working on a rocket launch or a surgery robot? Probably should shift left as much as possible. Changing the color of a button? Maybe it’s not the end of the world to verify the fix in production. For most of us, projects fall somewhere between these two extremes–so sometimes shifting left is necessary, and other times shifting right is pragmatic. A careful analysis of the risks should be performed to determine what kind of testing is needed.

    Although teams tend to be pretty good at analyzing the complexity of their software and making development process decisions, they often forget the complexity of other companies’ code which they depend on. Even a simple web application deployed today has tons of dependencies: The server it’s running on, the local ISPs, transit ISPs, load balancer, DNS provider, TLS certificate, CDN and browser, just to name a few! After analyzing the complexities of all of the dependencies across the internet, it becomes clear that finding every possible issue that might impact their application is a daunting task.

    The goal, again, is to find any issues before they reach production. Or if they reach production, fix them before they impact customers. Or if they impact customers, fix them as soon as possible after that. To do any of these things quickly, the level of observability of the overall system is paramount. With simple systems—or even complex systems that are entirely under one company’s control—tools such as application performance monitoring (APM) are common. But when the scope of the system is as big as the internet, internet performance monitoring (IPM) tools are necessary instead.

    With both APM and IPM tools, the goal is to know as much about the system as possible – enabling teams to identify problems quickly, pinpoint the root cause, and fix it. When an organization is equipped to monitor development and operational processes effectively from application to delivery, there’s a shift toward higher quality code, fewer errors, improved security and lower costs for the enterprise. There are also fewer disruptions and distractions for engineers, DevOps teams and others.

    Dealing With Disconnects

    Whether shift left or shift right is used, there will be problems that enter production. There is an invisible wall in many organizations–where engineering teams “throw code over the wall” to the team which is now responsible for running it. In some companies, this is the operations team; in others, it’s site reliability engineering or production engineering–the common trend is that in most companies, it’s not the same team who created the software.

    These operations teams often lack the nuanced information they need to resolve production issues. They might spend hours or even days attempting to track down the source of the problem. There’s often finger-pointing between different teams who created or own different portions of the system. Things can quickly devolve into chaos—and tempers can fray. After all, these kinds of problems always happen at 2:00 a.m., right?

    That’s where end-to-end observability enters the picture. Imagine a framework with proof that the development team has performed sufficient knowledge transfer to the operations team. That the operations team has asked all of the relevant questions about running and maintaining the application. Using the right tools, engineering and operations teams need to monitor every aspect and every component of the system. Creating this monitoring strategy means that everyone involved also builds an understanding of the system–everyone is on the same page. All of a sudden, there is a common language to discuss problems, and teams operating the software in production can take debugging a lot further before they need to ask for help.

    Ultimately, shifting left or shifting right is a personal choice for every organization. Even if every possible test case is executed before deployment to production though, issues will happen–so take the opportunity to implement a robust system-level monitoring strategy that can be used to solve technical questions during that 2:00 a.m. call.

  • Apple Outage Outrage | Linux Random Redo | Okta Hacked (or Not)

    Apple Outage Outrage | Linux Random Redo | Okta Hacked (or Not)

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Why Apple services were down, Linux gets a huge RNG overhaul, and we wonder if Okta was hacked again. (more…)

  • Ukraine Asks for DNS Censorship | Ukraine Snapchat Staff Danger | #BoycottKaspersky?

    Ukraine Asks for DNS Censorship | Ukraine Snapchat Staff Danger | #BoycottKaspersky?

    Welcome to Ukraine Week here at The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Censoring the Russian internet, Snap fears for hundreds of Ukrainian employees, and Eugene Kaspersky acts like a ****.

    1. ICANN, but IWONT

    First up this week: Ukraine has asked ICANN to drop Russia from the DNS. Andrii Nabok, the Ukrainian representative of ICANN’s Governmental Advisory Committee argues a DNS blockage would “help users seek for reliable information in alternative domain zones, preventing propaganda and disinformation.”

    Given today’s confirmation that Muscovite K12 students had been arrested for laying flowers at the Ukrainian embassy, this goal seems all the more laudable. But frankly, a moment’s thought says it’s a bad idea.

    Analysis: “The net interprets censorship as damage and routes around it”

    John Gilmore was right. Such a move would inevitably fail. And there’s a real risk of undesirable, unintended consequences.

    Kat Bouza and Noah Shachtman: Ukraine Pushes to Unplug Russia From the Internet

    Ukraine’s request to the Internet Corporation for Assigned Names and Numbers (ICANN) seeks to revoke domains issued in Russia and shut down primary Domain Name System (DNS) servers in the country. [It] would send a strong message to a regime pumping out digital propaganda.
    …
    The DNS root zone is a crucial element of the overall functionality of the internet, responsible for handling queries to top-level domains — such as .com, and country-specific domains like Russia’s .ru. Removing Russia’s access to this cluster of servers would prevent Russian internet-service providers from communicating.
    …
    But … the unprecedented request has the potential to do more harm than good. … Representatives for ICANN … declined to comment.


    But the request is pretty naïve, thinks this Anonymous Coward:

    All they would have to do is get the ISPs to add a zone to their DNS that says “ru. IN NS x.x.x.x” and then for all their users, any requests for a .ru domain would be directed to x.x.x.x

    OK, the rest of the world would miss out, but I doubt that they would care.


    Is there a better way? ArmoredDragon breathes fire:

    A better way … is to ask western certificate authorities to revoke all [Russian] certificates. Then ask the major OS and browser vendors to remove any CAs that don’t comply.

    That would really **** with them because there’s nothing [Russia] can do about that. And it would even **** with Russia’s ability to do business with Chinese companies and Chinese customers.


    2. Fear for 300 Ukraine Employees at Snap

    Snap, Inc.—formerly Snapchat—owns the Ukrainian company that created the technology used in the real-time video shenanigans behind “I’m not a cat.” It has hundreds of employees in Kyiv and Odesa.

    Analysis: Solidarity doesn’t Stop Wars, but Sanctions Might

    This very human DevOps story is but a footnote to the wider picture of tech companies acting to echo Western governments’ sanctions. It’ll cause us a little pain, but that’s nothing compared to the suffering felt by Ukrainians. And it’ll be worthwhile if it shortens the war.

    J. Clara Chan: Three hundred Snap employees are from Ukraine

    In a public note, … Snap — the parent company of Snapchat — said … that 300 of Snap’s “most creative and talented team members” are from Ukraine, … the birthplace of Looksery, the augmented reality platform that Snap acquired in 2015. … Snap said it has pledged over $15 million in humanitarian aid to support Ukraine. … The company is also providing emergency assistance to employees who need help safely relocating.
    …
    “We stand in solidarity with our Ukrainian team members and the people of Ukraine. … This is a devastating conflict with far-reaching consequences. … It is a direct threat to many of our team members and their families.
    …
    “Over many years working with our Ukrainian colleagues and friends, we have gotten to know the inventiveness, strength, and resilience of the Ukrainian people. … We will continue to work tirelessly to help our Ukrainian team members and their families, support our Snapchat community, and determine how we can help make positive contributions to the immense humanitarian efforts underway.”


    It’s bang on trend, reports Corinne Reichert:

    Snapchat has followed suit after Facebook, Twitter and YouTube began pausing ads on Friday, saying it has now stopped all advertising running in Russia, Ukraine and Belarus following the Russian invasion of Ukraine. … The Snapchat app remains live in Ukraine, Russia and Belarus, [but] Snap is monitoring the platform for misinformation or other misuse.
    …
    Apple has also halted sales of its products in Russia, as well as online transactions, Apple Pay and some Apple Maps features. Google [is] temporarily disabling some live Google Maps features, adding an SOS alert to Search and blocking Russian state-controlled media. It said Google Pay “may become unavailable.”


    But don’t hang up on Dade “+++ATH” Hayes: [ask your parents]

    In terms of any financial sacrifice from the foregone advertising opportunities, it isn’t clear that all that much is at stake for Snap. … While the company’s stance on the Russia-Ukraine situation clear and emphatic, it is also an easier one for for Snap to take than for many of its tech rivals.


    3. Yevgeny Valentinovich ‘Eugene’ Kaspersky has Chosen the Side of the Oppressor

    The figurehead eponym of AO Kaspersky Lab is being publicly shamed for comments he made about the war in Ukraine. In stark contrast to Snap’s statement, he is accused of overt neutrality, minimizing the war’s severity—and even of shoehorning in some marketing.

    Analysis: #BoycottKaspersky

    Desmond Tutu was right. But, while it’s tempting to “cancel” Eugene, dial down the emotion and think instead about your security posture. If you use Kaspersky products internally, what risks are you exposing your organization to? How might the Russian government exert pressure on the company to compromise your network?

    James Coker: Eugene Kaspersky’s Statement Provokes Controversy

    Cybersecurity industry veteran Eugene Kaspersky has provoked a strong reaction from leading figures in the sector. … Kaspersky, CEO of the Russian-headquartered IT security vendor of the same name, broke his silence on the conflict:
    …
    “We welcome the start of negotiations to resolve the current situation in Ukraine and hope that they will lead to a cessation of hostilities and a compromise. We believe that peaceful dialogue is the only possible instrument for resolving conflicts. War isn’t good for anyone. … The main thing we can do in this situation is provide uninterrupted functioning of our products and services.”
    …
    His description of the conflict as a “situation” and apparent attempt to advertise his firm led to a number of furious responses. … Following Kaspersky’s tweet, the hashtag #BoycottKaspersky appeared.


    With a balanced view, it’s Nick Farrell:

    Kaspersky sits on the fence … in Russia it is not a good idea to have a view. [But] whoever is advising Kaspersky on PR might need to have a rethink.
    …
    After all a compromise is [not] always possible when one side is firing missiles into residential areas. If he applied the same premise with ransomware, he would be suggesting, … “It is better that you talk to the people who are holding your servers hostage.” … How much is Kaspersky, the company, influenced by the Russian government?


    Does your shop use Kaspersky? Nick Lange cuts to the chase:

    I would like to remind all readers that, in case of war, … software with access to companies and the capacity to be updated worldwide in matter of hours at the same time is a high class target to get forced to cooperate by a autocratic government.


    The Moral of the Story: If an elephant has its foot on the tail of a mouse and you say that you are neutral, the mouse will not appreciate your neutrality.


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Brandon Morales (via Unsplash; leveled and cropped)

  • NS1 Touts a Common Delivery Platform for Devs and Ops

    NS1 Touts a Common Delivery Platform for Devs and Ops

    NS1 has extended the capabilities of its DNS and network management platform with the release of NS1 Connect. The platform is designed to serve as a delivery platform that integrates developer and network operations teams’ tasks for the creation and management of applications across highly distributed cloud-native and on-premises environments.

    While DevOps teams typically rely on dozens of tools for the deployment of software and the management of the network connections on which the software runs, NS1 was created to—among other things—reduce the number of different tools DevOps teams require by improving visibility, automation and management for these processes.

    In this way, NS1 Connect provides a unified API framework to automate DNS, traffic optimization and network services management across heterogeneous networks, Sanjay Ramnath, vice president of products at NS1, told devopsy.kinsta.cloud. NS1 Connect’s reach covers internal and external applications deployed via on-premises, cloud or hybrid infrastructure. This enables NetOps and DevOps teams to integrate NS1 solutions into their infrastructure and CI/CD pipelines with lower complexity and enables NetOps teams to accelerate management and minimize the risk of misconfiguration, Ramnath said. “Prior to this, teams had to deal with complex manual processes or disparate API frameworks that slowed down their ability to develop, deploy and deliver applications.”

    In his talk, Your Business Depends on Eliminating Boundaries Between Applications, Users, Infrastructure, and Data during INS1GHTS 2021, NS1’s annual user conference, Ramnath described how a number of NS1 products and services are integrated into a single platform under the NS1 Connect umbrella. They include cloud-native network services for DNS, DHCP and IPAM (DDI), edge-to-cloud networking for NS1’s managed DNS, dedicated DNS, private DNS and DNS for China and application traffic automation for VPN traffic steering, pulsar active traffic steering and general traffic management. Other commonly used network services are provided as well, including IP address management that can be centralized across the network, said Ramnath.

    The idea was to remove many of the roadblocks DevOps teams face when seeking to deploy on highly distributed infrastructures. “The goal here is to remove some of those boundaries and barriers that are talked about on the infrastructure side,” says Ramnath. “[This helps to make sure] that our control plane is centrally provided and centrally consumable as a cloud solution and that our products can be deployed wherever they are needed in a zero-touch way, in a lightweight fashion, making sure that we are aligned with cloud-native operations.”

    The need to maintain low latency across distributed infrastructures is also key. As David Coffey explained during the conference in The Future of Application Traffic Intelligence and Automation session, low latency is essential for creating “that great experience for users.”

    “Low latency is not an attribute of this feature, it is a top, tier-one priority for people to accomplish, but it’s an incredibly complicated aspect, given all of the frameworks that are out there—the tools, the DevOps capabilities and the components required to support and deploy these modern-day applications,” said Coffey. “That’s what enables application companies to deliver that low latency and treat it as an attribute in the system; not having to go out of their way to create this. We want to enable all developers to embed this into their system—and we can provide them with a degree of success for delivering that great experience to their customers and to their networks.”

  • Building Amazing Apps, Part 3: Optimizing the Network

    Building Amazing Apps, Part 3: Optimizing the Network

    This is part three of a series of three articles on building and delivering amazing apps at the edge. Read part one here, and part two here.

    Introduction

    There are millions of apps available today, and it’s incredibly easy for any user to download and install any of them with just a few taps on a screen or click of a mouse. If you want to build and deliver an app that will amaze users, you need to be able to differentiate from the rest of the pack.

    Over the past 14 years, I’ve been working closely with the top companies delivering the most popular apps, helping them optimize their apps for speed and scale so users get a great experience. In this three-part series, we’ll explore the fundamental steps you must take to build and deliver your own amazing app at scale that will keep users coming back for more.

    We’ll break these steps down into three key areas:

    • Front end: Covers the parts of the app that end users interact with.
    • Back end: Looks at the required origin infrastructure to support the app, including databases, application, web servers and more.
    • Network: The glue that connects the front end and the back end.

    In the final article of this series, we’re going to explore four things you can do on the network to optimize your app and make it more amazing:

    • Distributed DNS
    • Protocol optimization
    • Latency optimization
    • Edge computing

    Distributed DNS

    The Domain Name System (DNS) is a critical part of making sure your clients can access your applications (in fact, it’s a very popular attack vector by hackers today because if your DNS is down, nobody can access your PPP). Distributing your DNS across many servers can help ensure that your DNS (and thus, your app) is fast and highly available, so you should use a DNS vendor that has a large DNS server network in a wide range of locations.

    Validating whether you have a properly distributed DNS infrastructure is rather easy. Just follow these steps:

    • Identify how many name servers your domain uses and their IP addresses. Ideally, you want a number between three and 12—the more the better. You can use command-line tools such as dig and nslookup to identify the name servers of a given domain.
    • Use an IP geolocation system to find out the physical location of the name servers. At this point, you can get a rough idea if those servers are close enough to your end users. Hint: Some CDNs offer diagnostic tools that can be scripted via APIs to automate IP geolocation.
    • If you want to be more thorough, you may want to run connectivity tests between the main locations of the users and the name server IP addresses to make sure the network latency—i.e., how long it takes for information to go back and forth between the client and the server—stays under 100 ms (the lower the better). Some internet providers and CDNs provide looking glass technology, which enables you to run connectivity tests that originate from servers around the world.

    Protocol Optimization

    Once your clients know where to connect, how they connect and the protocol used is something that shouldn’t be neglected. HTTP/2 is the modern, superior network protocol, as it uses a more efficient binary format, header compression, single network connection, stream multiplexing and server-push. Actually, I was very involved in the early years of HTTP/2 (the standard was approved in February 2015) and have since helped some of the largest websites on the internet implement HTTP/2 and still spend a considerable amount of my time testing the new protocol in all sorts of environments. I’ve shared my findings in talks at conferences including Velocity and in the “Learning HTTP/2” O’Reilly book, which I co-authored. In a nutshell, most sites can improve page load performance between 5% and 15% by just switching the protocol from HTTP/1.1 to HTTP/2, which is as simple as flipping a switch if your website uses HTTPS.

    To illustrate that improvement, I created a demo site that tests the download speed of many small images over HTTP/1.1 and over HTTP/2 and displays the differences.

    Figure A

    As you can see in Figure A, the content loaded eight times faster over HTTP/2 than over HTTP/1.1 on my mobile phone over an LTE connection. That’s a huge improvement considering how simple it is to implement HTTP/2.

    According to the HTTP Archive, as of June 2018, 20% of hosts support HTTP/2, but those hosts are responsible for 40% of all requests—meaning HTTP/2 powers some of the web’s most popular sites.

    However, as explained in the HTTP/2 Performance chapter of the Learning HTTP/2 book by O’Reilly, there are a lot of factors that can impact the benefits of HTTP/2. Some of them you can control, while others—such as network packet loss—are much more difficult to tackle and can impact HTTP/2 performance because HTTP/2 runs over a single TCP connection.

    Note: TCP stands for Transport Control Protocol and some parts of its implementation (e.g., slow to start and cumbersome network congestion algorithm) can impact HTTP/2 performance because all the communication with the server goes through a single TCP connection.

    The future of the web is HTTP/3, which uses HTTP/2 over QUIC (a newer and more efficient transport layer protocol). However, as of April 2019, HTTP/3 is still in draft state and hasn’t been approved as a standard.

    Ideally, your servers should monitor the network connection and use different network protocols depending on the current conditions. Certain content delivery networks (CDNs) offer this functionality. For example, to maximize throughput, a top CDN provider uses custom software on its edge servers to dynamically use different network protocols, TCP settings and congestion control algorithms, depending on the specific conditions of each individual connection the server makes with an end user. In short, using the optimal protocol makes your app faster and more amazing.

    Latency Optimization

    Latency is a physical limitation of nature that impacts the throughput on computer networks. You probably know that whenever you are outdoors and there is a storm with lightning, you can count the number of seconds until you hear the thunder to identify how close the storm is (and if you count less than a second, you better get inside!). In computer terms, latency is measured as the time it takes to send a network packet back and forth between two systems. This concept is known as round-trip time (RTT) and is typically measured in milliseconds. You can use network tools such as ping or mtr to easily measure the RTT between two systems.

    Latency has a huge impact on performance because the further away you are from the server you want to connect, the slower the communication will be. As a rule of thumb, you want to ensure your latency is well under 100 ms. The easiest way to achieve that is by leveraging the local device cache to avoid extra network connections to the server and using a CDN to put your server closer to the end user.

    Here is a graph (see Figure B) showing that latency plays a larger role on page load time than bandwidth. As you can see, doubling the bandwidth from 5Mbps to 10Mbps results in just 5% improvement in page load times, and any bandwidth increase over 10Mbps doesn’t make a big difference. Reducing the latency, however, drastically improves the page load times. That makes your app more amazing for your end users.

    Figure B

    Edge Computing

    If you are a developer or a DevOps engineer (or just love tech) chances are you are within the TOP 1% of world population in terms of mobile computing power. But things you take for granted, such as resizing images on a mobile phone, are going to take much longer for many millions of internet users who have less powerful mobile devices.

    So, what can we do if running logic at the origin causes delays because of the network latency and running logic on the client side may be too slow on less powerful devices? The solution is edge computing, which consists of moving the origin logic to a server close enough to the end users that latency does not impact performance. CDNs are in the best position to offer this proximity because of their natural distributed architecture; a CDN can set cookies, rewrite HTML and resize and deliver optimized images for the screen size of your users so those images don’t take a long time to display on low-end mobile devices with limited CPUs.

    Edge computing is the next step beyond cloud computing. You should try to use a CDN company that offers you the ability of running business logic as close as possible to where your users are, to ensure that your app is fast and can scale without impacting your infrastructure.

    Summary

    In this series of articles, we reviewed some basic things you should consider if you want to build and deliver amazing apps at scale. This is not a comprehensive list of all the things you can do but provides you with some of the most impactful areas for app improvement.

    — Javier Garza

  • Today’s DNS: DevOps’ New Best Friend

    Today’s DNS: DevOps’ New Best Friend

    Continuous development and deployment is a primary focus of DevOps teams. As part of this process, DevOps teams also require the automatic creation or removal of networks, and automated deployment to production. Automation is a closer guarantee that the process will be done right the first time because the fewer the number of manual steps involved, the less likelihood for human error. Intelligent DNS improves DevOps project life cycles by integrating DNS tasks into your software development and automation tools.

    DNS Needs an Upgrade

    The internet has come a long way since its humble beginnings, from the Advanced Research Projects Agency Network (ARPANET) of the 1960s to an entity that is mission-critical for the world’s commerce. This system is now tested and taxed in ways that were not even conceived of when it was originally designed. One of these early elements, DNS, is now a potential bottleneck for network performance in general and DevOps teams in particular.

    One of the benefits of cloud computing is that it enables DevOps teams to automatically create and remove servers as needed for development and testing. Infrastructure as code (IaC) enables cloud computing to build infrastructure automatically using robust APIs that build servers and add applications as needed. Traditional DNS, with its mostly manual management, becomes a bottleneck for this automated rapid configuration of infrastructure. This is why organizations need an intelligent DNS solution. Intelligent DNS with a robust API can seamlessly interface with current DevOps software, removing the bottleneck that traditional DNS can become.

    Extending Agile with DevOps

    The underlying ideas of agile programming are extended with DevOps. Rather than the traditional way of producing and testing the code, then tossing it over the wall to the network operations team to deploy, DevOps merges the development, testing and operations responsibilities. In short, DevOps should have the ability to break the silos within your IT environment and among the development, QA and operations teams.

    DevOps’ blend of cultural philosophies, practices and tools increases an organization’s ability to deliver applications and services at high velocity. DevOps allows organizations to evolve and improve products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market.

    Rapid Change Propagation is Essential

    In addition to rapidly delivering applications and services, a goal of DevOps continuous release is to remove bottlenecks to the development, testing, and deployment cycles. This is aimed at creating an environment where the work isn’t hampered with computer delays and frustrations.

    When you’re automating the creation or deletion of 100 virtual servers with a single mouse click using IaC, waiting for 30 minutes to see if DNS changes appear at the edge servers delays your project. Your DNS needs to move as fast as your IaC software and service discovery processes. At this point, intelligent DNS with rapid change propagation becomes essential. When choosing DNS vendors to support your DevOps environment, make sure to include speed testing for the network’s propagation for changes.

    Where IaC Comes In

    The aim of DevOps professionals is to create and sustain server environments that closely resembles the way software developers build and maintain application source code. IaC is the way DevOps teams make that happen.

    IaC is based on three primary constituents:

    • Unit/Integration Testing – Lets you validate your infrastructure code within various phases of the DevOps pipeline and lets you feel confident about what you are pushing to production.
    • Infrastructure Blueprints – Supports the reusable, consistent and rapid deployment part of IaC.
    • Version Control – Facilitates the method track and rollback changes to your infrastructure as needed.

    Infrastructure blueprints can be created using configuration as code (CaC). Using tools such as Terraform, Ansible and Chef allows DevOps teams to build these blueprints by enabling a declarative way of handling infrastructure configuration. Infrastructure blueprints should always go hand in hand with unit/integration testing. Having a robust, API-driven DNS network to couple the full automation of DNS changes alongside the infrastructure and configuration changes ensures a smoother process.

    IaC is next-generation technology for DevOps environments. When combined with CaC products, it changes the way software is built and delivered. Continuous integration and continuous deployment are a large part of the DevOps world that all still depends on DNS technology to connect everything on the network.

    The Role of Service Discovery

    Service discovery is needed to enable your apps or servers to identify which services are available on the network and which IP addresses and ports are associated with them. Previous methods of tracking this information no longer work in the DevOps age.

    Since modern infrastructure is so scalable and changes so quickly, keeping track of service information inside a periodically updated database doesn’t work anymore. Instead, you have to be able to discover services in real time and update your service information continually. Three common service discovery tools are: Consul, Apache Zookeeper and Etcd (with Registrator and confd).

    A tight integration with your managed DNS provider as part of your overall strategy means that service discovery can work hand in hand with the DNS configuration so both systems remain in sync.

    Moving Toward Intelligent DNS

    DevOps needs the same types of robust features you would want for your external internet traffic: end user-facing DNS. You should align with a vendor that can provide the flexibility, agility and reliability you require to deliver your applications.

    Look for features including:

    • Zone Transfers allowed and easily facilitated
    • An anycast DNS network
    • A robust API to support automation, IaC, service discovery and other architecture and tool needs
    • Native integration with common DevOps tools such as Terraform, Ansible and Chef
    • An SLA that meets your needs for reliability and performance

    These elements are necessary, but a more important item on your checklist needs to be redundancy in all layers of the stack, including DNS. Because no vendor is completely bulletproof, use at least two authoritative DNS providers to set up redundancy. Following these guidelines will improve your DevOps’ life cycle by using intelligent DNS.

    About the Author / Kimberly Lacerte

    Kimberly Lacerte has 14 years of experience in computer and technical writing. She is currently the Content Marketing Manager at NS1 in Bedford, N.H. Previously, she has written for Veritas, Symantec, Health Dialog and Dyn. Kimberly holds an English BA from UNH and an MBA from Southern NH University.