devopsy.kinsta.cloud

Author: Barry Phillips

Are You Paying a ‘Wait-Time Tax’ on Your Global DevOps?
Business increasingly is global, so DevOps has become global as well. In fact, for many enterprises, DevOps entails collaboration between development, test/QA and ops teams across multiple continents.

This is in part due to the economics of IT labor and in part because applications for global enterprises have to be deployed globally.

Unfortunately, enterprises executing DevOps globally must confront the physical realities of global code distribution. And those physical realities impose a “wait-time tax” that fatally undermines Agile processes and culture.

But they don’t have to. There’s a tax-free way to share large volumes of code across globally dispersed DevOps teams.

The Wait of the World

Every time you ship a massive build worldwide, you have to wait. Assume, for example, that you’re sending a 10GB build to five different locations over a 10Mbps MPLS connection. If a couple of your remote locations have 5Mbps connections, it will take them five hours to get each build. But if you have three locations with only 2Mbps connections (as is likely the case in Asia), they will have to wait 12.5 hours.

That wasn’t a problem when you iterated your builds with limited frequency. But as you try to become more aggressive about highly iterative Agile development and testing, this “time tax” becomes punishing.

For one thing, you lose productivity as your overseas developers and test teams wait idly for new artifacts. And these chronic wait times fundamentally undermine the Agile culture you’re trying so hard to build across your organization.

In fact, if your current model for distributing code globally remains as-is, you’ll never achieve the speed and iterative frequency your business needs. And you’ll certainly never come anywhere close to continuous delivery.

Faster Cycles from Dublin to Dubai

Fortunately, you can eliminate DevOps wait times, regardless of how globally dispersed your teams are and how thin their connections to your enterprise network may be.

The solution entails a hub-and-spoke model that:
- Lets you maintain a “gold copy” of your current codebase(s) in the cloud;
- Provides all your remote locations with their own local codebase copies that get continuously and automatically updated with any incremental changes as they occur; and
- Leverages each location’s high-throughput internet connection.
The combination of small, incremental code updates and high internet bandwidth dramatically changes the rules of the global DevOps game, allowing even the most geographically dispersed teams to work with even the largest codebases extremely iteratively and without delay.

It’s difficult enough to sync dev and test teams on different continents. And the pressures on your organization to respond immediately to ever-changing market and regulatory demands will only intensify. So now’s the time to consider abandoning your obsolete approach to global code distribution and adopt a model that better enables you to be fast, super-iterative and global.

— Barry Phillips
January 3, 2017
Infrastructure’s Impact on Shifting Left

If you lead big honkin’ software development projects at a big honkin’ enterprise, you’re probably doing everything you can to shift QA left. That’s because the earlier you discover defects and shortfalls, the faster and less expensively you can bring great software to market.

Unfortunately, above a certain scale, it takes more than just better processes and a more aggressive culture to shift test and QA left. It also requires a different approach to the infrastructure you use to distribute your builds.

Infrastructure and the Problem of Scale

Most software developers understand the scale issue primarily as one of complexity. The bigger the project, the greater the chance that something won’t work with something else exactly as planned.

But large software projects also canget badly impeded by the sheer physics of build distribution.

Take, for example, a 10GB build that you have to distribute to five different domestic and offshore locations from your primary facility over your company’s 10Mbps MPLS connection. If your two of your locations are domestic and have 5Mbps connections, it will take five hours to get them each build. If the other three are offshore and have 2Mbps connections, it will take them each 12.5 hours to get their copies of the build.

That’s a “time tax” you’ll have to pay again and again as you keep having to ship massive files over the network to your globally distributed test/QA team. Worse yet, the more Agile and iterative you get, the more you keep paying this “time tax” over and over.

So hours expand into days, and days into weeks—constantly frustrating even your best efforts to more aggressively shift left.

Avoiding the Scale-Driven Time Tax

Here’s how you can successfully avoid the time tax that chronically prevents you from shifting your biggest projects further left.

Instead of continuing to use IT infrastructure that was never designed to support time-compressed software development at scale, you can instead implement a hub-and-spoke build distribution system that virtually eliminates network-related delays.

This hub-and-spoke model lets you maintain a “gold copy” of your current build(s) in the cloud—while providing all your remote locations with their own local copies that get continuously and automatically updated with any changes as they occur.

This model eliminates network-related bottlenecks while allowing your geographically dispersed teams to collaborate without tripping over each other’s work. As a result, you can shift left much more aggressively.

Of course, infrastructure decisions typically aren’t made by the same folks who lead software development. So the two groups have to collaborate closely to make sure the network doesn’t impede critical development processes.

But both parties have lots to gain—because hub-and-spoke saves infrastructure owners lots of money in storage, bandwidth costs and network acceleration hardware at the same time as it empowers development to shift further left.

That’s why DevOps leaders and enterprise infrastructure owners need to work together ASAP. The digital success of the business depends on it!

— Barry Phillips

October 28, 2016
Big DevOps: Too Slow to Win?

There’s DevOps. And then there’s Big DevOps.

DevOps is a set of disciplines that accelerates time-to-value for digital deliverables. It is being embraced by all kinds of companies, from pure-play digital market disrupters to mainstream corporates trying to maintain competitive parity with their mobile apps.

Big DevOps is something else entirely. Big DevOps resembles mainstream DevOps, because it’s a process by which companies bring code to market. But if you’re a game developer or large bank, your code bases are massive—and your software life cycle requires you to distribute those massive code bases across globally dispersed dev and test teams.

Distribution of massive code bases on a global scale creates time-to-value challenges that are totally foreign to programmers checking bite-sized blocks of work in and out of GitHub. These scale-related distribution challenges chronically delay DevOps workflows—impeding your company’s digital agility and seriously undermining your ability to compete in today’s time-gated markets.

So you must overcome your Big DevOps code base distribution problem.

Choking on Code

The main culprit responsible for the slowness of Big DevOps is the network. It just takes too much time to push a massive code base through the relatively narrow pipe of an enterprise network. Even if you spend lots of money on bandwidth and network acceleration, the ratio between your bits of code and your bits-per-second of network is untenable. Every distribution of every large code artifact gets delayed. Over and over, day in and day out.

Those days add up. And when you pile days lost due to network-related delays on top of the other delays that plague large, complex software projects, you wind up persistently late to market.

And cloud alone can’t help you. Globally dispersed teams can’t work on a massive code base in the cloud. They have to work with it locally. So you have two problems. First, all your teams all over the world have to keep downloading and uploading massive files. Second, you have to make sure everything everybody does everywhere stays synced.

That’s why you need a much better way to globally distribute massive code bases.

Work Locally, Dedupe Globally

That better way is a hybrid hub-and-spoke model that lets you maintain a “gold copy” of your code base in the cloud—while giving everyone everywhere a local copy that is continuously updated with any changes to the current build. This approach eliminates network-related delays while ensuring conflict-free collaboration across geographically dispersed teams.

It’s a simple approach that works. In fact, it’s been used by CAD/CAM teams for years to address a very similar file distribution problem.

Hybrid cloud hub-and-spoke also saves money—because you can spend less on your network and your local storage infrastructure.

Cost, however, is not the existential issue facing Big DevOps. Time is. Companies that are slow to market with their digital deliverables can’t win against fast ones. So if your company is big—and if you’re doing Big DevOps—you have to solve your time-to-market problem. The realities of digital competition demand it.

— Barry Phillips

September 29, 2016