Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: No remedy for banned Facebook users, new ARM-based VMs on Microsoft Azure, and The Great Resignation will still be a Thing for the foreseeable. (more…)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Google’s FLoC proposal is dead, Meta/Facebook is buying RSC—a huge AI supercomputer, and Arm “will IPO” instead of selling to Nvidia.
First up this week: Google bows to the inevitable. Third-party cookies will soon go away, because people are fed up with being tracked. But Google’s FLoC proposal wasn’t the answer.
Google doesn’t want to invade your privacy. But Google does want to sell well-targeted ads. The key difference between FLoC and Topics is that FLoC buckets were automatically generated—and hence opaque, risking overlap with “protected categories”—e.g., race or gender. But the topic classifications in the Topics API are curated and have user-visible, descriptive names.
Frederic Lardinois: Google kills off FLoC, replaces it with Topics
Google’s controversial project for replacing cookies for interest-based advertising … is dead. In its place, Google today announced a new proposal. … Your browser will learn about your interests as you move around the web.
…
When you hit upon a site that supports the Topics API for ad purposes, the browser will share three topics you are interested in. … The site can then share this with its advertising partners to decide which ads to show you. … Users will be able to review and remove topics from their lists [or] turn off the entire Topics API.
…
It’ll remain to be seen if other browser vendors will be interested in adding the Topics API. [But] they all quickly turned a cold shoulder to FLoC. … The plan is to start trialing the Topics API at the end of this quarter.
A minor tweak and rename? “Yes and no,” equivocates mananaysiempre:
Yes, it still amounts to building user-tracking functionality into user agents. No, it’s not the same approach.
…
FLoC was doing unsupervised clustering of users, whereas Topics allocates users into predetermined clusters. This makes it potentially more transparent and also seems designed to work around the “protected categories” objection.
But Google has a long way to go to convince garett_spencley:
I’m glad FLoC is being ditched, but as an end-user I’m still not asking for this Topics nonsense. It doesn’t solve any problem that I have. It solves problems that the advertising industry has.
…
I’ll choose to use browsers that don’t support this.
Zuckerberg’s new toy is RSC: A $400 million Nvidia-based AI supercomputer. And that’s just Phase One.
If you rely on Open Compute Project systems for your DevOps, know that it’s no longer the bee’s knees over at Hacker Way. Meta/Facebook is investing eye-watering sums into a custom configuration of off-the-shelf hardware.
Timothy Prickett Morgan: Meta Buys, Rather Than Builds And Opens, Its Massive AI Supercomputer
If you thought it took a lot of compute and storage to build Facebook’s social network, you ain’t seen nothing yet. … To build the Metaverse … is going to take an absolutely enormous amount of supercomputing power.
…
But what is a surprise, given Facebook’s more than decade long focus on … opening up its hardware designs through the Open Compute Project, is that the … Research Super Computer [RSC] system is built from … commercially available … servers, storage, and switches. [It] probably comes down to … working a deal with some vendors who want the Facebook publicity and are willing to work the price and the parts availability.
…
[So Meta bought] 760 DGX A100 systems from Nvidia—which each … have a pair of AMD “Rome” 64-core Epyc 7742 processors … 1 TB of main memory … eight A100 GPU accelerators [and] eight 200 Gb/sec HDR InfiniBand controllers—all-flash storage from Pure Storage, and … caching servers from Penguin Computing. … At list price this machine would cost around $400 million … Phase one of the RSC machine is rated at 59 petaFLOPS … (FP64), and 118.6 petaFLOPS FP64 using the Tensor Core matrix engines. … With INT8 processing on the Tensor Cores, which is needed for AI inference, the phase one machine is rated at 3.79 exaOPS.
How much??? It doesn’t seem like news to kleiba:
I used to work at a university where my professor had been in automatic speech recognition for a long time, but basically gave up on that line of research about 10 years ago because he figured that universities simply cannot compete budget wise with the big industry players. I suppose the same will soon be true for most ML-related areas of research sooner or later.
…
Already, a substantial amount of research innovation in [Natural Language Processing] and [Computer Vision] has been coming from big companies in recent years. Of course there is a discussion to be had about what that means for society at large.
But 3.79 exoOPS? That’s the way the wind is blowing, according to sungazer:
Give it 15 or so years—this level of computation will be on your desktop. If desktops still exist.
…
Or desks.
Speaking of Nvidia, as I’ve said before, the Arm merger really shouldn’t go through. And it’s looking less and less likely. ARM chips are an increasing fixture in the data center—especially those that value “performance per Watt.”
A veritable hornets’ nest of Bloomberg journalists have put their names to an anonymously-sourced report. The story argues that the proposed Nvidia buyout of Arm from SoftBank Group is dead. Next stop: IPO.
Ian King, Giles Turner, Peter Elstrom, Dina Bass, David McLaughlin, Ruth David and Dinesh Nair: Nvidia Quietly Prepares to Abandon … Arm Bid
According to people familiar with the matter … Nvidia … doesn’t expect the transaction to close. … SoftBank, meanwhile, is stepping up preparations for an Arm initial public offering.
…
If Nvidia manages to get the deal over the line, it would be a massive coup. … But it will be an uphill fight. … The world’s biggest tech companies rely on Arm technology, and they fear they could lose unfettered access under Nvidia.
…
Within SoftBank, there are factions. [And] the ordeal has created divisions within Nvidia. … “We remain hopeful that the transaction will be approved,” a SoftBank spokesperson said in an emailed statement.
Post-Brexit Britain would no doubt like to keep Arm independent. Here’s simonh:
Arm has I think ⅔ of its employees in the UK—mainly Cambridge—and they’re expanding the campus there. The Semiconductor Physics Group at the Cavendish Laboratory at the University is at the cutting edge globally with 18 PhD projects currently.
…
Guess where scads of Arm’s top scientists are recruited from? Arm and the University are joined at the hip. That company’s not going anywhere.
More broadly, it’s good news, thinks hatchet:
Hopefully this means NVIDIA will focus on RISC-V instead. This will be good for everyone.
You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.
Image: Reed Geiger (via Unsplash)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: The return of the CEO who fired 900 staff on a Zoom call, Capitol Hill has a bill to kill ads, and a last-minute hiccup in the FAA/FCC 5G truce.
First up this week: Better.com, the DevOps-led mortgage lender, has its old CEO back. You probably remember Vishal Garg—he was the character who laid off 9% of his company on an open Zoom call.
Mr. Garg is being asked to help build “a respectful workplace.” But the famously combative CEO recently called investors “sewage,” threatened remaining employees with a “bloodbath,” and allegedly told a former business partner he would “burn him alive.”
Riley de León: Better founder Vishal Garg … returns as CEO
The move comes less than two months after Garg came under fire for laying off roughly 900 employees, or 9% of its workforce, via Zoom and subsequently stepped back at the request of Better’s board of directors. … According to an internal memo: … “CEO Vishal Garg has been taking a break from his full-time duties to reflect on his leadership, reconnect with the values that make Better great and work closely with an executive coach. … We are confident in Vishal and in the changes he is committed to making to provide the type of leadership, focus and vision that Better needs.”
…
The memo also reveals that board members Raj Date and Dinesh Chopra have resigned, though “not because of any disagreement with Better.” … Garg [had] cited market efficiency, performance and productivity as the reason behind the firings. [He later] accused the employees of “stealing” from their colleagues and customers by being unproductive and only working two hours a day.
…
In May, Better announced that it would go public. … Since then, multiple outlets have reported that Better has delayed its listing plans amid ongoing scrutiny.
With a reminder, here’s Emma Goldberg:
“If you’re on this call you are part of the unlucky group that is being laid off,” Mr. Garg told his workers [via Zoom]. “Your employment here is terminated effective immediately.”
…
Better.com has since conducted a “thorough, independent” review of its culture [and is] recruiting a new chairman for the board, a president and a chief human resources officer. … Some of the additional measures … include a training program on building “a respectful workplace” and a new ethics and compliance committee, reporting directly to the board.
Don’t hold back—tell us how you really feel, @PCSAemilianus:
What an utter pile of bull**** from this guy. He knew damn well what he was doing and didn’t care. You don’t “reflect” on that.
Dems want to ban targeted ads. A new bill would outlaw the sort of adtech that Google and Meta/Facebook have turned into an art form. It would also put the kibosh on data brokers’ business models.
This will be an uphill battle. But it’s one that’s worth fighting: Surveillance capitalism has gotten normalized, but if consumers fully understood the extent of the privacy invasion, they’d rise up with flaming pitchforks in hand.
Taylor Hatmaker: New privacy bill would put major limits on targeted advertising
The Banning Surveillance Advertising Act, introduced by Reps. Anna Eshoo (D-CA) and Jan Schakowsky (D-IL) in the House and Cory Booker (D-NJ) in the Senate, would dramatically limit the ways that tech companies serve ads. … Targeting based on “protected class information, such as race, gender, and religion, and personal data purchased from data brokers” would be off-limits.
…
Sen. Booker called the targeted advertising model “predatory and invasive.” … Rep. Eshoo said, “This pernicious practice … fuels disinformation, discrimination, voter suppression, privacy abuses, and so many other harms.”
…
The bill would empower the FTC and state attorneys general to [levy] fines of up to $5,000 per incident for knowing violations.
Trying to scan the bill, it’s Sam Biddle:
A federal ban on surveillance advertising sounds like an extremely radical long shot. But I think the only people who would genuinely oppose it are the relatively small sliver of the population who make their money conducting the surveillance.
…
If it goes anywhere, get ready for a barrage of “this will literally murder small business owners” propaganda from FB/Google/et al. [It] is orders of magnitude more significant than any of the wishy washy We Can Reform Facebook! proposals you’ve seen in recent years, in terms of privacy and countering the power of big tech.
Yay. Go GoTeam:
Exactly! It’s the government’s job to secretly track us, not advertisers!(?)
A couple of weeks ago, I told you the FAA backed down in its opposition to 5G NR band 77. Well, a lot can happen in two weeks. It seems the airlines weren’t happy and demanded extra interference-mitigation from the mobile carriers at the last minute.
The various industries involved—and their regulatory agencies—have had years to sort this out. And it smells very much like the FAA is the one failing in its duty.
Theo Leggett: Mobile firms agree another 5G delay at US airports
The much-hyped expansion of 5G networks in the US has been chaotic, to say the least. The rollout has been delayed twice — and now AT&T and Verizon have bowed to intense pressure, agreeing to defer opening some parts of the network near airports.
…
There was clearly time to come up with a mitigation plan — and other countries have been able to do just that. The question is, why were US regulators, telecoms operators, airlines and airports apparently unable to come up with a workable solution?
The FAA is “incompetent,” says David Von Drehle:
5G — the long-promised next step in cellular technology. … It wasn’t a secret. … Yet it seems to have caught the FAA by surprise. … Various compromises and delays [have been] offered by the wireless industry — all met with last-minute panic-mongering by the FAA.
…
The FAA’s foot-dragging raises a red flag over the agency’s competence. … If a challenge as slow-moving and widely publicized as 5G can catch the FAA unprepared, it’s hard to be hopeful about smooth landings in … other areas … such as keeping up with pandemic health measures, battling unruly travelers and finding the next generation of pilots, flight attendants and air traffic controllers.
Is there any doubt? No, opines @K0LWC:
There is no doubt this is a colossal screw up by the FAA, and less so, the FCC. The 5G rollout and the bands being bid on were not a surprise.
You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.
Image: Jan Valečka (via Unsplash)
There’s a lot of interest in GraphQL lately. The query language is excellent for data retrieval, and it could act as a more usable mechanism to evolve the traditional REST API integration approach. But as adoption grows steadily, architects are now considering the implications of managing GraphQL at scale.
Certain issues may hinder GraphQL in practice. To review, security concerns require a nuanced approach to authorization. This is because it wraps multiple data retrieval points into a single request. Without native caching, performance could also suffer. Changeability is also required to continually update the graph, which may ruffle feathers with those seasoned in the world of REST where a guiding tenet is to build for longevity using web-compliant methods.
According to Jens Neuse, CTO, WunderGraph, there is a simple way to mitigate most of these issues and harmoniously work with various API styles. To Neuse, the core problem is that GraphQL is not meant to be exposed over the internet. Instead, he advocates for interacting through a gateway. Such an intermediary would enable more implicit calls and proper authentication and authorization for specific underlying functions and resources. Using GraphQL as an internal integration layer could also help combine disparate APIs into a similar language, exposing GraphQL-like usability for frontend developers while allowing them to hit any API style on the backend.
Below, we’ll consider the drawbacks of exposing GraphQL directly and see how an intermediary layer could benefit development. We’ll also uncover what open source software is on the horizon that could power this setup.
Facebook, the creator of GraphQL, uses an internal GraphQL API to power all of its apps across Android, iOS and web browsers. (This is not to be confused with their public Graph API). Interestingly, if we look at Facebook’s use of GraphQL, we see these endpoints are not exposed to the public web at all.
It’s true. Open up Developer Tools in Chrome, and you’ll see there is not a single mention of a root HTTP endpoint like https://api.graphql/. Functions name GraphQL, sure, but they call it more implicitly. Other companies, like Twitter and Airbnb, use a similar abstracted approach to interact with a private GraphQL.
In fact, most GraphQL APIs are not public. APIs.guru only charts 35 public GraphQL APIs—notable services include GitHub’s GraphQL API, Shopify Storefront and GitLab. This number is minuscule compared to the total number of public APIs, at over 24,000 according to ProgrammableWeb.
If we treat it as Facebook intended (and how they use it in production today), it could reverse the issues above, said Neuse. “These problems come down to our core belief that GraphQL is not supposed to be exposed over the internet,” he said.
But if clients do not access a GraphQL endpoint, what are they interacting with? According to Neuse, the ideal architecture involves a layer between the client and the server. Such a layer would validate queries and represent verbose calls with simple functions. The process would involve a handshake as follows:
Such a process could also enact universal policies on a per-function basis, thus solving potential authorization issues when fetching data from multiple backend services with a single query.
“The issue is that using the query language comes with a cost,” Neuse said. “Not exposing GraphQL directly but ‘compiling’ it to a JSON-RPC API when deploying solves these problems.”
Thus, the scalability issue isn’t so much an issue of size but of access. “GraphQL can be very valuable, even in small environments,” said Neuse.
Another potential benefit of using GraphQL as an intermediary is combining APIs; a “virtual graph,” as Neuse calls it. This virtual graph offers a common integration language to interact with various API styles, whether REST, GraphQL, gRPC or webhook, in a GraphQL-like manner.
Consider the ease with which package managers like npm enable developers to instantly install dependencies. For example, installing a package like React.js is as simple as initiating the command npm install react within Terminal.
On the other hand, the API integration is very cumbersome. Users must read through documentation, create an account on a third-party site and download an SDK to call the service in their preferred language. So, why isn’t looping in dependencies like APIs and cloud services as simple as other common dependencies?
According to Neuse, one reason is that the REST paradigm is obsessed with the resource constraint—aside from HATEOAS, it lacks a concept of relationships between data and fields. A virtual graph could enable a straightforward process similar to package managers, said Neuse. Interestingly, the WunderGraph team plans to open source a project that allows just that ability.
Many folks assume that GraphQL and REST are mutually exclusive. However, as we’ve seen, there are ways to use them in combination. Plus, with an abstraction layer, the REST versus GraphQL comparison becomes just an underlying technical nuance.
“If you compare GraphQL to other specifications, like OpenAPI spec, you see GraphQL has the fastest-growing community,” said Neuse. As of 2021, JSON Schema is the most popular, used by 75% of developers, followed by Swagger 2.0 (54%), OpenAPI 3.0 (40%) and GraphQL (31%) according to the Postman State of API Report. While this number may be small now, 35% of respondents also said they plan to use GraphQL in the coming year.
“It’s a very, very interesting movement,” said Neuse. What began at Facebook as a framework for data fetching for React apps has now expanded into something much bigger. Within this second new movement, largely driven by companies like Apollo and Hasura, GraphQL becomes the new data layer, he said.
“Still, everybody is looking at GraphQL thinking they must migrate from REST—this is the wrong direction,” said Neuse. Instead of migrating all existing REST APIs to GraphQL, he predicts the industry will coalesce on using GraphQL as the integration language for querying REST APIs. A meta layer, if you will.
Of course, meta has been a bit co-opted lately with Facebook’s rebranding …
As part of a bug bounty program, the AppSecure cybersecurity research team found a vulnerability on the authentication mechanism of Facebook. It gave them the ability to potentially gain full control of the social media giant’s more than 1 billion users. The team won a $15,000 bounty for its discovery.
This vulnerability was found on a niche API, which reminds us that in many cases the most interesting bugs don’t exist on main APIs but on secondary ones that have fewer protection mechanisms in place.
Anand Prakash, CEO of AppSecure and who was credited with finding the bug, explained what he found: “This gave me full access to other user accounts by setting a new password. I was able to view messages, their credit/debit cards stored under their payment section, personal photos, and other private information.”
Even though the vulnerability was discovered in 2016, similar weaknesses have been discovered — and in many cases exploited — ever since. Here is a closer look at the Facebook case and lessons to be learned in modern application development.
The vulnerability exploited the password recovery mechanism on Facebook. The steps of the process were:
Facebook had implemented an anti-brute force mechanism on this API that blocked the user after 10 failed attempts. However, during their research, the AppSecure team found that the same API endpoint existed on different API hosts, under “beta.facebook.com” and “mbasic.beta.facebook.com.” These API hosts didn’t implement the anti-brute force mechanism, allowing the attacker to easily iterate through the secret token and reset the victim’s password.
Keeping track of your APIs it’s not an easy task. Modern organizations might have dozens or even hundreds of API hosts for different environments, regions, or versions. Each API host can expose multiple API endpoints that are related to authentication processes including login, forgot password, and one-time login link. Many security solutions in the market focus on protecting your main APIs and don’t have enough visibility into the less common or less used APIs (such as Facebook’s beta API in the example above). Sophisticated attackers choose to target those niche APIs.
Traceable AI monitors all APIs including the different environments, versions, and regions. For each API host, we automatically and manually detect the authentication endpoints that your APIs expose (including login, forgot password, etc).
This granular understanding of the API structure allows us to set very strict protection for your most sensitive assets.
Watch our recorded demo and see Traceable Defense AI in action!
Facebook’s useful but neglected DevOps tool Osquery has gotten a new lease on life, thanks to The Linux Foundation.
Anyone who has been tasked with monitoring the security of server instances in a data center or cloud knows how laborious and time-consuming it can be. Osquery, a project started by Facebook, aims to lessen this burden by reframing how developers engage with their infrastructures. DevOps professionals can use Osquery to expose an operating system as a high-performance relational database, making it possible to use SQL commands to access data about a system, just as they would for a database.
Osquery works on Mac, Linux and Windows systems and is provided as an open source download via GitHub. Although Osquery was developed by Facebook to monitor and safeguard the security of its own platform, the social media giant quickly realized the utility of the platform would extend to other enterprises that depend upon insight into the low-level behavior of operating systems.
Facebook publicly released the project’s code and documentation in 2014, in response to feedback from a small number of other companies that tested Osquery. The public availability of Osquery resulted in a number of major companies including Airbnb, Dropbox, Netflix, Etsy and Uber to bring Osquery into their software development environments. Nevertheless, users have voiced frustrations with Facebook’s handling of Osquery, and have accused the internet giant of neglecting the project.
To address those complaints, The Linux Foundation has taken over the mantel of Osquery and recently announced the formation of a new foundation to support the Osquery community.
The new Osquery Foundation brings together engineers and developers from Dactiv, Facebook, Kolide, Trail of Bits, Uptycs and many others committed to supporting the project with technical contributions and long-term stewardship. A Technical Advisory Board will be created to handle the shift in management from Facebook to the Osquery Foundation and will oversee the priorities initiated by the foundation’s members.
This restructuring of management promises to extend the usability of the project as a number of developers contribute to the codebase.
“Osquery has drastically simplified the process of operating system monitoring, which has unlocked new methods for securing infrastructure, detecting anomalies and more. We look forward to working with the Osquery community to develop the foundation in a manner that will continue to foster the growth and adoption of Osquery while supporting the community’s diverse needs,” said Jim Zemlin, executive director at the Linux Foundation.
Developers working to maintain the security of their systems believe it would be wise to incorporate Osquery into their workflow. “Although Osquery will never do everything that a security team needs, it gets about 80% of what you need for endpoint insight in one package,” said Doug Wilson, director of security for Uptycs.
Although the platform requires some configuration, developers familiar with SQL will feel accustomed to using Osquery to retrieve information about their systems. Furthermore, Osquery’s cross-platform support and customizability make it especially attractive to businesses with newer types of infrastructures and ones that work at scale.
The move away from Facebook toward a more community-friendly foundation is sure to breathe new life into the project, and as development on Osquery expands, it is likely to become a more valuable resource for security teams moving forward.
This article was co-authored by Tyler Ohlhorst, research analyst and freelance copy editor and copywriter at Magnum Consulting.
MariaDB is a database growing in popularity. Its lead developer is Michael “Monty” Widenius, who was part of the team that created MySQL. When MySQL was acquired by Oracle, Widenius and some of the other original MySQL developers decided to launch MariaDB as a fork of MySQL and keep it as open source to mitigate any issues around Oracle’s ownership of MySQL.
In 2009, Sun Microsystems acquired MySQL. Sun’s subsequent acquisition by Oracle brought about a movement headed by Widenius to “save MySQL.” Widenius and many others in the open source community felt Oracle’s ownership might be a conflict of interest, since Oracle already had a competing closed source commercial database. They believed Oracle would be slow to further develop the MySQL database, given the potentially greater focus on the commercial Oracle database.
In response to this, Widenius began working on MariaDB and helped form the MariaDB Foundation, which uses a community governance model. Because of the separation between the open source and commercial sides of the foundation, its governance generally is seen by the open source community as positive. This is in contrast to Oracle, which has both its commercial Oracle database and the open source MySQL both under the same governance. Often this is perceived as a conflict of interest, especially when it comes to keeping MySQL up to date.
While MariaDB isn’t yet replacing MySQL, it has brought about good competition between the two, which can be good for innovation. With its backing of Oracle, MySQL may reign supreme for at least the near future; however, technically speaking, MariaDB could replace MySQL if a company decides it wants to make the switch.
The developers designed it as a drop-in replacement of MySQL. Since it was forked from MySQL, basically all of the structures MariaDB uses are the same. This means switching from MySQL almost surely won’t require a company to alter its application in any way, since the data and data structures won’t need to be adjusted.
In fact, the command-line tools such as mysqladmin, mysqldump, the daemon and others all keep their original names, so MariaDB is very much a drop-in replacement for database administrators as well. Indeed, many organizations have already either switched to it or simply began using it on its own: It replaced MySQL as the default database in Fedora and Red Hat. It is also used by Booking.com, CraigsList and Wikipedia, among others.
Community improvements to MariaDB have been made by notable organizations such as Facebook, Twitter and Google. For example, Google contributed data-at-rest encryption to release 10.1.3, which is quite helpful given the security issues that can arise quickly in this ever-changing environment. In addition, it offers performance improvements such as subquery optimization and faster joins. It also has added row-based replication.
While MariaDB may not replace MySQL immediately, it is worth watching: It is gaining traction as a potentially serious contender in the marketplace.
This is the first article in a series of pieces on digital innovation, accelerating change and the rise of the coded business from innovators in the Chef (click here) community.
How does the need for speed and scale affect businesses today?
Let’s start with scale. As a technology company grows, its infrastructure grows – more machines (or machine images), more networking, more data storage needs, more code, more developers, and more users. The only way to scale with growth is with automation. And, as more and more of the world’s interactions happen online (be it commercial transactions, email, social networking, video streaming, etc.), that scale of infrastructure is breaking records all the time. With these new levels of scale come ever-increasing automation requirements.
On the speed side, first-to-market has always been an important part of business – but what that means has evolved. It’s no longer simply about producing that new cool item before the other guys. Now it also means you must be able to iterate quickly, adjust quickly for changing user tastes, and adapt quickly to the ever-increasing pace of new technologies. To be able to do this effectively, instead of just developing fast, you need to be able to deploy quickly, react quickly, troubleshoot quickly, and update quickly. For many companies, this is no longer icing on the cake, but instead, the structure underpinning their business – in the same way that having an e-commerce site was 10 years ago.
Speed is becoming ever more critical for business in the digital age. With the rise of social media, a lot of customer feedback is public or semi-public, allowing people to unite behind specific complaints or desires and provide more clear and unified messages to companies. This helps companies get more feedback of higher quality and – hopefully – to react to it more quickly. This is also dramatically accelerating the cycle of getting features/products/releases to users, and, perhaps more significantly, prompting users to provide more useful feedback faster. As I mentioned, the scale and speed of business operations in every industry are growing daily, and, because of that, automation has gone from being beneficial to critical. In the technology world, this automation comes in the form of code.
Pushing your code faster and faster means you have to be able to update and deploy infrastructure faster, which in turn means infrastructure/systems teams must also be able to deploy, react, troubleshoot and fix faster.
But code, in and of itself, isn’t necessarily sufficient to meet the challenge of escalating speed and scale. How quickly you deliver that code is what’s important. Continuous Delivery is one technique used to address this need.
Continuous Delivery (CD)
People talk a lot about CD and it’s important to remember that this phrase refers to a methodology that has pros and cons.
For most technology companies, especially those whose product is primarily Web-based, CD is often the best choice to enable speed of delivery. The main benefit is that the closer to “continuous” your release-cycle gets, the easier it is to have an iterative process. Faster feedback from users means you can more rapidly deliver updates based on that feedback. As you approach CD, you approach near-real-time feedback, which allows you to hone your product or feature far faster than if it were sitting in long dev/qa cycles. Another benefit is best described with a comparison. In troubleshooting, a key rule is make one change at a time, so you know which change fixed the problem – or made it worse. CD is an extension of this idea, although it comes earlier in the lifecycle. If you are pushing/delivering a codebase more often, there are less changes per push, and so breakages are easier to track down – and it’s easier to push fixes.
A common misconception is that an organization either achieves CD or it does not. CD is a spectrum with frequent releases on one end and less frequent on the other. A CD solution for a bank may mean something very different than a CD solution for a gaming Web site. The idea of the CD methodology is continuing to push toward faster deployments, in order to reap the benefits that come from doing so.
In the end, no matter what part of the CD spectrum a company aims to achieve, they must be willing to adapt and change.
From an internal IT perspective, this means realizing that this isn’t the old world, where businesses lock users into small boxes to make things simpler. We’re in a world where everyone has the power of the Internet in his or her pocket and there’s enough information out there for anyone to break out of his or her box. Businesses need to provide engineers, designers, assistants, VPs, etc. the flexibility and freedom to work and think in ways they do best, so they can take the business to the next level.
From an external (production) IT perspective, companies must be able to quickly adapt to new growth and new technology. It took the Internet a (relatively) long time to start adapting to the small displays on mobile devices, and it adapted only slightly faster to tablets. But new form-factors are just the beginning. Broadband is becoming ubiquitous in first-world countries, and companies are just starting to rely on that fact. Yet there are still many users the Web isn’t taking into account very well. Mobile is becoming the default interface for a variety of locales and ages, and many countries have limited bandwidth and/or high latency. Further, it seems likely that we will see drastically different user input mechanisms, as well as display mechanisms, in the near future.
Technologists need to be agile and readily adapt to technology and infrastructure changes, as well as speed and approach changes, such as CD, which will be required for the next generation of user and technology demands.
About the author
Phil Dibowitz has been working in systems engineering for 12 years and is currently a production engineer at Facebook. Initially, he worked on the traffic infrastructure team, automating load balancer configuration management, as well as designing and building the production IPv6 infrastructure. He now leads the team responsible for rebuilding the configuration management system from the ground up. Prior to Facebook, he worked at Google, where he managed the large Gmail environment, and at Ticketmaster, where he co-authored and open sourced a configuration management tool called Spine. He also contributes to, and maintains, various open source projects and has spoken at conferences and LUG’s on a variety of topics from Path MTU Discovery to X509.
