devopsy.kinsta.cloud

Tag: edge computing

  • Building Amazing Apps, Part 3: Optimizing the Network

    Building Amazing Apps, Part 3: Optimizing the Network

    This is part three of a series of three articles on building and delivering amazing apps at the edge. Read part one here, and part two here.

    Introduction

    There are millions of apps available today, and it’s incredibly easy for any user to download and install any of them with just a few taps on a screen or click of a mouse. If you want to build and deliver an app that will amaze users, you need to be able to differentiate from the rest of the pack.

    Over the past 14 years, I’ve been working closely with the top companies delivering the most popular apps, helping them optimize their apps for speed and scale so users get a great experience. In this three-part series, we’ll explore the fundamental steps you must take to build and deliver your own amazing app at scale that will keep users coming back for more.

    We’ll break these steps down into three key areas:

    • Front end: Covers the parts of the app that end users interact with.
    • Back end: Looks at the required origin infrastructure to support the app, including databases, application, web servers and more.
    • Network: The glue that connects the front end and the back end.

    In the final article of this series, we’re going to explore four things you can do on the network to optimize your app and make it more amazing:

    • Distributed DNS
    • Protocol optimization
    • Latency optimization
    • Edge computing

    Distributed DNS

    The Domain Name System (DNS) is a critical part of making sure your clients can access your applications (in fact, it’s a very popular attack vector by hackers today because if your DNS is down, nobody can access your PPP). Distributing your DNS across many servers can help ensure that your DNS (and thus, your app) is fast and highly available, so you should use a DNS vendor that has a large DNS server network in a wide range of locations.

    Validating whether you have a properly distributed DNS infrastructure is rather easy. Just follow these steps:

    • Identify how many name servers your domain uses and their IP addresses. Ideally, you want a number between three and 12—the more the better. You can use command-line tools such as dig and nslookup to identify the name servers of a given domain.
    • Use an IP geolocation system to find out the physical location of the name servers. At this point, you can get a rough idea if those servers are close enough to your end users. Hint: Some CDNs offer diagnostic tools that can be scripted via APIs to automate IP geolocation.
    • If you want to be more thorough, you may want to run connectivity tests between the main locations of the users and the name server IP addresses to make sure the network latency—i.e., how long it takes for information to go back and forth between the client and the server—stays under 100 ms (the lower the better). Some internet providers and CDNs provide looking glass technology, which enables you to run connectivity tests that originate from servers around the world.

    Protocol Optimization

    Once your clients know where to connect, how they connect and the protocol used is something that shouldn’t be neglected. HTTP/2 is the modern, superior network protocol, as it uses a more efficient binary format, header compression, single network connection, stream multiplexing and server-push. Actually, I was very involved in the early years of HTTP/2 (the standard was approved in February 2015) and have since helped some of the largest websites on the internet implement HTTP/2 and still spend a considerable amount of my time testing the new protocol in all sorts of environments. I’ve shared my findings in talks at conferences including Velocity and in the “Learning HTTP/2” O’Reilly book, which I co-authored. In a nutshell, most sites can improve page load performance between 5% and 15% by just switching the protocol from HTTP/1.1 to HTTP/2, which is as simple as flipping a switch if your website uses HTTPS.

    To illustrate that improvement, I created a demo site that tests the download speed of many small images over HTTP/1.1 and over HTTP/2 and displays the differences.

    Figure A

    As you can see in Figure A, the content loaded eight times faster over HTTP/2 than over HTTP/1.1 on my mobile phone over an LTE connection. That’s a huge improvement considering how simple it is to implement HTTP/2.

    According to the HTTP Archive, as of June 2018, 20% of hosts support HTTP/2, but those hosts are responsible for 40% of all requests—meaning HTTP/2 powers some of the web’s most popular sites.

    However, as explained in the HTTP/2 Performance chapter of the Learning HTTP/2 book by O’Reilly, there are a lot of factors that can impact the benefits of HTTP/2. Some of them you can control, while others—such as network packet loss—are much more difficult to tackle and can impact HTTP/2 performance because HTTP/2 runs over a single TCP connection.

    Note: TCP stands for Transport Control Protocol and some parts of its implementation (e.g., slow to start and cumbersome network congestion algorithm) can impact HTTP/2 performance because all the communication with the server goes through a single TCP connection.

    The future of the web is HTTP/3, which uses HTTP/2 over QUIC (a newer and more efficient transport layer protocol). However, as of April 2019, HTTP/3 is still in draft state and hasn’t been approved as a standard.

    Ideally, your servers should monitor the network connection and use different network protocols depending on the current conditions. Certain content delivery networks (CDNs) offer this functionality. For example, to maximize throughput, a top CDN provider uses custom software on its edge servers to dynamically use different network protocols, TCP settings and congestion control algorithms, depending on the specific conditions of each individual connection the server makes with an end user. In short, using the optimal protocol makes your app faster and more amazing.

    Latency Optimization

    Latency is a physical limitation of nature that impacts the throughput on computer networks. You probably know that whenever you are outdoors and there is a storm with lightning, you can count the number of seconds until you hear the thunder to identify how close the storm is (and if you count less than a second, you better get inside!). In computer terms, latency is measured as the time it takes to send a network packet back and forth between two systems. This concept is known as round-trip time (RTT) and is typically measured in milliseconds. You can use network tools such as ping or mtr to easily measure the RTT between two systems.

    Latency has a huge impact on performance because the further away you are from the server you want to connect, the slower the communication will be. As a rule of thumb, you want to ensure your latency is well under 100 ms. The easiest way to achieve that is by leveraging the local device cache to avoid extra network connections to the server and using a CDN to put your server closer to the end user.

    Here is a graph (see Figure B) showing that latency plays a larger role on page load time than bandwidth. As you can see, doubling the bandwidth from 5Mbps to 10Mbps results in just 5% improvement in page load times, and any bandwidth increase over 10Mbps doesn’t make a big difference. Reducing the latency, however, drastically improves the page load times. That makes your app more amazing for your end users.

    Figure B

    Edge Computing

    If you are a developer or a DevOps engineer (or just love tech) chances are you are within the TOP 1% of world population in terms of mobile computing power. But things you take for granted, such as resizing images on a mobile phone, are going to take much longer for many millions of internet users who have less powerful mobile devices.

    So, what can we do if running logic at the origin causes delays because of the network latency and running logic on the client side may be too slow on less powerful devices? The solution is edge computing, which consists of moving the origin logic to a server close enough to the end users that latency does not impact performance. CDNs are in the best position to offer this proximity because of their natural distributed architecture; a CDN can set cookies, rewrite HTML and resize and deliver optimized images for the screen size of your users so those images don’t take a long time to display on low-end mobile devices with limited CPUs.

    Edge computing is the next step beyond cloud computing. You should try to use a CDN company that offers you the ability of running business logic as close as possible to where your users are, to ensure that your app is fast and can scale without impacting your infrastructure.

    Summary

    In this series of articles, we reviewed some basic things you should consider if you want to build and deliver amazing apps at scale. This is not a comprehensive list of all the things you can do but provides you with some of the most impactful areas for app improvement.

    — Javier Garza

  • DevOps Chat: Computing on the Edge, with Swim.ai’s Simon Crosby

    DevOps Chat: Computing on the Edge, with Swim.ai’s Simon Crosby

    Simon Crosby has made a career of technology on the edge. From leading the charge with the Xen hypervisor to microVMs with Bromium and much more in between, Crosby has led some of the most innovative technology developers in the industry.
    He is now the CTO of Swim.ai which is literally bringing computing to the edge (almost an anti-cloud). This approach promises some revolutionary capabilities.

    If Simon is involved, there is a good chance it will succeed as well. Have a listen to what Simon is all about now.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone this is Alan Shimel, devopsy.kinsta.cloud, Security Boulevard, and we’re here for another DevOps Chat. I’m really lucky to be joined by an old friend of mine for this chat, none other than Simon Crosby who is the CTO of a hot company out there called swim.ai. Simon, welcome.

    Simon Crosby: Thanks, Alan. It’s great to speak to you again.

    Shimel: Thank you. It’s actually my pleasure. Simon, maybe we have some young ones in the audience who maybe aren’t familiar with the one and only Simon Crosby. Why don’t you give folks a little background of how you came to be here today? Little bit of _____ –

    Crosby: Well, I was lucky enough to cofound a company called “XenSource,” which built and maintained the open-source Xen hypervisor, which was – so XenSource was bought by Citrix, but then went on to fame because we built Amazon AWS. And then on, from there, I then cofounded a company called “Bromium,” which used micro-virtualization to do security. And that has, funnily enough, turned also into Amazon’s micro-virtualization, which they very recently open-sourced, as well as the micro-virtualization in Azure.

    Shimel: So, in many ways, you’ve given birth to so – “given birth to”? – you’ve helped launch so many of the key technologies that are powering the cloud today, right?

    Crosby: It’s fun.

    Shimel: Yeah. You know, I think a lot of people tend to think hypervisor, VMware, and they forget, really, how important Xen was and still is.

    Crosby: Still is. It still has more VMs than any other hypervisor on the planet.

    Shimel: Absolutely. And then Bromium too – I mean, Bromium micro-segmentation, micro-perimeterization, whatever you wanna call it, again, was a – I mean, it was really a defining moment, you know? It was a –

    Crosby: Yeah. So I –

    Shimel: – _____ approach security.

    Crosby: So what Bromium does is every single task on the OS, every tab in your browser, we document your residency – its own little micro-VM.

    Shimel: Right.

    Crosby: And those things are tiny. And so if bad guys show up, it’s _____ _____. That technology has gone into Windows 10. It’s coming out in stages from Microsoft and it’s also available in Azure and in AWS.

    Shimel: Fantastic. But enough about that. We’re here to talk about SWIM.AI. And I’m gonna assume that most of our audience, probably, has not heard of SWIM.AI. They may get from the title that it is based on SWIM, but they may or may not even know SWIM. Why don’t you clue us in, Simon?

    Crosby: Well, the name is kind of a clue – it’s “software in motion.” We’re focused on the edge – in fact, intelligent edge computing – and so I’m gonna bend your mind a bit with a definition here. We’re all familiar with cloud and cloud computing as a set of abstractions for building stuff, not as a place, right? So, when we think of it, I’d like you to think of edge computing as needing a set of abstractions. And I think the current abstractions we use for cloud don’t apply very well at the edge. So let’s think of edge computing as a new set of abstractions, the ones that we need to deal with the world, which is generating more data than we can ever hope to store. And so what are the abstractions to do intelligent computing on that?

    Now, when we talk about edge intelligence, I mean, I guess four things: first of all, self-training and inference; analytics; any form of query – MapReduce, relational graph cap query, or any old other thing that you might wanna throw into data to analyze. And the key challenges with the current paradigms, the cloud world, which has been so tremendously successful, is really three things: REST, stateless computing, and the notion of a database. And all of these things are vital for all of the apps that we know and love today, but they completely screw up everything that you’re trying to do at the edge.

    So just think about this way: if I have an event coming of some sensor at the edge and I throw it up to an AWS Lambda thing, which then goes and looks up the old value in a database, does some computation and puts a new value in database, you know, that’s probably taken 100 milliseconds, 50 milliseconds maybe. If I’m running towards the edge in a distributed application, that’s tens of billions of CPU cycles.

    Shimel: Okay.

    Crosby: And so this notion of forgetting everything, which is really is focused scaling out the cloud, is terrible at the edge. What you want is a STAPLE model. And so STAPLE computation is fundamental in the edge – anybody who’s written a device driver knows this – and distribution is fundamental. Computing on data as it is generated, which means learning on the fly, analyzing on the fly, are fundamental. Okay? So not storing things on disks and pulling it off later because then you’re analyzing past data and you can only reason about the past. And we have to reason about things as they happen and respond in real time.

    Shimel: You know, what’s interesting about this to me, Simon, is – so I guess it’s gotta be, what, 2004, 2005, we start really seeing the cloud go mainstream, if you will. And I –

    Crosby: 2000 – you know what? AWS is beginning of 2006.

    Shimel: Okay. So fair enough.

    Crosby: Oh, but Salesforce, obviously, is a long time ago, but there you go.

    Shimel: Yeah, but, you know, this thing “the cloud,” right, and how it kind of exploded on the scene. And your company sort of started in ASP, back in the late ’90s – it was hosting Lotus Notes before there were hypervisors and clouds and _____ bandwidth. But, anyway, part and parcel of the cloud, though, was this notion that there would be this centralized – if not, but it would be distributed – sort of center where certain computation and hosting, like database, et cetera, would – and storage – would happen at that nebulous cloud. But there was always supposed to be the other piece of it, the edge. Right? There was the cloud and the edge. And we got so enamored with the cloud and so much happened with AWS and Azure and Google and a million other clouds out there, hybrid and all privates, we forgot about the edge for about –

    Crosby: Well, I think –

    I mean, I wanna be a little bit generous here. I think job one was to get enterprise IT out of bring in IT. In other words, get out of running exchange _____ _____ use things like Boomawipe because, gosh, let them run – let somebody else run their stuff for you. It’s not mission-critical. So the trillion dollars of traditional enterprise IT needs to get pushed to the cloud, and that’s happening at a huge pace today, and you see the winners emerging there. But there’s another trend which is happening and that is that, well, if you look at ARM, you know, ARM is licensing 20 billion cores a year.

    Shimel: Wow.

    Crosby: That’s a big number of CPUs. And every one of those is gonna be connected and generating data. And so that is a trend which is caught up. It’s because everything is smart now, right? You can embed these cores in anything. And so, when you deal with a world which has got that much data coming out of it, we have to do something other than simply save everything in a big data store and think about it later because there will never be a later. There is so much data coming out. And so there is a fundamental requirement that we process this data on the fly. I wanna introduce you a slightly mind-twisting thought. There is going to be so much data that things have to write their own programs. There aren’t enough humans to write programs for the things, right?

    Shimel: Yeah, no, I agree.

    Crosby: The things generating data will learn experientially, a little bit like you do – okay, you form a hypothesis about the world; you see the way the world develops; and then you change your hypothesis – so things will do that and they will learn on their own data. And, in “learning,” what I mean is you might – they’re writing a program, effectively, these things, right? So the program might be the wait of a DNN or something, and you might not understand what it means, but, when they have absorbed enough data, they’re really good at understanding their world and predicting the way it will evolve around. And that’s the way the world is going, that things write their own programs.

    Shimel: You know, I had a conversation with a guy named Luke Kiely, about two or three years ago. Luke was the founder and CEO, at the time, of Puppet. And he had a similar thing – he said, “Alan, it may not be this year, next year, five years or maybe even 10 years or more, but there will come a time where the software needs to write its own software.”

    Crosby: Yeah.

    Shimel: But, you know, it just – the humans are the bottleneck. Right? The human is the resource constraint.

    Crosby: Well, actually, I mean, they’re – we’re on DevOps podcast, but, you know, there will never be DevOps for my younger _____ _____ _____ _____. I’m telling you, there just will never be that. There aren’t enough people to do this, right? And so we have to back up –

    Shimel: Go ahead. No, no, you go on ’cause I agree with you. I mean, the _____ – I mean, the human is the single – just like in security, a lot of times, it was – that was the problem. [Chuckles]

    Crosby: Yep. No, that’s right. And so more automation is key. By the way, there’s another skillset which I think is hypothesized too much, and that is data science. I mean, programmers are rare, good programmers rarer. Data scientists? Good luck trying to find one.

    Shimel: You are not kidding.

    Crosby: Hard to find, right? And so there is this workflow which both Amazon and – well, Amazon, Microsoft, and Google posit, which is this: collect all the data; somebody builds a model, trains the model, and then pushes this model towards the edge, where it can do inference. You know, that’s just hypothetical nonsense. Okay? It’s not gonna happen. [Chuckles]

    Shimel: Go ahead.

    Crosby: And, by the way, so, first of all, you’re gonna collect all the data and, by the way, they’re happy if you do that ’cause then you’re stuck.

    Shimel: Right.

    Crosby: Right? ‘Cause then you’re stuck on that cloud.

    Shimel: Exactly.

    Crosby: And then, you know, you gotta find this person to build a model and train it. That’s really hard. And if the data is wrong, then you get over- or under-fitting, and then you gotta push all that stuff to edge. Nobody knows how to do that. It’s just not –

    Shimel: No, they don’t. You know, I was talking – I guess it was maybe last week. A friend of mine, actually from this gym we go to – she’s a recruiter. She’s recruiting down in San Jose and they’re trying to get Hadoop folks, just basic Hadoop programmers. And, Simon, look, I know things are crazy in the valley. I think they were paying maybe $240,000.00 or $250,000.00 and they –

    Crosby: Unbelievable.

    Shimel: They can’t find them, though. They can’t hire them. There’s not enough people with that data scientist, with that type of big data kind of background.

    Crosby: There aren’t. And I think, you know, as we move towards a more real-time business environment, organizations are going – let’s posit a future for enterprise IT. Enterprise IT, as you know, today, it needs to change towards something where the gunk is dealt with by somebody else. And then enterprise IT needs to feed real-time insights for different components of the business to the stakeholders, so they can make better decisions.

    Shimel: Okay.

    Crosby: Yep. And whether that’s supply chain or employees or whatever it happens to be, we need to put enterprise IT into position to deliver real-time insights. And how are they gonna do that? That’s really interesting problem.

    Shimel: Absolutely. But there’s something else to what you said too, Simon, that I wanted to hit on and that is – you know, you mentioned – what’d you say? – ARM had 20 billion or licensed 20 billion CPUs?

    Crosby: Yeah, per year.

    Shimel: Per year. You know, to me, you look at Uber, right? Some people will argue, “Is Uber a software company? Is Uber a drive company? What is Uber?” What Uber really was is they saw a glut of resource, right? People had cars that they weren’t using all the time, that they could monetize. Right? To a certain extent, it’s the same thing here – the amount of computing power on the edge or potential computing power on the edge.

    Crosby: Yeah.

    Shimel: So massive. Right?

    Crosby: Yeah, it’s huge. And so let me give – you’re actually right – so let me give you a really shocking example. So we solve this problem of prediction of traffic and I’ll give you two ways of doing it. And my example is the city of Palo Alto in California.

    Shimel: Mm-hmm.

    Crosby: And so we get about four terabytes a day from Palo Alto. That’s bigger than Twitter.

    Shimel: Wow.

    Crosby: Bigger than all of Twitter. Okay?

    Shimel: Mm-hmm.

    Crosby: So you could take all that data and push it up to AWS Lambda, into a model, and then learn and predict. Okay? Cool. That’ll cost you about $8,000.00 a month. Okay? And learning at the edge, as the data streams off the infrastructure, we bought no new hardware and it costs us pennies.

    Shimel: Wow.

    Crosby: Okay? So I’m saying thousands of times cheaper than AWS. Okay? So REST, statelessness, and databases are my enemy, number one, and then you’re absolutely right – there is a ton of unused CPU towards the edge, out in this distributed real world.

    Shimel: Nature abhors a vacuum.

    Crosby: And if I have 10 billion spare CPUs, I mean, I can do an inferencing cycle on Raspberry Pi in the time that it takes a packet to get from the city to AWS.

    Shimel: To AWS and then coming back. Mm-hmm.

    Crosby: We’re done, right?

    Shimel: Yeah. Yeah. Hey, Simon, as I mentioned, the time goes really quick. I wanna focus, ’cause your marketing people appreciate it, Swim.ai. How can people engage? What do you recommend how do they engage?

    Crosby: Okay. So what is Swim? Swim is a platform for application developers that want to develop massively distributed, stateful, real-time applications. Okay? It’s a tiny, two-megabyte extension to JVM, okay, which deploys and replaces a huge stack of cloud-based stuff, including the analytics and learning software, the pipelines, Pub/Sub, all the databases you might want. Okay? So we replace millions and millions of lines of code with something very simple.

    And it’s an extension of JVM because, that way, we get architecture independence. We found ourselves running ARM and X86 and everything else, all at the same time. And all you have to do is define what’s in your data and then SWIM builds a model for you, trains the model, and does the inference and prediction. Okay? So, simple, right? So the goal here is to make it simple for somebody who is a competent Java programmer, can do a bit of the business of doing really complex analytical stuff on streaming data, towards the edge.

    Shimel: Excellent. And, obviously, the website is swim.ai, yeah?

    Crosby: Yeah, developer.swim.ai is where you’d go and download it and run apps and everything else. You can download the platform, as well as the example app, so, if you wanna do something like New Relic, download Swim, stick it in each one of your VMs, and you’re up and running in a flash.

    Shimel: Excellent. So, Simon, I think I already know the answer to this, but I’m gonna ask it anyway. You have – I mean, your reputation precedes you, your accomplishments. What got you really excited about Swim.ai?

    Crosby: You know, if you can deliver improvements in performance and timeliness, which are in the orders of thousands – not 10X, 1,000X, okay? – and really change the world in dramatic way, that’s really exciting. So let me just – let me give you a somewhat critical view of the cloud, right? The cloud vendors make huge margins keeping memory warm for lots and lots of copies of Windows or Linux. Okay?

    Shimel: Uh-huh. Okay.

    Crosby: And most of that is idle.

    Shimel: I got it.

    Crosby: Most of those things are sitting there, doing nothing –

    Shimel: Just waiting.

    Crosby: – while REST happens. And we’ve moved compute from binary into these ASCII-based REST interfaces with lots and lots of pausing. Okay, cool. And now you’re paying for it.

    Shimel: Yeah.

    Crosby: What I’m saying is that you can do that same computation really enormously cheaply, towards the edge, on stuff you already own. And that’s very interesting.

    Shimel: It really – and I could say, “Well, it’s sort of as expected.” Well, Simon, people can go to the website to find out more. You’re talking at any conferences or the company is _____ –

    Crosby: Yeah, so we’re launching Swim app at the developer week in Oakland, on Thursday.

    Shimel: This Thursday? Which would be February 21st.

    Crosby: Yeah.

    Shimel: Well, by the time this is up, that might have passed, but they can probably check it out there and see in the website. Well, Simon, if it’s anything like the other stuff you work on, this is gonna be great.

    Crosby: Thank you, sir. It’s a privilege to spend time with you.

    Shimel: We’ll be watching you closely. We wish you lots of luck. And, you know, living life on the computing edge –

    Crosby: It’s fun.

    Shimel: It is fun. Hey, man, it’s great catching up with you and best of luck, Simon.

    Crosby: Thank you, Alan. It’s great to speak to you again.

    Shimel: All righty. Simon Crosby, CTO of Swim.ai. Check him out, computing on the edge. This is Alan Shimel for devopsy.kinsta.cloud. Have a great day, everyone.

    — Alan Shimel

  • Edge Computing: Saving the Cloud from Ephemeral Data

    Edge Computing: Saving the Cloud from Ephemeral Data

    Edge computing is rapidly gaining momentum. Last week’s Linux Foundation news on the launch of LF Edge is one more sign of accelerating growth at the edge. What’s driving this sudden intense interest? It’s more than just the sheer number of edge devices or mounting concerns around interoperability and security. Pushing compute resources closer to the edge can save the cloud and networks from waves of ephemeral data.

    The concept of edge computing certainly isn’t new. In years BC (“before connectivity”), there were plenty of edge computing boxes. These featured single-board computers designed to specifications such as VMEbus, STDbus, PC/104, CompactPCI, AdvancedTCA and many others. Processing happened at the edge as data arrived. Ethernet connections made administration and results-sharing easier, but very few architectures relied on shipping all the data into the network.

    Of course, there was one big exception: a business model pulling all the data into the network. Telecom providers aggregated traffic from the edge into their core networks. As data traffic joined voice traffic, core networks grew with monstrous bandwidth and increased services. Core networks interconnected. Data traffic rose even faster, pushing the network into a cloud architecture. In years AD (“after distribution”), everything from the edge is now connecting to the cloud.

    Much of the increase in data traffic came from the shift to video content. Users expect higher resolution video programming with the same lag-free performance. Streaming networks and quality of service (QoS) implementations have made great strides in content delivery. 5G infrastructure rollouts in progress will keep mobile a strong downstream contender.

    What about upstream applications such as the IoT? Nearly the same terminology applies there. Devices gather data on the edge and transmit it to the cloud through a gateway. There are two key differences with most IoT applications, however. Real-time data may be lost if the streaming connection into the cloud is interrupted. Also, the fallacy of low bandwidth does not work at scale if all the data is transmitted. At some point—perhaps ten thousand, a hundred thousand, or a million or more sensors—an aggregate bandwidth problem develops.

    Just as in the BC age, it’s time to rethink shipping all that IoT data upstream, instead taking steps at the edge. Cisco Systems has studied data trends extensively, producing its white paper, “Cisco Global Cloud Index: Forecast and Methodology, 2016-2021.” The company points to the IoT ushering in the “yottabyte era” very soon. Cisco estimates that 220ZB of data was generated in 2016 and projects that by 2021, there will be nearly 850ZB from people, machines and things combined.

    Now for the interesting part: Cisco also estimates that as much as 90 percent of that data is ephemeral, short-lived data not useful to save. Even if only 10 percent of data is useful, it will swamp projected data center traffic by an estimated factor of four in 2021. Here’s Figure 24 from the Cisco white paper:

    Edge computing, or its close cousin fog computing, allows two things to happen: It can pre-process ingested data, ironing out protocols and format differences such that everything arriving at the cloud is immediately ready for algorithms. Any savings in the cloud from not shuffling data before computation are valuable, increasing processing and network bandwidth availability. Parts of an algorithm could easily be farmed out toward the edge, perhaps decimating or filtering data before shipping it upstream for heavy analysis.

    A bigger use for edge computing may be using algorithms to avoid shipping ephemeral data upstream altogether. Using generative models at the edge could make a huge difference in identifying useful data. Most of us have heard that no human can sit and watch hours and hours of surveillance video from multiple sources looking for one moment of a problem developing. Consider another example with an EKG monitor attached to a smartphone: While the patient is at home, at rest and unstressed, the EKG is in a perfectly normal range as learned by the application. Doctors are alerted when readings slip out of normal range, with raw data transmitted for observation if required. A similar example is waiting for a motor to fail while monitoring its operation.

    Both the LF Edge initiative and work happening in the OpenFog Consortium are creating specifications for edge computing. These initiatives are aimed at interoperable, secure architectures where cooperative processing mines data. If items of interest can be extracted efficiently from useful and ephemeral data at the edge, loads on the cloud and networks would be lighter. Edge computing will likely detect exceptions more quickly, bringing people in to look at what is going on faster. The result will be more scalable IoT applications with better chances of delivering value from all that data.

    — Don Dingee

  • How DevOps Can Nurture the IoT Brownfield For Results

    How DevOps Can Nurture the IoT Brownfield For Results

    Your latest IoT project sailed through proof-of-concept but started running into problems during rollout. Fortunately, your DevOps superhumans got things running. Now, perhaps a year or more later, you’re seeing more and more issues crop up as the project grows. If this scenario sounds vaguely familiar, or you’re worried you may be headed there, you’re not alone. Welcome to the IoT brownfield.

    “Nah, can’t be. We selected and evaluated all-new hardware and software during the PoC. This has been a greenfield project from the start.” Be careful of this trap. Many IoT projects indeed start off as greenfield efforts, free of the constraints of integrating legacy hardware and software. Your team developed requirements with collaboration between IT and line-of-business folks. You introduced these new tools, and got results on a small scale. According to Cisco’s 2017 IoT survey of 1,845 IT and line-of-business decision makers, 60 percent of IoT projects stall at PoC, so you’re ahead of the game.

    Or so it would seem. Now comes the escalating commitment. Management assumes that if it worked on a small scale, it should just work when rolled out across the enterprise. Data and insight will just come pouring in, and everyone will have visibility to what’s happening everywhere in real-time. They write a big check for the rollout, and away you go.

    First Signs of Trouble

    At the first rollout project review, folks start splitting hairs over the definition of “complete success.” You rolled out X locations, Y of them are working, X-Y of them are in various states of triage. How can that be? You took the “same” configuration and scaled it. As the team investigates the non-working population, you’re finding little things that aren’t the same, calling for tweaks here and there.

    Tossing out the trivial, over scale and time there is no such thing as an IoT greenfield project. Every IoT project with an extended life cycle becomes a brownfield, with a mix of new and old hardware and software. Rigorously controlling a configuration, à la defense or medical applications, is prohibitively expensive and unnecessary for most IoT applications. Your vendors won’t sell you the exact same hardware or software two years from now, five years from now, 10 years from now. Unless, that is, you pay them for things such as lifetime buys of electronic components or freezing open source code releases.

    Ideally, we should send DevOps teams into IoT projects. With a big tool stack acquiring data, the line-of-business types figure out more things to do as the picture develops. The whole point is gaining insight and adjusting for optimized results. Cycles of learning and security enhancements are part of the game.

    Coping with an IoT Brownfield

    Here’s the paradox: the longer your IoT application survives, the more complex this situation gets. How can you improve your chance of success by applying IoT brownfield thinking? Technical requirements vary, but some core philosophical tenets can help.

    Roll with the changes. Your team is already skilled at quickly rolling application software releases. Get more aggressive with ongoing changes at the firmware and network level. Incorporate over-the-air (OTA) technology for updating code on any wireless devices. Deploy multi-protocol gateways that can deal with revised protocol stacks (or entirely new ones) as IoT specifications continue evolving. Update security vulnerabilities relentlessly.

    Abstract everything. You’ll read tons about IoT interoperability, and what people typically mean is wire-level interoperability of protocols. Those multi-protocol gateways have one job—getting real-time data onto an IP network and into the cloud. What keeps me up at night is data-level interoperability. Use edge or fog computing for pre-processing streams so the cloud isn’t stuck sorting out various formats of the same class of data. Abstract your data using JSON or other lightweight techniques so that if a next-generation sensor outputs data in a slightly different format, pre-processing can be revised close to the source. Consider a publish-subscribe protocol for easier routing between nodes.

    Optimize the cloud. Scale can bite hard in the cloud tier. Heterogenous, distributed architectures usually win. Deploy a hybrid cloud solution where there is real-time data. Let the public cloud do what it’s good at: presentation, business intelligence and archiving. In a private cloud instance for heavier analytics lifting, use a framework such as Apache Spark to minimize data movement, with faster nodes sprinkled in for critical algorithms. Try dedicating a small processing core to each data stream for predictable deadlines, or using FPGAs in workflow optimized nodes for boosting compute performance.

    After All, It’s Just Business

    Ultimately, the evaluation of complete success of an IoT application will be up to the line-of-business decision-makers who wrote that big rollout check. A DevOps model is ideal for nurturing the IoT brownfield over scale and time, featuring ongoing strong collaboration between technologists and line-of-business types. Early in the project, reach out for some external IoT expertise until you have a few cycles of learning under your belt.

    As that Cisco 2017 IoT survey alluded to, 61 percent of practitioners believe we’re barely scratching the surface of IoT transformation for businesses. Avoiding the greenfield trap and incorporating these IoT brownfield coping strategies sets you up for the best possible chance of success in the long run.

    — Don Dingee