The good news to start 2019 is that cloud adoption is growing rapidly. Most people realize this, but along with major growth comes some less welcome news. Wasted cloud spend is also up—often dramatically. Chances are, if you’re spending money on cloud, some of that money is being wasted. For some, a lot of that money is being wasted. Here are the numbers.
Predicted Cloud Spending 2019
The latest predictions from Gartner estimate that overall IT spending will reach $3.8 trillion this year, a growth of 3.2 percent over IT spending in 2018. Of this spend, public cloud spending is expected to reach $206.2 billion—of which, the fastest growing segment is infrastructure as a service (IaaS), which Gartner says will grow 27.6 percent in 2019 to reach $39.5 billion, up from $31 billion in 2018.
Now we can subdivide the public cloud spend number further to look just at compute resources—typically two-thirds of cloud spend is on compute, or about $26.3 billion. This segment of spend is especially vulnerable to waste, particularly from idle resources and oversized resources.
Wasted Cloud Spending from Idle Resources
Let’s first look at idle resources. This includes resources that are being paid for by the hour or minute, but that are not actually being used. Typically, this kind of waste occurs in non-production environments; that is, those used for development, testing, staging and QA. According to internal data available to my company, ParkMyCloud, about 44 percent of compute spend is on non-production resources.
Most non-production resources are only used during a 40-hour work week, and do not need to run 24/7. That means that for the other 128 hours of the week (76 percent), the resources sit idle, but are still paid for.
What does all this add up to?
$26.3 billion in compute spend x 0.44 non-production x 0.76 of week idle = $8.8 billion wasted on idle cloud resources.
Wasted Cloud Spending from Oversized Resources
The other source of wasted cloud spend is oversized infrastructure; that is, paying for resources at a larger capacity than needed.
RightScale found that 40 percent of instances were sized at least one size larger than needed for their workloads. Just by reducing an instance by one size, the cost is reduced by 50 percent. Downsizing by two sizes saves 75 percent.
The data we see in our users’ infrastructure confirms this, and in fact we find that it may even be a conservative estimate. Infrastructure managed in our platform has an average CPU utilization of 4.9 percent. Of course, this doesn’t take memory into account, and could be skewed by the fact that resources managed are more commonly used for non-production resources. However, it still paints a picture of gross underutilization, ripe for rightsizing and optimization.
If we take a conservative estimate of 40 percent of resources oversized by just one size, we find the following:
$26.3 billion in compute spend x 0.4 oversized x 0.5 overspend per oversized resource = $5.3 billion wasted on oversized resources.
Total Cloud Spending to be Wasted in 2019
So, between idle resources and overprovisioning, wasted cloud spend will exceed $14.1 billion in 2019. In fact, this estimation of wasted cloud spend is probably low. This calculation doesn’t even account for waste accumulated through orphaned resources, suboptimal pricing options, misuse of reserved instances and more.
Fortunately, cloud cost control is becoming top-of-mind as the industry is able to leverage cloud computing growth statistics to calculate the massive scale of this problem. If you are involved in your organization’s cloud infrastructure, why not make reducing cloud waste one of your 2019 objectives?
It’s 2019, which means a whole new year of advancements in all areas of technology. One topic that is near and dear to my “cloud” heart is the acceleration and maturation of development and test (DevTest) scenarios and solutions in the public and hybrid cloud space.
I’ve been talking with folks and presenting on the movement of Dev/Test to the public and hybrid cloud for a couple of years now, and I have progressively learned a couple of things from those with which I have had great and sometimes very frank discussions. First, I learned that the definition of DevTest has expanded exponentially, with the movement to the different clouds. Second, I found that the terminology of “DevTest” just doesn’t fit a single threaded way of thinking any longer, such as “It’s just for developers.” Let’s take a closer look at these.
Back when I was a young technician just starting in the new world of IT, I was working primarily with programmers (they weren’t called developers or devs back then) who would sit in front of their green screens, writing huge applications with COBOL and RPG, or maybe writing shell scripts on a Unix system. They would test their code in different environments that were always on the same infrastructure that the production code was running (which we all know these days is wrong, right?). They didn’t have secondary hardware or OS installations to test against because it was just too expensive to accommodate and most of the code could not support that separation.
Fast forward the clock almost 30 years, and now the developers need to have logically separated and secure environments to provide accurate and meaningful testing of the code that is “born in the cloud” as well as those created on-premises. The next few years will bring a big push for developers to have their own self-service environments, extensive code portability and more secure isolation for their code. By utilizing container technologies, automation and rapid deploy processes for their code, more development groups will eventually become their own application development as a service.
DevTest solutions are also moving far beyond just testing code. Quality Assurance (QA) is already a substantial consumer of the cloud for their load, smoke and regression testing. I predict that there will be a huge advent across all roles within IT—and not just the ones of development and QA—that will take the plunge into the DevTest solutions. They will be running release scenarios on their rapidly and continuously updated OS versions, on all of the software-defined environments, while also adding software patch testing and issue mitigations.
Knowledge transfer, training and E-Learning also will increase the use of these landscapes over the coming years, with everything from purpose-built labs, globally connected classrooms and even enterprises expanding their hands-on learning management system (LMS) offerings to their employees and customers.
Finally, I would suggest that there will be an inflection point in the coming years in which the use of private, hybrid and public clouds for DevTest—and, eventually, production—will become transparent to the user to the point that, within the overall vision of what you need to accomplish, what cloud or combinations of clouds you utilize won’t matter any longer. Eventually, for infrastructure as a service (IaaS) at least, the cloud will go the way of hardware, which has largely become a commodity in the new evolution of the modern software-defined data center.
Technology is moving incredibly fast these days, especially when compared to the days of old when I was coming through the ranks in IT. The DevTest solutions will continue to evolve, progress much faster and touch more people in technology—and even other professions—than ever before. Start building a solid DevTest design and implementation plan for whatever your business and IT goals may be, and invest in the solution for all of your IT functions. By embracing and utilizing what is available in the hybrid and public cloud space for DevTest, IT will be empowering more productivity and faster execution moving into 2019 and well beyond.
Edge computing is rapidly gaining momentum. Last week’s Linux Foundation news on the launch of LF Edge is one more sign of accelerating growth at the edge. What’s driving this sudden intense interest? It’s more than just the sheer number of edge devices or mounting concerns around interoperability and security. Pushing compute resources closer to the edge can save the cloud and networks from waves of ephemeral data.
The concept of edge computing certainly isn’t new. In years BC (“before connectivity”), there were plenty of edge computing boxes. These featured single-board computers designed to specifications such as VMEbus, STDbus, PC/104, CompactPCI, AdvancedTCA and many others. Processing happened at the edge as data arrived. Ethernet connections made administration and results-sharing easier, but very few architectures relied on shipping all the data into the network.
Of course, there was one big exception: a business model pulling all the data into the network. Telecom providers aggregated traffic from the edge into their core networks. As data traffic joined voice traffic, core networks grew with monstrous bandwidth and increased services. Core networks interconnected. Data traffic rose even faster, pushing the network into a cloud architecture. In years AD (“after distribution”), everything from the edge is now connecting to the cloud.
Much of the increase in data traffic came from the shift to video content. Users expect higher resolution video programming with the same lag-free performance. Streaming networks and quality of service (QoS) implementations have made great strides in content delivery. 5G infrastructure rollouts in progress will keep mobile a strong downstream contender.
What about upstream applications such as the IoT? Nearly the same terminology applies there. Devices gather data on the edge and transmit it to the cloud through a gateway. There are two key differences with most IoT applications, however. Real-time data may be lost if the streaming connection into the cloud is interrupted. Also, the fallacy of low bandwidth does not work at scale if all the data is transmitted. At some point—perhaps ten thousand, a hundred thousand, or a million or more sensors—an aggregate bandwidth problem develops.
Just as in the BC age, it’s time to rethink shipping all that IoT data upstream, instead taking steps at the edge. Cisco Systems has studied data trends extensively, producing its white paper, “Cisco Global Cloud Index: Forecast and Methodology, 2016-2021.” The company points to the IoT ushering in the “yottabyte era” very soon. Cisco estimates that 220ZB of data was generated in 2016 and projects that by 2021, there will be nearly 850ZB from people, machines and things combined.
Now for the interesting part: Cisco also estimates that as much as 90 percent of that data is ephemeral, short-lived data not useful to save. Even if only 10 percent of data is useful, it will swamp projected data center traffic by an estimated factor of four in 2021. Here’s Figure 24 from the Cisco white paper:
Edge computing, or its close cousin fog computing, allows two things to happen: It can pre-process ingested data, ironing out protocols and format differences such that everything arriving at the cloud is immediately ready for algorithms. Any savings in the cloud from not shuffling data before computation are valuable, increasing processing and network bandwidth availability. Parts of an algorithm could easily be farmed out toward the edge, perhaps decimating or filtering data before shipping it upstream for heavy analysis.
A bigger use for edge computing may be using algorithms to avoid shipping ephemeral data upstream altogether. Using generative models at the edge could make a huge difference in identifying useful data. Most of us have heard that no human can sit and watch hours and hours of surveillance video from multiple sources looking for one moment of a problem developing. Consider another example with an EKG monitor attached to a smartphone: While the patient is at home, at rest and unstressed, the EKG is in a perfectly normal range as learned by the application. Doctors are alerted when readings slip out of normal range, with raw data transmitted for observation if required. A similar example is waiting for a motor to fail while monitoring its operation.
Both the LF Edge initiative and work happening in the OpenFog Consortium are creating specifications for edge computing. These initiatives are aimed at interoperable, secure architectures where cooperative processing mines data. If items of interest can be extracted efficiently from useful and ephemeral data at the edge, loads on the cloud and networks would be lighter. Edge computing will likely detect exceptions more quickly, bringing people in to look at what is going on faster. The result will be more scalable IoT applications with better chances of delivering value from all that data.
Normalization of infrastructure through open source technologies is one way to make the move to the cloud an easier process for organizations.
For many modern companies, growth often means growing through acquisition. As organizations become larger, it often makes sense to acquire existing companies who are already creating the solutions they need than try to build them from scratch. However, this can often create a variety of conflicts and challenges—one of which is around infrastructure. For other companies that have grown more organically, it often can be difficult to modernize their legacy systems and bring their technology stack into the 21st century.
But cloud comes with big benefits: Some studies estimate a cloud-based infrastructure can reduce IT overhead costs by as much as 40 percent while also giving IT leaders a flexible way to scale up and down as needed. It also lends itself well for automation tasks, helping free valuable development time to add meaningful value to applications. And, the cloud isn’t just for tech startups. For example, a recent McKinsey & Company study noted that financial services companies have the highest percentage of server images deployed in private or public clouds, approaching nearly 100 percent (versus a median adoption rate of 19 percent). Cloud is certainly a hot topic, and for good reason.
In an ideal world, the IT landscape would be a greenfield, where every organization could start with a blank slate and build out their optimum infrastructure. But for most businesses, it is often more of a brownfield—muddied with old systems and stodgy practices—and it’s not always clear the best way to build on top of it. As more and more organizations look to a hybrid cloud approach as the infrastructure mode of the future, the need for a way to migrate legacy systems more easily is becoming an increasing necessity.
One of the ways to approach this is through the concept of normalization. This is a focus on utilizing industry-leading open source stack technologies that can create a hybrid cloud solution that is consistent and just works immediately: Kubernetes for containers, Fission for serverless functions and OpenStack for VMs, for example. As anyone in IT knows, this goal of normalization is nothing new, but achieving it can go a long way in helping your organization move into the cloud.
The reason behind this focus on providing a solution that leverages open source technology is the goal of creating this normalized structure that works across different cloud providers. This approach is because all the big three cloud providers have their own special services: Google has TPUs, Amazon has Lambda and Azure has Dynamics CRM. As you move data centers into the public cloud, you still have islands of infrastructure that are all built and run differently. How can DevOps truly have a consistent way of building and operating software in this sort of atmosphere? Open source technologies are a very powerful way to normalize that because they provide a solution that plugs into any infrastructure that you want.
One of the other keys to help bring your legacy systems into the cloud is to embrace change as it comes. While this is obviously a less technical recommendation than the above, it is still an important component to achieving success. It is no surprise that this sort of approach is one of the main tenets of the DevOps community that dates back to the beginning of the philosophy. It really started with the Puppets and Chefs of the world in the late 2000s and then continued with the rise of the OpenStack community. Finally, containers really started to form, from Vagrant and now to Kubernetes. Along this route, the IT industry has become much more sophisticated and there are a greater number of opportunities to achieve the normalization necessary to ease the transition to cloud—but only if you’re flexible and willing to adapt to change.
A related important distinction is to understand that complexity exists, and it’s not feasible to eliminate that complexity. Rather, you can take an approach of subdividing it and trying to contain and limit complexity. The strategy of normalization allows you to do this and keep complex parts of your infrastructure isolated as much as possible.
While there are many approaches to moving legacy systems into the cloud—from rehosting (a lift-and-shift approach) to refactoring (completely re-imagining how the application is architected and developed)—turning your current infrastructure into a cloud by leveraging existing and powerful open source tools is one of the most effective approaches that extract the most value for many organizations. This hybrid cloud approach gives organizations the ability to not only leverage cutting-edge cloud solutions that are agnostic to provider, but also provides a real way for companies to continue utilizing legacy systems that still provide meaningful value for their applications. This flexible, bespoke approach to hybrid cloud enables an entirely new approach to software infrastructure and opens many new doors for sustained success.
Here’s a full discussion on this topic with Alan Shimel, editor in chief of devopsy.kinsta.cloud, posted on DevOpsTV:
Moving to the cloud is becoming a must for many companies if they want to grow beyond their current footprint and be more innovative. But for many companies—especially those in the SMB space—their transition to the cloud is hindered by a lack of knowledge and experience, which often results in difficult migrations that sometimes just aren’t successful.
Since its inception, nClouds has helped organizations of all sizes move to the cloud successfully. Its experience as an AWS Premier consulting partner and its well-architected approach to cloud migration helps companies get the most from their infrastructure by engaging with the cloud, whether they’re adopting a hybrid environment or going all in on cloud.
In this DevOps Chat, JT Giri, nClouds founder and CEO, and I speak about the triggers that push companies to adopt cloud, some issues companies face in their decision-making and infrastructure’s role in an increasingly cloud-based IT world.
As usual, the video is immediately below, followed by the transcript of our conversation.
Transcript
Alan Shimel: Hey, everyone, it’s Alan Shimel, and you’re listening to another DevOps Chat, or in this case, perhaps, watching a DevOps video. Happy to be joined here by my friend J.T. Giri, who is the CEO and founder of nClouds. J.T., welcome.
J.T. Giri: Hey—good morning, Alan. Thanks for having me.
Shimel: My pleasure, nice to have you here. J.T., let’s first of all say Merry Christmas, happy new year.
Giri: Thank you, and you.
Shimel: Yeah, it’s been an exciting—we were talking off camera, off mic—it’s been an exciting quarter for you. You’re a new time dad and moving homes and at the same time, nClouds’ business seems to be just booming along with everything else associated with Amazon Web Services and the cloud. So, congratulations.
Giri: Thank you. Exciting times.
Shimel: Yeah, it is. So, J.T., I wanted to talk a little bit about—you know, there’s people listening to this or watching this who are saying, “You know, I wanna be like Google,” or, “I wanna be Facebook” or, “I wanna be one of these unicorns in that I wanna harness the cloud. You know, we have our servers in a closet here or a data center or a third-party data center, but we’re not really cloud. And everything seems to be in the cloud. We wanna move to the cloud, but you know, we’re only maybe 250, 500 employees. How do we do that, right? Do we need a helper, like an nClouds. Do we need to update everything? Do we throw out the baby with the bath water and start new?”
I want you to tell us a little bit, sort of, what’s the typical sort of customer journeys that you see when you guys engage?
Giri: Yeah, absolutely. So, Alan, you know, we’ve talked in the past. I’ve been migrating companies to AWS ever since EC2 was in beta. And, as I would migrate these companies, people would realize that they need someone to manage it, right? So, that’s how, sort of, nClouds came to be. And as you mentioned, this has been an incredible year for us. We doubled the size and revenue and the number of people.
What we were seeing in an SMB type of market, when companies migrate or when they try to migrate to AWS, they still need to follow—obviously, they need to follow best practices because cloud or AWS is constantly changing, and the skill set is just not there. So, that’s where we come in play. We can make sure that your infrastructure is well-architected. We take an approach of migrating in a way where you’re really taking advantage of the cloud—so, for example, using containers, things that are really gonna help you to harness the power of cloud. And then, for most of our customers, you know, we’re providing 24/7 support. And when we see people try to do this themselves, they get a bad taste of cloud. They’re like, “Oh, this is not for us. We’re not really innovating fast.”
So, that’s why, Alan, we’ve been very successful, because there is a huge gap in SMB type of companies. There is a skill gap and just experience. We bring that to the table, and it’s been incredible. We’re able to add a lot of value.
Shimel: Sure. So, J.T., early on there, you said something about when an organization needs someone to help them manage their cloud migration—when does an organization recognize that they need someone to help them? What’s sort of a trigger point? You know, as, again, we spoke off camera a little bit—is it when someone dives in the pool and says, “Oh, my God, I can’t swim, it’s too deep?” Or is it, before you dive in the pool, a prudent person hires a swim coach, you know what I mean?
What are sort of the trigger things that people should see as a red flag or as a trigger event that says, “Hey, man, I should be getting some help, here”?
Giri: Yeah, that’s a great question. So, basically, it depends when the companies come to that realization that we need some outside help, right? Sometimes, it is that they will try to do the project themselves, or in other cases, they realize that they really want to just focus on delivering value to their customer, right? They really wanna just innovate fast.
So—and they see how they can just stay focused on that, and that’s something we do incredibly well. We have a build—we build systems and processes together where our key focus is to make sure the customer can deliver value to their customers fast and we want to make sure the infrastructure is not getting in their way.
So, you know, it’s just, this is becoming more and more popular, like, DevOps as a service, where companies are reaching out to partners like nClouds to help them with DevOps related tasks, like, with the migration, setting up CI/CD pipeline, becoming an extension to their team, so they can focus on their core business, right? And, you know, it really depends when they—sometimes, they come to that realization pretty fast, or sometimes they try to do things themselves, and when things don’t work out, they try to use someone like nClouds.
Shimel: So, J.T., generally, one of the hardest things in life is learning when to ask for help.
Giri: Right.
Shimel: And that includes cloud migrations and everything else. At the SMB level, mid-enterprise level, you don’t have an elaborate or as big a structure as, let’s say, you do with a large enterprise where you have layers and layers of management and hierarchy—a lot of the organizations you deal with are rather flat in their management structure.
So, who is it in the organization who finally kinda raises his hand and says, “Wait a second, I think we should engage in some professional help, right? I think we should being in some experts to help us.” Is it the CEO, CIO, someone lower than that? What’s your experience?
Shimel: Yes, normally, CTO, VP of Engineering type of person. And also, you know, as you know, we became a premier partner with AWS last year. So, AWS is—Amazon, it’s no secret that they’re very customer obsessed. So, they will—Amazon, when they’re speaking to a customer and if they see the customer is struggling, they will introduce someone like nClouds to help them out with the migration or just modernize their infrastructure. And sometimes, you know—actually, many times, they can come in with funding where they will offset some of the costs.
So, for a customer, it’s just like a no-brainer to see if we can have some external experts come in and fix up the infrastructure or streamline the infrastructure. So, you know, the conversation, normally, we do get a lot of business directly from people reaching out, like, “Hey, we need DevOps help,” but we also get a lot of business directly from AWS where they identify this customer needs help, and we get involved that way.
Shimel: So, I’m just gonna bring up two questions. The first one is on track with what we’re talking about. So, now the CTO or CIO or an executive says, “Yeah, we’re gonna bring in nClouds,” whether AWS referred it or is partially subsidizing it or what have you, but, “We’re gonna bring in nClouds or some other type of help”—is there any sort of pushback or resistance normally with, you know, when you’re dealing with people? The—not first responders, but the individual contributors who feel like, “Hey, man, I don’t wanna be replaced. I don’t wanna be outsourced. I don’t want to give up my cheese to some outside company who’s helping us” or, if so, how do you overcome that? How do you work with an organization like that?
Giri: Yeah, so, normally, there’s not a pushback, Alan. So, I was actually re-reading “The Venus Project” yesterday, and I was just reading the story of Brent, right?
Shimel: Mm-hmm.
Giri: So, when you’re talking to these customers, some of these guys are on their honeymoon with their pager with them, right? And then, when you’re coming in and you’re saying, “Hey, we’re gonna help you out with 24/7 support so you can get your life back” or, you know, “We’re gonna help you to streamline this DevOps work which you never really got the opportunity or time to work on, we’re gonna help you out with that,” right? And it’s incredible.
Sometimes, maybe the initial impression might be the same as you describe, like, “Hey, are you gonna take my job?” But as they understand the value, they normally are super happy. And another thing we do, Alan, in most of our engagements, we do this assessment called architected assessment where we’re coming in and we’re just reviewing your infrastructure to see how well you’re architected, and after that assessment, we give you, like, a roadmap, like, what are some things you should focus on the first month, and what are some things you should focus on short term and long term? And the assessment goes over cost, security, reliability, performance, and operation excellence.
So, you know, we can get, in the short-term road map, we can show some quick wins where the customer sees the value and they also get confident in our ability, so that’s sort of the path that works really well for us right now.
Shimel: Got it. J.T., another thing you mentioned is that nClouds is a premier Amazon partner. And, you know, in today’s world, we throw out platinum and gold and these sort of levels all the time—but really, within the Amazon hierarchy, a premier partner has sort of, they don’t hand that out like candy. Talk to us a little bit about what’s entailed in being an Amazon premier partner.
Giri: Yes. It’s a top tier status with AWS and in order to get there, like you said, there is only 25 premier partners in North America, and you have to show case studies, you have to show the training path of your employees, and you also have to have multiple competencies, like, we have DevOps competency, we have migration competency, and we’re also the well-architected partner.
And it’s just very subjective. You know, if you work closely with Amazon, even if you hit all the check boxes, they want to make sure that quality is there for premier partners. So, they look how you work well with your customer, what does your customer experience looks like, because that’s what Amazon cares about that as well.
So, you know, it’s just very hard to get to this status, and we’re honored to be at this level right now. This is great.
Shimel: Yep. So, J.T., let me go next kind of where I want to take this, and that’s—you know, two or three years ago, migrating a company onto an AWS, or it doesn’t have to be AWS, migrating a company onto the cloud, you know, you set up some virtual instances, maybe they had some separate storage, stuff like that, EC2 kinda stuff. But, you know, I just got back from Seattle a couple weeks ago from KubeCon and CloudNativeCon—Reinvent was the week before that. And certainly, Kubernetes and containers are kind of really quickly, quickly, quickly changing the landscape, here.
You know, are you seeing that as well? And what does that mean for—typically, SMB businesses are not leading edge or bleeding edge, right in [Cross talk] option?
Giri: Yeah.
Shimel: What are you guys seeing?
Giri: Yeah. You know, it’s very interesting. Who would’ve thought that the infrastructure we built a couple years ago would be considered a legacy infrastructure, you know? People using EC2 instances and an auto scaling group in Amazon’s world will kinda be called outdated right now.
So, we’re seeing incredible push and a lot of sort of initiatives on our customer side where they wanna go containers, pretty much like 90 percent of our engagements are around containers. Because it’s a no brainer, right? It makes the CI/CD pipeline much easier. You know, most of the time, containers are automatically integrated with scaling, so you can really binpack your resources, it has impact on your cost. So, you know, this is just—it just makes the infrastructure far more streamlined and far more easier to manage.
And as I mentioned, Alan, I’ve been migrating companies to Amazon and EC2 from the last 10 years, and I’ve been managing infrastructure as long as I’ve been migrating. So, it’s just, when I’m building infrastructure right now, it’s a no-brainer to build something using Kubernetes or on AWS we use EKS or—you know, depending on what the customer wants to do, and we had a customer the other day where they were saying, “I just wanna focus on delivering product to our customers, so what is the easiest way?” We use Fargate for them. Depending on the customer requirement, we’ll go Kubernetes, but long story short, it just makes a lot of sense these days to build infrastructure on containers, because they really streamline how you manage it in the long run. It just makes life easier.
Shimel: And that’s even at the SMB level?
Giri: Oh, absolutely, yeah. I mean, because the needs are the same. You need CI/CD pipeline, you need a way for you to scale infrastructure, you need a way for you to run batch or cron job type of jobs. And when you go with something like containers, you get there—like, you know, you don’t have to build home grown tools to facilitate these things. You know, containers or EKS or Kubernetes, they allow versioning when you’re deployed or auto scaling is sort of built in, you know, you can schedule batch jobs.
Just, it’s a no-brainer, you know? If you’re building out infrastructure, you should definitely explore containers each time, and that’s what we try to do every time. And it’s just incredible, the amount of success we’ve had before and after.
Shimel: Sure. So, as we sit here at the dawn of a new year, if I would’ve asked you in early 2018, do you think Kubernetes and containers would be one of the major stories of the year, I don’t know if you would’ve said yes or no. But as we sit here in the dawn of 2019 and we project ahead—let’s say I interview you and everything’s all well and at the end of next year, the end of this coming year, 2019, we sit down and talk about, “Well, what’s 2019 really show?”—what do you think are the big stories this year, J.T.? What do you think, coming down the pike, especially—I’m not talking about the world view. For nClouds and for your customer base, what do you think is really big for you in the coming year?
Giri: Yeah, so, I think in general, the application modernization will continue. We continue to see that trend, and this is also going to be big in SMB. So, we will continue to double down on AWS. It’s been an amazing partner. Our core values, our partnership is based on shared goals, so we’ll continue to kinda double down on that relationship.
And we’ll continue to utilize this well-architected framework. It helps us to assess the customers when we’re onboarding and it helps us to manage the customer, like, every time the customers were managing, we continued to run a well-architected assessment to make sure they’re well-architected, and they continue to stay well-architected.
And, yeah, I mean, just continue to focus on SMB, AWS, continue to align with well-architected framework. We also just spun off another company, it’s called nOps, and basically, it just helps to make sure the customers are well aligned with well architected framework, and it’s available on AWS Marketplace. So, we’ll continue to invest in that, because that solution has been incredible, nOps. A lot of our customers are seeing value in that, so, we are actually working with Amazon to give that solution to other partners like nClouds who can use it to manage their infrastructure to make sure the infrastructure is well architected, and we are also offering it to end customers who can use that tool to assess their infrastructure and continuously stay in compliance with well-architected framework.
Shimel: Excellent. You know, nOps, that’s something we’ll have to discuss on our next chat, J.T.
Giri: Yeah, absolutely.
Shimel: Well, you know what? We’ve been on almost a half hour here, so, unfortunately we’re about out of time for this one. J.T. Giri, CEO and founder of nClouds, thanks for being our guest on DevOps Chat.
Giri: Thank you so much for having me.
Shimel: My pleasure. Best of luck to nClouds this year, best of luck to you personally with your family and everything else, you know, it’s gonna be a busy year for you, my friend.
Giri: Absolutely. Thank you. Thank you.
Shimel: Alright. Hey, J.T. Giri, nClouds—you’re here with Alan Shimel on DevOps Chat. You’ve just listened to another chat. Bye bye, everyone.
Google also plans to localize its cloud learning and certification programs in Japanese, French, German, Spanish and Portuguese to make them accessible to more people.
Rochana Golani, director of Google Cloud learning and enablement for Google, said Google is trying to increase the number of qualified engineers available to manage IT infrastructure as code. Most of the IT operations community is made up of IT administrators that don’t have much in the way of actual programming skills. The Google cloud certifications are all steeped in the site reliability engineer (SRE) principles that Google defined in 2003. As a more prescriptive instance of a DevOps practice, the goal is to make the skills needed to successfully transition to DevOps more widely accessible, said Golani.
Google is especially keen to increase the number of networking and security professionals that have demonstrated cloud skills, as the critical role these functions play in cloud application environments is often underestimated, added Golani.
Those challenges likely will only increase as organizations increasingly embrace microservices based on containers running on Kubernetes clusters that eventually will be tightly coupled to serverless computing frameworks. In fact, the first era of the cloud was defined by a comparatively simple replacement of virtual machines from on-premises IT environments with similar virtual machines residing on a shared public infrastructure. The next wave of the cloud is being defined more by so-called cloud native applications constructed using microservices. That approach is inherently more flexible from a business perspective, while paradoxically being more difficult to manage from an IT perspective.
On the upside, however, DevOps in general and SRE disciplines more specifically enable IT organizations to operate more hand in glove with the business as it grows and evolves.
Of course, none of that is achievable without the skills required to achieve that goal. Savvy IT professionals naturally will pursue certifications that enable them to earn higher salaries. But organizations that encourage their personnel to achieve those certifications should be able to at the very least engender more loyalty among IT professionals they help advance their skills.
In the meantime, it should be apparent that transformation is all but inevitable. Not every workload will move into to the cloud. But the principles applied to managing workloads in the cloud will be increasingly applied within on-premises IT environments. Those on-premises IT environments may not be as programmable and resilient as a public cloud, but the need to process data locally is not going to go away anytime soon. The challenge IT organizations are facing today is how best to define a set of best DevOps practices that can span both environments.
Migrating to the cloud isn’t as simple as signing up for a SaaS application or deploying applications on a cloud infrastructure or platform; there are other factors at play. And the move to cloud shows no signs of slowing, with Gartner predicting the worldwide public cloud services market to grow 17.3 percent in 2019. Getting cloud deployments right is more critical than ever.
As organizations shift more to the cloud, it means they will increasingly rely on networks and infrastructure they don’t own and directly manage. Yet, this infrastructure is just as critical to consume and deliver the applications and services as when it was in the data center. Being able to explore these networks to identify choke points and routing issues in advance helps to inform better network investment and configuration decisions. It is essential for a successful cloud deployment as well as ensuring an optimized environment on an ongoing basis.
There are six key network considerations IT managers should take into account before shifting to the cloud.
Baseline Performance Before Deployment
Getting a baseline measure of network performance when moving to the cloud, first and foremost, requires adopting a different set of data points than organizations have typically used in the past. The move to IaaS, SaaS—or any cloud service for that matter—means organizations are beholden to those providers as well as third-party service provider networks through which the application or service traffic traverses.
Prior to cloud adoption, the network was essentially under an organization’s control. And it’s not that networks are necessarily less complex, it’s just that network teams had access to all the network data like packets, flows and device level information to monitor performance and security. And when issues arose, teams had the ability to troubleshoot and triage problems. Traditional monitoring approaches worked in a world where the network was managed and contained and boundaries were known. However, these data sources are no longer available for monitoring the performance of the cloud network, where devices can’t be directly instrumented or polled. Visibility into cloud and third-party networks can only come from data sources such as synthetic and end user monitoring. These techniques can provide quality of experience metrics that network teams can rely upon to test configurations and baseline performance before moving to the cloud.
See and Understand Bottlenecks in your Infrastructure
It’s very possible that, based on existing configurations and routing policies, certain offices and sites are not optimized to consume applications or services over the internet. For example, a branch office in India may have issues accessing Salesforce because of transcontinental latency and a bandwidth constrained MPLS circuit. Similarly, a branch in Austin may have better and more reliable access to applications being served from El Paso rather than San Antonio.
But how do you know what the ideal configurations are before deploying? Time to have a look at your new data sources. The first in your arsenal is actually derived from a technique you may already be familiar with for monitoring application experience in terms of page load and user transaction timings. Synthetic monitoring can be used to gain insight into infrastructure and network performance as well as your application. What’s great about synthetic monitoring is that it works in cloud environments and across the internet in addition to your data center for in-depth information about how each portion of the network is performing.
Map Out Real Traffic Paths
Having visibility into specific areas of a distributed network that are problematic, as well as pertinent details that can help determine a root cause, results in more tangible and actionable information for the network you own and those of your providers. To achieve a more complete picture of how traffic is traversing the cloud network, it is essential to combine synthetic tests with ping and traceroute functionality. Now network teams can not only test the reachability of an endpoint or host through the cloud, but also use that data to determine the route or path of packet through a distributed network while measuring transit delays.
Another useful data point is Border Gateway Protocol (BGP) routing table information. BGP knits the internet together by exchanging routing information between networks. And these routes determine how traffic flows to or from your apps and services. Layering this information into your diagnostic approach helps understand how routing configurations and changes impact reachability, latency, loss, jitter or other performance metrics.
Collaborate with SaaS and Cloud Providers
So, when it comes time to work with your your SaaS, ISP and cloud providers, it helps to share this information, related visualizations and diagnostic information in an easily consumable way. That way, when a trouble ticket is generated, it is dealt with much more quickly by the vendor or provider. Knowing exactly where a problem exists and the related cause analysis empowers teams to take immediate action and enable the responsible party take immediate action as well.
Get Things Right Before Deployment
With detailed visualizations and hop-by-hop metrics, it becomes possible to try out a routing change, plan for a new data center or test a roll out of a new application or SaaS service. Synthetic testing layered with contextual data and a detailed visual network topology allows teams to test performance and gain insight into initial infrastructure configurations, plan changes and understand their impact on cloud applications to get things right before deployment.
Continually Monitor the Performance of Your Network and Its Impact on Applications
The same techniques and data sources teams are used to baseline performance and achieve optimal cloud configurations can also be used on an ongoing basis to ensure continued performance. Active monitoring with additional layered information allows organizations to constantly keep an eye on performance degradation to optimize end user experiences across their network. And the additional inclusion of visual analysis reduces mean time to troubleshoot and repair issues. As the internet and cloud computing reshape the enterprise network, incorporating these new data sources, correlation of pertinent data points with a unified visualization is critical to the successful deployment and management of cloud applications and services.
This virtuous cycle of better performance becomes even more important in the future as organizations increasingly rely on the internet for their critical operations. Growing levels of software-based automation will only increase the number of different network paths traversed and the resulting end user experience. Accurate insights into the entire cloud network enable organizations to troubleshoot and solve their cloud performance problems, alleviating potential cloud migraines and ensuring the business runs smoothly … and painlessly.
Achieving hyperscale is a work of technical art for any business. As companies grow and customers begin to rely more heavily on your service, ensuring reliability during peak business times is mission-critical. An enormous amount of work goes on behind the scenes to ensure scalability, and oftentimes it becomes necessary to redesign or rethink processes and systems to improve them as they face increased pressure.
When customer demand reaches its apex, it’s the ultimate test for any organization’s systems and processes; therefore, it’s important to prepare a comprehensive strategy that focuses on scalability, reliability and hypergrowth in a targeted, calculated and data driven methodology.
The first step on your path to hyperscale is forecasting and gathering data analytics around business and growth patterns for a full year to make smarter forecasts. Set aside time each day throughout the year to go over the allocation of your data and analytics, measuring them for accuracy to see how they stack up to your current predictions for that specific day. This may seem taxing and cumbersome when engineers have other priorities to work on, but devoting time to this in the early months of the year will help you be that much more accurate when it comes to predicting the needs of your customers and how you should go about scaling to meet those needs.
Especially during the holiday season, it’s important to have a minute-by-minute projection of how much traffic you expect and need. That’s where forecasting and an advanced data science model come into play. A common mistake made when it comes to scalability is not getting granular enough about the real pressure points for your systems. Having accurate projections and forecasts in place that feed into your data science model means you can better test your systems and production limits while gaining greater insight into meeting your scaling points. If you find fragilities ahead of time through this, you can assign your engineering team to fix them well in advance.
Another way to truly bolster your scalability efforts is by introducing an operability development model. This entails constantly deploying some controlled chaos into your systems to see how they really stack up and function under pressure, with the goal of up-leveling the capacity of both your operations and system growth. The operability development model allows you to really test the grit of your systems by squeezing the most out of them, prepares you to respond quickly and methodically should the worst case scenario occur and sets you down the path toward continued performance growth and scale.
From here, you’ll want to institute a preventative maintenance model. Think of it like getting an oil change for your car. Instead of waiting until the very moment you need your software to perform exceptionally, start taking a look at the little things that could or are causing problems ahead of time and fix them on a monthly or quarterly basis. A preventative maintenance model helps you inspect, adapt and solve these little nuisances before they become serious problems, granting you the cleanest operating environment possible.
Scalability is king, but availability and reliability are arguably just as critical. Improving availability means reducing latency and getting as close as you can to where your customers utilize your services. This can be achieved by deploying locations around the world to make connections much faster and services more independent. To maintain reliability, you should have processes in place that can quickly respond and offer support to customers 24/7 to maximize consistency and performance.
Another strategy to consider—if it makes sense for your business or if you’re evaluating cloud providers as a growth option—is to double down on your efforts by re-architecting your platform and systems as cloud native from the ground up to support greater levels of scalability and cost efficiency. By going cloud-native, you can achieve independently scalable services so that you don’t have to scale all of your services together. This means your engineers and developers don’t have to spend way too much time figuring out where issues and fragilities occur.
A platform architected as cloud-native allows you to leverage microservices and containerization, creating an environment with more reliable, self-healing, high-tenancy and auto-scaling features that take the most advantage of your hardware and services. An added benefit of a cloud native platform is that you can use your data to find the most opportune times to both scale in advance and descale based on performance and customer expectations. The efficiencies that come from re-architecting as cloud native enable engineers to spend more time on automating software and the delivery pipeline, which is a huge boon for scalability.
At the end of the day, a well-thought-out preparation strategy for mass scalability makes the whole process much more exciting and obtainable. It also greatly reduces lead-up anxiety by getting a head start on planning so that you have more time to sit back and enjoy the fruits of your labor. Lastly, a yearlong scalability strategy enables you to experiment and try new things without the fear of failure. Testing new approaches ahead of time to see what does or doesn’t work will only help you learn and make your scalability efforts that much more effective for the future as your company continues down the path of hypergrowth and hyperscale.
Driven by the encouragement from 2018 progress, cloud technology is poised to be even bigger in 2019. However, one major hurdle continues to haunt the cloud trend: security.
An overwhelming number of firms in the IT industry are preparing for cloud adoption, yet security continues to be a big question for many of them. Early DevOps adopters faced a similar challenge in their early stages of DevOps implementation. They relied on a wide variety of DevOps tools to address their issue completely.
Today, however, DevSecOps is making a big difference in the IT industry, including security and ensuring a seamless software development life cycle (SDLC).
Breaking the regular trend of having security as a separate process, DevSecOps calls for security integration across all stages of the software process chain, addressing security concerns at the very first instance of every stage.
The same DevSecOps now comes as a savior for cloud, too.
Applying DevSecOps to Cloud
Last year saw the cloud market taking a new position in the IT industry, driven by the wide penetration of leading cloud computing service providers: Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), Oracle Cloud and IBM Cloud, among others.
To stay relevant in today’s market, many IT firms have begun their efforts to achieve high and highly scalable performance with all-round digital transformation.
Yet, security limitations associated with cloud continues to be a concern many of these organizations. DevSecOps principles can help. Many research reports show that the majority of firms working on the cloud have high reliability on DevSecOps tools and principles for improved agility and high reliability.
DevSecOps approach to cloud security requires detailed planning that might even demand cultural change in an IT environment, especially for security automation and configuration of cloud assets.
The planning requires security teams to:
Collaborate with development teams as they move code to cloud alongside close monitoring of quality in the production cycle.
Work with the quality analysis and development teams in deciding qualifier and parameter aspects required for code promotion.
Overall, DevSecOps principles offer security advantage along with software agility and high reliability across the life cycle.
Implementing DevSecOps in Cloud
Following six factors decide the success of DevSecOps implementation in a cloud environment:
Code Analysis – Continuous improvements to software mean revisiting code, which also reflects in code analysis, quality assurance and delivery cycles.
Automated Testing – Automated testing minimizes efforts and also saves time. As a key aspect of DevSecOps process, automated testing makes the testing process easier through easy execution of repeatable cases.
Change Management – Linking teams and making them aware of each other’s operations, wherever required, is an important aspect of change management. Keeping developers informed about security-related activities helps in timely address of existing and possible vulnerabilities.
Compliance Monitoring – Compliance continues to play a key role in an organization’s growth path as part of corporate governance. Regulations help in the creation of code and also in modifying the source code. This helps in real time at times of audit.
Threat Investigation– Threat investigation is important to defining the security readiness of any organization. It’s important for organizations to have a close and continuous watch on discovering possible threats, regular security scans and code reviews to address security challenges.
Personnel Training – Holding hands-on training sessions and certification courses build the company’s strength, equipping teams with appropriate domain knowledge.
The above-mentioned six steps act as building blocks for implementing DevSecOps strategy in cloud. What’s next? The implementation part: Get DevSecOps to cloud.
In 2018, digital transformation ushered in a radical shift in how enterprises harness customer insights, technology capabilities and rapid experimentation to drive revenue growth, profitability and market leadership. Enterprises spent $1.3 trillion in 2018 on digital transformation technologies such as public cloud platforms, microservices and containers, edge computing, machine learning and artificial intelligence to improve customer experiences, business agility and employee engagement.
Heading into 2019, digitalization will accelerate business model innovation and drive the adoption of these emerging technologies for sustainable competitive advantage. Here’s a closer look at five technology trends that will change the way enterprises approach digital operations management in 2019:
Multi-Cloud Adoption Goes Mainstream
In FY18 Q3, the three leading cloud platforms (Amazon Web Services, Microsoft Azure and Google Cloud Platform) accounted for 65 percent of the market share of public cloud infrastructure and platform services. Enterprises are increasingly relying on multiple cloud providers to avoid lock-in and harness innovative product offerings, global presence and flexible pricing strategies. When it comes to multi-cloud adoption in 2019:
Lift-and-Shift Will Give Way to Rebuilding. In the initial phase of enterprise cloud adoption, CIOs preferred “lift-and-shift” strategies for migrating complex, legacy applications to public cloud services. In 2019, enterprises will rebuild legacy portfolios to truly take advantage of the cloud’s elasticity, security and utility pricing. Rebuilding is not only the right approach to modernize legacy systems, but also a viable solution for reducing technical debt and unleashing enterprise agility.
The Rise Of Managed Private Cloud for On-Premises Requirements. While analyst firm Gartner predicts that 28 percent of enterprise spending will shift to public cloud offerings by 2022, the reality is that the vast majority of enterprise workloads still run inside the corporate firewall. Public cloud providers are taking aggressive measures to conquer the final bastion of data center gravity with private cloud offerings. Managed private cloud infrastructure will become essential for enterprise services that will need data sovereignty, latency or compliance, yet still want the flexibility and agility of cloud. Some recent managed private cloud announcements from the big three cloud platforms show how these providers are taking a pragmatic approach to meet customer demand:
In 2016, Andy Jassy, CEO of Amazon Web Services, declared, “Whether it’s 10 years or 20 years, very few companies will own their own data centers, and those that do will have a smaller footprint than they have now.” During his 2018 re:Invent keynote, Jassy sang a different tune by launching AWS Outposts to bring the power of cloud-native services to enterprise data centers for hybrid interoperability and seamless management.
During the 2018 Cloud Next conference, Google introduced its Google Kubernetes Engine (GKE) On-Premises that delivers an abstraction layer for running container clusters either on the public cloud or in a corporate data center.
In 2017, Microsoft announced the general availability of Azure Stack, which offers the same programmable interface and deployment tooling of Microsoft Azure in a private cloud environment for a consistent hybrid cloud experience.
The Cloud-Native Skills Gap Widens
Software containers have fast emerged as the standard deployment vehicle for agile, reliable, scalable and highly available enterprise services. The transition from virtual machine management to container orchestration and monolithic development patterns to agile microservices will transform the way enterprises build, deploy and optimize cloud-based applications. IT operations teams will need to develop a new set of competencies to manage the health and performance of containerized services. IT practitioners can longer rely on manual scripts to manage large, distributed systems but will need to embrace infrastructure as code to provision, deploy, monitor, alert and auto-scale cloud-native services.
Commodity Monitoring Evolves to Strategic Observability
DevOps teams will use multiple data sources (metrics, logs, and traces) to understand, verify and debug cloud-native systems. Modern observability requires the ability to consolidate and extract relevant insights from observability pipelines using tools such as Prometheus for monitoring, Jaeger for distributed tracing and Fluentd for logging. While IT teams will need new tools to manage autonomous cloud-native services, it is still critical to understand relationships between individual components and the overall system to effectively troubleshoot microservices in production.
AIOps Emerges as the Way to Tame Alert Storms
Artificial intelligence for IT operations (AIOps) is a clear necessity for modern IT infrastructure teams to keep up with the diversity, velocity and volume of IT operational and event data. IT teams can proactively prevent service outages, analyze real-time incidents using contextual knowledge and remediate problems using AIOps tools. Here’s how AIOps will help enterprises run and optimize mission-critical systems:
Improve Response Time For Digital Interactions. AIOps has the power to boost key metrics for incident management including mean-time-to-detection, mean-time-to-response, mean-time-to-restoration and incident volume handled within a service window. The combination of machine learning and data science techniques in AIOps not only delivers faster incident coordination and response, but also reduces the human time spent per alert with advanced analytics and probabilistic root cause analysis.
Eliminate Siloed/Redundant Processes. AIOps offers the ability to consolidate event and incident insights from different IT management tools across on-premises and public cloud environments. A shared AIOps platform offers centralized visibility, faster impact analysis and improved collaboration for a diverse set of stakeholders, including application owners, infrastructure teams and business sponsors.
Business-Aware, Service-Centric Delivery Can Optimize Customer Experiences
To help the business remain competitive, IT teams simply must transition from managing hybrid IT infrastructure assets to prioritizing and supporting business-critical services. Service-centric operations management combines business insights, customer knowledge and relevant metrics to improve digital performance, enhance customer interactions and support business goals. IT teams will need to focus on the business value of IT services while offering holistic infrastructure visibility and control at the business process level.
Clear strategy and focused execution will separate the digital winners from losers in 2019. How enterprises embrace and respond to these key trends will be key to unlocking disruptive innovation, operational transformation and business value.
