In most cases today, resolving performance issues will require architecture changes or code refactoring
Imagine you’re a software reliability engineer with your service-level objectives clearly laid out on your Grafana dashboard. Suddenly, an alert appears: Your 99th percentile latency is going through the roof!
A quick glance at the relevant metrics reveals the culprit: A spike in traffic has exceeded the current system throughput. It’s clearly an issue caused by a lack of capacity, which adding a few extra instances to the cluster should solve, right? After all, more machines mean more capacity. You remain calm as the charts stabilize and the alert fades away. There’s no cause for concern; you automated the scaling process a long time ago.
But this story could have resulted in a very different, far worse scenario. Capacity is easily accessible these days, especially in cloud environments that offer simplified scaling processes. But the ease of provisioning and scaling also incurs rising infrastructure costs.
In many cases, handling performance issues or improving performance can’t necessarily be achieved simply by adding more machines due to application bottlenecks or resource management inefficiencies. To paraphrase a wise uncle: With greater compute power (adding more machines) doesn’t come greater performance. Unfortunately, in most cases today, resolving performance issues will require architecture changes or code refactoring.
Optimizing the Software
Since increasing the node count rarely helps with improving the performance, let’s explore what can be accelerated. The stack is composed of hardware, an operating system (OS), libraries and the application, among other components. Making improvements at the hardware level is not always feasible, especially when running in the cloud.
Rather than spending more on your cloud bill, consider checking how fast the userspace code is. Application developers have a lot on their plates already, so there is only so much they can optimize. While it is possible to invest R&D efforts and time in replacing a poorly performing library with a more performant one or doing the occasional performance-focused rewrite in the hopes of resolving the issue, it’s possible neither will work.
When fidgeting with the hardware isn’t an option and the developers are unavailable to help, another option is to attempt to address the problem at the OS level.
A Trip Down Memory Lane
The history of computing has not only been about smaller transistors and faster clocks. Back in the mainframe days, machines ran a single program at a time that was encoded on a punch card and inserted by a computer operator.
Then businesspeople came along looking for a way to make computers more efficient since, during a program switch, the machines remained idle. This led to the creation of the operating system, a program that executed other programs and managed resource allocation between them. Operating systems were designed for users behind the keyboard running simultaneous tasks; therefore, they were designed to provide this illusion of parallelism by optimizing internal resource management to achieve high interactivity and fairness.
Today, it’s not uncommon to command a fleet of generic virtual Linux boxes that are focused mainly on running a specific application, a microservice. But the OS underneath hasn’t changed much. It still behaves as if it is supposed to execute multiple programs and share resources between them, which isn’t necessarily the most efficient in such a case and doesn’t provide optimal performance for the application.
Optimizing the OS
There are a number of potential performance improvements that can be tested and applied directly at the OS level. Tuning sys controls can have a significant impact on the performance of many subcomponents, such as networking.
The I/O scheduler may be worth looking into as well. For example, with databases, the default CFQ (Completely Fair Queuing) can yield results that are inferior to those of the deadline scheduler. On the other hand, a no-op scheduler allows you to avoid having to schedule I/O operations twice in cloud environments. After all, the VM’s hypervisor often manages the hardware already.
No matter what your approach, though, constant meticulous measurement in a production-like environment (or even the production itself, if you’re into chaos engineering) is recommended. Performance tuning is a highly advanced field that requires specialistic system knowledge.
Advanced OS Tweaks
Some OS features are not so easily accessible for performance tuning. For example, the Linux process scheduler utilizes the Completely Fair Scheduler (CFS), which is perfectly sensible in most cases. However, it can sometimes create a significant performance gap that cannot be easily found with standard profiling tools. And even if you do discover it, you can’t simply change a parameter in one of the configuration files; rather a kernel patch and a rebuild are required.
Let’s say you’re perfectly fine with the algorithm, but you’d simply like to state that some threads are more important than others. By default, you can’t do this, as the niceness setting only works at the process level.
I/O-bound applications are also complicated. Even when using raw sockets and epoll, there is no runtime mechanism to provide selection logic or even a priority to the sockets in the queue. And there is no way for a kernel to know the performance budget for a request.
Additional Means for Performance Optimization
In an ideal world, your application will consist of purpose-built operating systems tailored for each microservice and exploiting every opportunity to boost performance and synergize with the application. In such an ideal world, the internal resource management mechanisms within the operating system will be tailored to the application-specific utility function to drive optimized performance and, in turn, also deliver reduced infrastructure costs.
Unfortunately, we’re not there yet, and such solutions are currently only available to corporate giants who can afford to hire a few dozen people to do just that full time. In the words of William Gibson, “The future is already here – it’s just not evenly distributed.”
So what is left to those with finite budgets? A new approach for real-time continuous optimization that enables organizations to leverage AI-driven infrastructure optimizations that are suited specifically to the running workload. Using application-driven scheduling and prioritization algorithms, it is possible to identify contended resources, bottlenecks and prioritization opportunities and solve them in real-time.
The extension of a partnership between Cisco Systems and Google to drive tighter integration between their respective software-defined wide area network (SD-WANs) offerings and cloud platforms promises to accelerate the convergence of NetOps and DevOps.
John Apostolopoulos, vice president and CTO for Intent-Based Networking Group at Cisco, said in the wake of the COVID-19 pandemic it’s clear more organizations will be employing SD-WANs as an alternative to virtual private networks. Many of those SD-WAN deployments, however, will be on public clouds rather than in a traditional on-premises data center, he added.
As that transition occurs, DevOps teams will increasingly invoke SD-WAN services via application programming interfaces (APIs) residing in the cloud to provide users with access to cloud applications, noted Apostolopoulos.
Cisco and Google have a technology alliance in place that spans everything from Kubernetes to cloud management software. That partnership is now being extended to include forthcoming deployments of Cisco SD-WAN software on Google Anthos, a cloud platform based on Kubernetes that can be deployed on any cloud or on-premises IT environment, sometime in the first half of 2021.
The availability of SD-WAN services from Cisco on multiple cloud platforms will naturally have significant implications for NetOps teams. Rather than having to manage individual routers and switches, the future of SD-WANs is moving more toward being a set of software services that are programmatically invoked. It’s not clear to what degree NetOps teams might be folded into DevOps teams as that transition occurs. IT teams will always require networking experts to optimize networking services to one degree or another, said Apostolopoulos. However, the days when DevOps teams had to wait days or weeks for a NetOps team to manually provision network services on their behalf are coming to an end.
Apostolopoulos said in the wake of the COVID-19 pandemic, flexibility has become a highly prized IT asset. As such, more applications will soon be deployed in the cloud than ever. Cisco is betting that rather than having to manage multiple cloud network services, more organizations will prefer to extend enterprise networking services from Cisco across multiple clouds. That approach will enable IT organizations to more easily centralize the management of network services via a management console built on top of a consistent set of APIs, he noted.
Just as importantly, Cisco will provide the integrated tools needed to secure those network services, added Apostolopoulos.
In effect, Cisco is betting that enterprise IT organizations are going to consolidate networking and security services as part of an effort to reduce the total cost of IT. Cisco may not always be the least expensive provider of an individual router or switch. However, the total cost of integrating networking and security offerings from multiple vendors can add up quickly. The degree to which organizations have the political will and economic capital required to achieve that goal, of course, remains to be seen. Nevertheless, it’s also clear that maintaining the current networking and security status quo is not going to be able to hold for very much longer.
Chef today updated its tool for automating the management of IT infrastructure as part of an effort to further eliminate the need to rely on manual processes that generally fail to scale.
John Wyss, vice president of product for Chef, said Chef Infra 16 adds a Unified Mode that provides a more streamlined means of executing policies and adds support for YAML, which means IT teams can now employ Chef to automate the management of infrastructure without having to write code.
In addition, Chef has made it easier to update legacy instances of cookbooks built using Chef and added support for Arm processors.
Interest in automating the management of IT infrastructure is expected to accelerate considerably in the wake of the economic downturn brought on by the COVID-19 pandemic. Many IT teams are likely to be considerably reduced in size. The remaining IT staff will need to automate the management of IT at much greater degrees than ever before. Much of that infrastructure soon may also be residing in a public cloud rather than in an on-premises IT environment.
Of course, Chef is not the only way to automate the management of IT infrastructure. However, IT professionals who master Chef typically are able to create workflows that automate a wider range of tasks.
As an early pioneer of using code to manage infrastructure, Chef has been making a case for employing the Ruby programming language to make it easier to manage IT infrastructure at scale. While adoption of Chef largely has been limited to IT professionals who could master Ruby, Wyss noted those individuals who have mastered Chef will be in much higher demand as organizations look for IT professionals who have the skills required to manage tasks that otherwise would have required a much larger staff.
It remains to be seen to what degree IT professionals might embrace IT automation frameworks further. Many have developed their own scripts to automate a task. The challenge IT organizations have encountered is that those scripts are usually poorly documented and generally fail to scale. Automation frameworks provide a means to consistently manage IT infrastructure at scale in a way that is easily repeatable.
In the meantime, the debate of who inside the IT organization will automate the management of infrastructure continues to rage. Some IT organizations want developers to be responsible for the entire stack as part of an effort to reduce headcount, while other organizations prefer to have developers for a variety of reasons focus all their energies on writing application code. Regardless of who automates the management of infrastructure, the one thing that just about everyone agrees on is that legacy processes for managing IT infrastructure that typically involve many rote mundane tasks are no longer the way to work.
In fact, increased reliance on IT automation might accelerate the adoption of best DevOps practices because the frameworks required to automate the management of the underlying infrastructure might have been put in place already by the IT operations team, noted Wyss. Otherwise, the DevOps team itself has to put those IT automation frameworks in place with or without the help of the rest of the IT operations staff, which is a current state of affairs that many DevOps teams would just assume live without.
2019 was an exciting year for DevOps practitioners, with DevOps adoption continuing to grow at a fast pace. Most organizations were either executing or evaluating their DevOps strategies. So, what is 2020 going to bring in DevOps, infrastructure and application automation? Here are seven DevOps predictions for emerging trends in the new year.
Customer Experience
CX will become the guiding light of the DevOps journey, and not necessarily just technology. This is a big mindset shift from the early days of pure-technologist enthusiasm. As DevOps matures, we have to remind ourselves that it’s all about making business faster, not just the cool tech.
Take advantage now:
Tie DevOps achievements to business needs and results to get executive buy-in and budgets.
Successful QA teams will start looking at their role as guardians of customer experience and not just testers.
Trust
Everyone will be talking about trust—not only in the context of zero trust networks but also in the context of making it possible for Dev and Ops to trust one another as the foundation for DevOps.
Take advantage now:
Proactively invest in creating a culture of trust. Identify causes for lack of trust in your value stream and make trust a 2020 goal.
Encourage product-level conversation and define methodology and ownership when it comes to value stream management, application and infrastructure automation. It is the foundation that everyone is working on.
Invest in staging environments—Staging environments are needed to ensure things work before you put them in production. For many of us, jumping directly to automatically deploying to production is a big step. The risk can be mitigated by investing in staging environment automation, which will be the first step toward deployment to production.
Value Stream
The DevOps toolchain is going away. In 2020 we will think value stream. The industry is maturing beyond the notion of chaining together dozens of open source tools and trying to make it work together. In previous years many felt comfortable saying that if they have a CI system in place it means they “do DevOps.” This is no longer the case. DevOps is not just CI, and DevOps cannot be established in silos. Mature DevOps needs to take into consideration the entire value stream—from planning a new capability and the business requirement, all the way to the customer getting more value and a wonderful experience.
In 2019, Gartner announced it’s the last year of the ARO MQ, as it’s not the right perspective. In 2020 we finally gain a system-level view on the value stream. More players are announcing end-to-end value stream management including Atlassian’s new Forge platform, GitLab and CloudBees. Companies are maturing and adopting system-level solutions.
Take advantage now:
Plan for platform operations. System-level value stream perspective requires roles that take ownership of orchestration and automation throughout the value stream and not just in silos.
The SRE Role Will Continue to Mature
The SRE role that started with Google bringing software development practices into Operations will continue to evolve. Increasing volume, velocity and variety of data will make AIOps solutions thrive. Also, increasing levels of automation will require smart ways to handle dynamic infrastructure and applications without losing control, and being able to track changes all the way back to the coder.
Take advantage now:
If you haven’t implemented the SRE role yet, consider how SREs can be part of your teams in 2020. Define SRE areas of responsibility and KPIs, and how the SREs communicate with the teams. Is it a centralized position or are the SREs part of the team?
Make sure you provide SREs with the tools they need to succeed including:
What are the observability solutions you use? Who uses them today? How do you analyze it and how is it communicated?
What are the production environment orchestration, deployment and configuration tools that you use today? How do you connect them to application and infrastructure automation that happens before production?
Seamless Infrastructure
We will have to make infrastructure seamless for our teams. Internal customers are also customers, and their experiences will be just as important. Self-service and empowering teams will be the main themes in the bridges that need to be built between Dev and ITOps. Public cloud, Kubernetes and serverless technologies will grow. And while it makes automation more feasible than ever, the emphasis will shift to self-service and to the end users. With more companies adopting similar approaches, the need to develop internal expertise will decrease, and we will see more off-the-shelf solutions.
Take advantage now:
Map how infrastructure is used throughout your value stream, and work on unifying the approach in product-level.
Make sure you build self-service solutions with strong focus on the end-user in order to enable and balance both speed and control. Make sure developers and ITOps are equally represented when determining an approach to infrastructure automation.
Infrastructure Automation
Infrastructure automation is going to get smarter, with a lot of AI and ML. Making sure automation is properly governed will also increasingly be important. Take into consideration security and compliance, as well as cost.
Take advantage now:
Assess infrastructure automation practices throughout your value stream. Who builds the automation? What teams and systems consume it? How do you enforce security best practices (e.g. no passwords in public repositories, using proper data sets for testing, compliance of automation blueprints)? How do you enforce cost control (e.g. tagging, reviewing cloud spend reports)?
Invest in control mechanisms before you lose control, especially in public cloud automation. If you don’t have governance in place, look at baking governance into automation, rather than adopting traditional cloud management approaches. Invest in automatic de-commissioning of infrastructure, automatic tagging, analysis of infrastructure consumption data and connecting it back to the business requirement, secrets and key management and access control.
SaaS
Open source is nice. SaaS is great. While big open source projects set the pace (K8S, Github) and are strongly backed up and funded by the industry’s giants, smaller projects are increasingly difficult to scale and glue together, especially when trying to assure enterprise readiness. The need to “do it yourself” is gradually fading, as more vendors offer products and services for supporting DevOps with clear business and support models and, as important, with OOB integrations with the ecosystem. Just Rest API is no longer enough.
Take advantage now:
If your enterprise-readiness requirements make it difficult for you to scale DevOps initiatives, consider evaluating new SaaS offerings that eliminate much of the heavy-lifting and gluing that were typical to the skills-heavy early-days implementations. If you were concerned with security, cost and skill requirement, today you can find tools that are purpose-built to support DevOps for enterprise at scale. The good news is that most of the vendors are well-aware of the criticality of integrating to your ecosystem, making it easier to achieve your end-to-end value stream vision.
Want to learn more about what to expect in 2020? Join us Jan. 23 for our Predict 2020 Virtual Summit featuring discussions from some of the industry’s best and brightest offering up their visions for the future. Sign up today for this free daylong virtual event.
Recently, I had a fascinating conversation with my old friend Joe–the name has been changed but the conversation is real and unedited–an IT Infrastructure program manager from a large IT service provider.
After a bit of chit-chat on weather and football, we started talking about infrastructure in a typical large enterprise.
Joe: I am not sure how agile applies to infra space, I can understand it works well for software development, but not in IT Infra space.
Me: OK, why do you think so?
Joe: Well, we have lots of coupling between components, infra projects tend to be big migration or upgrade moves and highly risky. I know many large enterprises in Europe that we work for and many of them haven’t looked at agile in the infra space.
Forty minutes and couple of beers later…
Joe: I have to send four reports every Friday afternoon on status, risks, issues.
Me: Why do you need so many reports?
Joe: I can’t share all the same info to the client that I share internally. There are some risks that we manage internally. My client wants the info on Jira and confluence. My regional manager wants it on an internal tool (some five-word acronym that I forget!). My account manager wants it as a PPT, as this is an important account and he says this needs to be tracked closely. My global manager wants it on a different tool, inherited from a previous merger as my employer was recently acquired by another service provider.
Basically, most of my Friday is spent copy pasting information across these reports. Then I spend most of Mondays reconciling the numbers and responding to follow-on queries, and Tuesdays justifying the cost of resources, change in resource costs because many projects are usually in Amber or Red. I think it usually works well if I have a good PMO.
Me: How many people do you have on the current programme?
Joe: We have four PMs and seven infra resources on this infra upgrade and migration programme. There is a different provider who looks at the application changes and they have a team of managers and developers etc. The seven infra resources I referred to are also busy with BAU work.
Me: So, you have seven part time infra people who are the actual doers, one programme manager, four PMs and PMO, client side vendor manager(s) and you are running half a dozen projects.
Joe: Yes, as you can see, we can’t be agile in infra.
Me: Why do you need projects? And, why do you need so many of them?
Joe: What? No projects? Surely, that’s how it is always done! How do we ensure control, governance?
Moving to the cloud is becoming a must for many companies if they want to grow beyond their current footprint and be more innovative. But for many companies—especially those in the SMB space—their transition to the cloud is hindered by a lack of knowledge and experience, which often results in difficult migrations that sometimes just aren’t successful.
Since its inception, nClouds has helped organizations of all sizes move to the cloud successfully. Its experience as an AWS Premier consulting partner and its well-architected approach to cloud migration helps companies get the most from their infrastructure by engaging with the cloud, whether they’re adopting a hybrid environment or going all in on cloud.
In this DevOps Chat, JT Giri, nClouds founder and CEO, and I speak about the triggers that push companies to adopt cloud, some issues companies face in their decision-making and infrastructure’s role in an increasingly cloud-based IT world.
As usual, the video is immediately below, followed by the transcript of our conversation.
Transcript
Alan Shimel: Hey, everyone, it’s Alan Shimel, and you’re listening to another DevOps Chat, or in this case, perhaps, watching a DevOps video. Happy to be joined here by my friend J.T. Giri, who is the CEO and founder of nClouds. J.T., welcome.
J.T. Giri: Hey—good morning, Alan. Thanks for having me.
Shimel: My pleasure, nice to have you here. J.T., let’s first of all say Merry Christmas, happy new year.
Giri: Thank you, and you.
Shimel: Yeah, it’s been an exciting—we were talking off camera, off mic—it’s been an exciting quarter for you. You’re a new time dad and moving homes and at the same time, nClouds’ business seems to be just booming along with everything else associated with Amazon Web Services and the cloud. So, congratulations.
Giri: Thank you. Exciting times.
Shimel: Yeah, it is. So, J.T., I wanted to talk a little bit about—you know, there’s people listening to this or watching this who are saying, “You know, I wanna be like Google,” or, “I wanna be Facebook” or, “I wanna be one of these unicorns in that I wanna harness the cloud. You know, we have our servers in a closet here or a data center or a third-party data center, but we’re not really cloud. And everything seems to be in the cloud. We wanna move to the cloud, but you know, we’re only maybe 250, 500 employees. How do we do that, right? Do we need a helper, like an nClouds. Do we need to update everything? Do we throw out the baby with the bath water and start new?”
I want you to tell us a little bit, sort of, what’s the typical sort of customer journeys that you see when you guys engage?
Giri: Yeah, absolutely. So, Alan, you know, we’ve talked in the past. I’ve been migrating companies to AWS ever since EC2 was in beta. And, as I would migrate these companies, people would realize that they need someone to manage it, right? So, that’s how, sort of, nClouds came to be. And as you mentioned, this has been an incredible year for us. We doubled the size and revenue and the number of people.
What we were seeing in an SMB type of market, when companies migrate or when they try to migrate to AWS, they still need to follow—obviously, they need to follow best practices because cloud or AWS is constantly changing, and the skill set is just not there. So, that’s where we come in play. We can make sure that your infrastructure is well-architected. We take an approach of migrating in a way where you’re really taking advantage of the cloud—so, for example, using containers, things that are really gonna help you to harness the power of cloud. And then, for most of our customers, you know, we’re providing 24/7 support. And when we see people try to do this themselves, they get a bad taste of cloud. They’re like, “Oh, this is not for us. We’re not really innovating fast.”
So, that’s why, Alan, we’ve been very successful, because there is a huge gap in SMB type of companies. There is a skill gap and just experience. We bring that to the table, and it’s been incredible. We’re able to add a lot of value.
Shimel: Sure. So, J.T., early on there, you said something about when an organization needs someone to help them manage their cloud migration—when does an organization recognize that they need someone to help them? What’s sort of a trigger point? You know, as, again, we spoke off camera a little bit—is it when someone dives in the pool and says, “Oh, my God, I can’t swim, it’s too deep?” Or is it, before you dive in the pool, a prudent person hires a swim coach, you know what I mean?
What are sort of the trigger things that people should see as a red flag or as a trigger event that says, “Hey, man, I should be getting some help, here”?
Giri: Yeah, that’s a great question. So, basically, it depends when the companies come to that realization that we need some outside help, right? Sometimes, it is that they will try to do the project themselves, or in other cases, they realize that they really want to just focus on delivering value to their customer, right? They really wanna just innovate fast.
So—and they see how they can just stay focused on that, and that’s something we do incredibly well. We have a build—we build systems and processes together where our key focus is to make sure the customer can deliver value to their customers fast and we want to make sure the infrastructure is not getting in their way.
So, you know, it’s just, this is becoming more and more popular, like, DevOps as a service, where companies are reaching out to partners like nClouds to help them with DevOps related tasks, like, with the migration, setting up CI/CD pipeline, becoming an extension to their team, so they can focus on their core business, right? And, you know, it really depends when they—sometimes, they come to that realization pretty fast, or sometimes they try to do things themselves, and when things don’t work out, they try to use someone like nClouds.
Shimel: So, J.T., generally, one of the hardest things in life is learning when to ask for help.
Giri: Right.
Shimel: And that includes cloud migrations and everything else. At the SMB level, mid-enterprise level, you don’t have an elaborate or as big a structure as, let’s say, you do with a large enterprise where you have layers and layers of management and hierarchy—a lot of the organizations you deal with are rather flat in their management structure.
So, who is it in the organization who finally kinda raises his hand and says, “Wait a second, I think we should engage in some professional help, right? I think we should being in some experts to help us.” Is it the CEO, CIO, someone lower than that? What’s your experience?
Shimel: Yes, normally, CTO, VP of Engineering type of person. And also, you know, as you know, we became a premier partner with AWS last year. So, AWS is—Amazon, it’s no secret that they’re very customer obsessed. So, they will—Amazon, when they’re speaking to a customer and if they see the customer is struggling, they will introduce someone like nClouds to help them out with the migration or just modernize their infrastructure. And sometimes, you know—actually, many times, they can come in with funding where they will offset some of the costs.
So, for a customer, it’s just like a no-brainer to see if we can have some external experts come in and fix up the infrastructure or streamline the infrastructure. So, you know, the conversation, normally, we do get a lot of business directly from people reaching out, like, “Hey, we need DevOps help,” but we also get a lot of business directly from AWS where they identify this customer needs help, and we get involved that way.
Shimel: So, I’m just gonna bring up two questions. The first one is on track with what we’re talking about. So, now the CTO or CIO or an executive says, “Yeah, we’re gonna bring in nClouds,” whether AWS referred it or is partially subsidizing it or what have you, but, “We’re gonna bring in nClouds or some other type of help”—is there any sort of pushback or resistance normally with, you know, when you’re dealing with people? The—not first responders, but the individual contributors who feel like, “Hey, man, I don’t wanna be replaced. I don’t wanna be outsourced. I don’t want to give up my cheese to some outside company who’s helping us” or, if so, how do you overcome that? How do you work with an organization like that?
Giri: Yeah, so, normally, there’s not a pushback, Alan. So, I was actually re-reading “The Venus Project” yesterday, and I was just reading the story of Brent, right?
Shimel: Mm-hmm.
Giri: So, when you’re talking to these customers, some of these guys are on their honeymoon with their pager with them, right? And then, when you’re coming in and you’re saying, “Hey, we’re gonna help you out with 24/7 support so you can get your life back” or, you know, “We’re gonna help you to streamline this DevOps work which you never really got the opportunity or time to work on, we’re gonna help you out with that,” right? And it’s incredible.
Sometimes, maybe the initial impression might be the same as you describe, like, “Hey, are you gonna take my job?” But as they understand the value, they normally are super happy. And another thing we do, Alan, in most of our engagements, we do this assessment called architected assessment where we’re coming in and we’re just reviewing your infrastructure to see how well you’re architected, and after that assessment, we give you, like, a roadmap, like, what are some things you should focus on the first month, and what are some things you should focus on short term and long term? And the assessment goes over cost, security, reliability, performance, and operation excellence.
So, you know, we can get, in the short-term road map, we can show some quick wins where the customer sees the value and they also get confident in our ability, so that’s sort of the path that works really well for us right now.
Shimel: Got it. J.T., another thing you mentioned is that nClouds is a premier Amazon partner. And, you know, in today’s world, we throw out platinum and gold and these sort of levels all the time—but really, within the Amazon hierarchy, a premier partner has sort of, they don’t hand that out like candy. Talk to us a little bit about what’s entailed in being an Amazon premier partner.
Giri: Yes. It’s a top tier status with AWS and in order to get there, like you said, there is only 25 premier partners in North America, and you have to show case studies, you have to show the training path of your employees, and you also have to have multiple competencies, like, we have DevOps competency, we have migration competency, and we’re also the well-architected partner.
And it’s just very subjective. You know, if you work closely with Amazon, even if you hit all the check boxes, they want to make sure that quality is there for premier partners. So, they look how you work well with your customer, what does your customer experience looks like, because that’s what Amazon cares about that as well.
So, you know, it’s just very hard to get to this status, and we’re honored to be at this level right now. This is great.
Shimel: Yep. So, J.T., let me go next kind of where I want to take this, and that’s—you know, two or three years ago, migrating a company onto an AWS, or it doesn’t have to be AWS, migrating a company onto the cloud, you know, you set up some virtual instances, maybe they had some separate storage, stuff like that, EC2 kinda stuff. But, you know, I just got back from Seattle a couple weeks ago from KubeCon and CloudNativeCon—Reinvent was the week before that. And certainly, Kubernetes and containers are kind of really quickly, quickly, quickly changing the landscape, here.
You know, are you seeing that as well? And what does that mean for—typically, SMB businesses are not leading edge or bleeding edge, right in [Cross talk] option?
Giri: Yeah.
Shimel: What are you guys seeing?
Giri: Yeah. You know, it’s very interesting. Who would’ve thought that the infrastructure we built a couple years ago would be considered a legacy infrastructure, you know? People using EC2 instances and an auto scaling group in Amazon’s world will kinda be called outdated right now.
So, we’re seeing incredible push and a lot of sort of initiatives on our customer side where they wanna go containers, pretty much like 90 percent of our engagements are around containers. Because it’s a no brainer, right? It makes the CI/CD pipeline much easier. You know, most of the time, containers are automatically integrated with scaling, so you can really binpack your resources, it has impact on your cost. So, you know, this is just—it just makes the infrastructure far more streamlined and far more easier to manage.
And as I mentioned, Alan, I’ve been migrating companies to Amazon and EC2 from the last 10 years, and I’ve been managing infrastructure as long as I’ve been migrating. So, it’s just, when I’m building infrastructure right now, it’s a no-brainer to build something using Kubernetes or on AWS we use EKS or—you know, depending on what the customer wants to do, and we had a customer the other day where they were saying, “I just wanna focus on delivering product to our customers, so what is the easiest way?” We use Fargate for them. Depending on the customer requirement, we’ll go Kubernetes, but long story short, it just makes a lot of sense these days to build infrastructure on containers, because they really streamline how you manage it in the long run. It just makes life easier.
Shimel: And that’s even at the SMB level?
Giri: Oh, absolutely, yeah. I mean, because the needs are the same. You need CI/CD pipeline, you need a way for you to scale infrastructure, you need a way for you to run batch or cron job type of jobs. And when you go with something like containers, you get there—like, you know, you don’t have to build home grown tools to facilitate these things. You know, containers or EKS or Kubernetes, they allow versioning when you’re deployed or auto scaling is sort of built in, you know, you can schedule batch jobs.
Just, it’s a no-brainer, you know? If you’re building out infrastructure, you should definitely explore containers each time, and that’s what we try to do every time. And it’s just incredible, the amount of success we’ve had before and after.
Shimel: Sure. So, as we sit here at the dawn of a new year, if I would’ve asked you in early 2018, do you think Kubernetes and containers would be one of the major stories of the year, I don’t know if you would’ve said yes or no. But as we sit here in the dawn of 2019 and we project ahead—let’s say I interview you and everything’s all well and at the end of next year, the end of this coming year, 2019, we sit down and talk about, “Well, what’s 2019 really show?”—what do you think are the big stories this year, J.T.? What do you think, coming down the pike, especially—I’m not talking about the world view. For nClouds and for your customer base, what do you think is really big for you in the coming year?
Giri: Yeah, so, I think in general, the application modernization will continue. We continue to see that trend, and this is also going to be big in SMB. So, we will continue to double down on AWS. It’s been an amazing partner. Our core values, our partnership is based on shared goals, so we’ll continue to kinda double down on that relationship.
And we’ll continue to utilize this well-architected framework. It helps us to assess the customers when we’re onboarding and it helps us to manage the customer, like, every time the customers were managing, we continued to run a well-architected assessment to make sure they’re well-architected, and they continue to stay well-architected.
And, yeah, I mean, just continue to focus on SMB, AWS, continue to align with well-architected framework. We also just spun off another company, it’s called nOps, and basically, it just helps to make sure the customers are well aligned with well architected framework, and it’s available on AWS Marketplace. So, we’ll continue to invest in that, because that solution has been incredible, nOps. A lot of our customers are seeing value in that, so, we are actually working with Amazon to give that solution to other partners like nClouds who can use it to manage their infrastructure to make sure the infrastructure is well architected, and we are also offering it to end customers who can use that tool to assess their infrastructure and continuously stay in compliance with well-architected framework.
Shimel: Excellent. You know, nOps, that’s something we’ll have to discuss on our next chat, J.T.
Giri: Yeah, absolutely.
Shimel: Well, you know what? We’ve been on almost a half hour here, so, unfortunately we’re about out of time for this one. J.T. Giri, CEO and founder of nClouds, thanks for being our guest on DevOps Chat.
Giri: Thank you so much for having me.
Shimel: My pleasure. Best of luck to nClouds this year, best of luck to you personally with your family and everything else, you know, it’s gonna be a busy year for you, my friend.
Giri: Absolutely. Thank you. Thank you.
Shimel: Alright. Hey, J.T. Giri, nClouds—you’re here with Alan Shimel on DevOps Chat. You’ve just listened to another chat. Bye bye, everyone.
Migrating to the cloud isn’t as simple as signing up for a SaaS application or deploying applications on a cloud infrastructure or platform; there are other factors at play. And the move to cloud shows no signs of slowing, with Gartner predicting the worldwide public cloud services market to grow 17.3 percent in 2019. Getting cloud deployments right is more critical than ever.
As organizations shift more to the cloud, it means they will increasingly rely on networks and infrastructure they don’t own and directly manage. Yet, this infrastructure is just as critical to consume and deliver the applications and services as when it was in the data center. Being able to explore these networks to identify choke points and routing issues in advance helps to inform better network investment and configuration decisions. It is essential for a successful cloud deployment as well as ensuring an optimized environment on an ongoing basis.
There are six key network considerations IT managers should take into account before shifting to the cloud.
Baseline Performance Before Deployment
Getting a baseline measure of network performance when moving to the cloud, first and foremost, requires adopting a different set of data points than organizations have typically used in the past. The move to IaaS, SaaS—or any cloud service for that matter—means organizations are beholden to those providers as well as third-party service provider networks through which the application or service traffic traverses.
Prior to cloud adoption, the network was essentially under an organization’s control. And it’s not that networks are necessarily less complex, it’s just that network teams had access to all the network data like packets, flows and device level information to monitor performance and security. And when issues arose, teams had the ability to troubleshoot and triage problems. Traditional monitoring approaches worked in a world where the network was managed and contained and boundaries were known. However, these data sources are no longer available for monitoring the performance of the cloud network, where devices can’t be directly instrumented or polled. Visibility into cloud and third-party networks can only come from data sources such as synthetic and end user monitoring. These techniques can provide quality of experience metrics that network teams can rely upon to test configurations and baseline performance before moving to the cloud.
See and Understand Bottlenecks in your Infrastructure
It’s very possible that, based on existing configurations and routing policies, certain offices and sites are not optimized to consume applications or services over the internet. For example, a branch office in India may have issues accessing Salesforce because of transcontinental latency and a bandwidth constrained MPLS circuit. Similarly, a branch in Austin may have better and more reliable access to applications being served from El Paso rather than San Antonio.
But how do you know what the ideal configurations are before deploying? Time to have a look at your new data sources. The first in your arsenal is actually derived from a technique you may already be familiar with for monitoring application experience in terms of page load and user transaction timings. Synthetic monitoring can be used to gain insight into infrastructure and network performance as well as your application. What’s great about synthetic monitoring is that it works in cloud environments and across the internet in addition to your data center for in-depth information about how each portion of the network is performing.
Map Out Real Traffic Paths
Having visibility into specific areas of a distributed network that are problematic, as well as pertinent details that can help determine a root cause, results in more tangible and actionable information for the network you own and those of your providers. To achieve a more complete picture of how traffic is traversing the cloud network, it is essential to combine synthetic tests with ping and traceroute functionality. Now network teams can not only test the reachability of an endpoint or host through the cloud, but also use that data to determine the route or path of packet through a distributed network while measuring transit delays.
Another useful data point is Border Gateway Protocol (BGP) routing table information. BGP knits the internet together by exchanging routing information between networks. And these routes determine how traffic flows to or from your apps and services. Layering this information into your diagnostic approach helps understand how routing configurations and changes impact reachability, latency, loss, jitter or other performance metrics.
Collaborate with SaaS and Cloud Providers
So, when it comes time to work with your your SaaS, ISP and cloud providers, it helps to share this information, related visualizations and diagnostic information in an easily consumable way. That way, when a trouble ticket is generated, it is dealt with much more quickly by the vendor or provider. Knowing exactly where a problem exists and the related cause analysis empowers teams to take immediate action and enable the responsible party take immediate action as well.
Get Things Right Before Deployment
With detailed visualizations and hop-by-hop metrics, it becomes possible to try out a routing change, plan for a new data center or test a roll out of a new application or SaaS service. Synthetic testing layered with contextual data and a detailed visual network topology allows teams to test performance and gain insight into initial infrastructure configurations, plan changes and understand their impact on cloud applications to get things right before deployment.
Continually Monitor the Performance of Your Network and Its Impact on Applications
The same techniques and data sources teams are used to baseline performance and achieve optimal cloud configurations can also be used on an ongoing basis to ensure continued performance. Active monitoring with additional layered information allows organizations to constantly keep an eye on performance degradation to optimize end user experiences across their network. And the additional inclusion of visual analysis reduces mean time to troubleshoot and repair issues. As the internet and cloud computing reshape the enterprise network, incorporating these new data sources, correlation of pertinent data points with a unified visualization is critical to the successful deployment and management of cloud applications and services.
This virtuous cycle of better performance becomes even more important in the future as organizations increasingly rely on the internet for their critical operations. Growing levels of software-based automation will only increase the number of different network paths traversed and the resulting end user experience. Accurate insights into the entire cloud network enable organizations to troubleshoot and solve their cloud performance problems, alleviating potential cloud migraines and ensuring the business runs smoothly … and painlessly.
IT has long had a problem with hiring. I’m not certain where it started; most of us blame HR, and that’s probably incorrect, but gives us someone to blame. It’s that we post jobs with insane requirements. Sometimes this is done to keep the applicant pool down, but most of the time it is a simple case of throwing in the kitchen sink. We have laughed at advertisements that wanted applicants to have more years in a given technology than the technology has existed. The laughter was because someone quipped, “So what we really want is only liars to apply.”
But it doesn’t stop there. How many advertisements have you seen for a person to join an already good-sized team of developers, and the requirements include experience in every single language/tool the person might encounter—and sometimes ones the organization might use in a year or so? And DevOps, with its increase in toolsets and broadening complexity, make this problem worse.
If you want good DevOps team members, you have to focus your search. We’d all love to have unicorns who could do it all on day one, but those people don’t really exist. Even with experience in every single tool in your chain, they still have to learn apps and environment. Try to craft job requirements that focus on the couple of core tools, and then filter for ability to learn and adapt. Because the only certainty about IT at this point is that technology will be shifting, so focusing on the tools you are using today is pretty limited.
Role plays a part in this also. While we like to pretend DevOps is one thing, it is a rare application or portfolio for which there are not specializations on the team. If you are looking for someone who will lean more toward the DEVops side, do you really need them to be operational wizards with three years each in storage, networking and container management? Conversely, if you are hiring someone more for the devOPS side, do you need them to have expansive experience in Javascript, or do they just need to know who to call if it gets beyond their ability?
Maybe your company is one of those shops where everyone on team X is expected to be an expert at everything. Kudos for you, but there aren’t a ton of those people out there. There is a different mindset to DEVops versus devOPS, and even if unofficial, most employees lean one way or have a more proficient side. If a candidate fits the bill in an area you are weakest, consider training them on the job in the area you are strongest, rather than waiting for who knows how long for the perfect person to fill the open req.
And we still have a need for hardware specialists. I’ve said it before, but do you want your EMC (or Cisco or whomever) person to be a generalist, or do you want that person to make the infrastructure that underlies a huge chunk of your applications hum? Hire specialists for specialized skills, hire broad knowledge for app-spanning needs.
It’s been a problem for years, and only you can fix it. Call out overambitious job postings and make managers ponder what they’re actually looking for. You know you have a need, you know what the position will do. Help hone down postings so that they fit the actual requirements, not some blue-sky version of potential requirements.
And get the best people for the job. After all, you are going to have to work with them.
It seems every organization is chasing digital transformation. Some companies are succeeding, while some are wallowing a bit, trying to figure out how to do it. Box is a company that is both successfully navigating digital transformation and being a catalyst for digital transformation at other organizations.
In this DevOps Chat we speak with Paul Chapman, CIO of Box, about digital transformation for both Box and other companies, and he gives us his own take on what the environment is like for companies and what they should do to thrive today. Great conversation, enjoy!
As usual, the streaming audio is immediately below, followed by the transcript of our conversation.
Transcript
Alan Shimel: Hey, everyone. It’s Alan Shimel and you’re listening to another DevOps Chat. I’m really happy to have as a guest on today’s DevOps Chat, Mr. Paul Chapman, CIO at Box. Paul, welcome to DevOps Chat.
Paul Chapman: Thanks. Great to be here, Alan.
Shimel: Great to have you here. So Paul, before we jump into a couple of the items we wanted to talk about. I guess in case anyone in our audience maybe is not familiar with Box or yourself, why don’t you just quickly level set your background, Box, and we’ll jump in from there.
Chapman: Sure. Well, maybe I’ll start with just a little background, accelerated bio of sorts. Been in the technology space for quite some time now. I like to joke that although this is the truth, I was a COBALT programmer and an RPG programmer back in the late ’80s. So I’ll fast forward to today and say a lot has gone on in between. I’ve had to reinvent myself many times over to stay relevant. I often tell my team that the pain of change is mandatory. It’s the suffering that’s optional.
So get comfortable with being uncomfortable if you’re in the technology space. It’s never boring. That’s for sure. I’ve done a lot of things in between but thoroughly enjoying my time over here at Box. We’re born in the cloud, grown up digital company. Very forward thinking, innovative. And we’re helping tens of thousands of organizations around the world to help to transform and digitize the workplace.
Shimel: Fantastic. So Paul, you mentioned Box and we talk about unicorns and cloud natives and these kinds of things. Box is certainly a member of the club. And while they themselves, generally, these kinds of companies like box eat their own dog food in terms of what they preach and everything, we almost take for granted the transformative wave that they’ve ushered in. Again, I’m your age or maybe more. I remember the 1984 commercial with the hammer –
[Crosstalk]
Chapman: Mm-hmm. The Apple one, the famous Apple one.
Shimel: The Apple commercial. Sure. The Mac commercial. So in a lot of ways, companies like Box and as their poster child for it have broken through the silos that in the IT world, you and I grew up and kinda held us bound. If you can, give our audience a little bit of – I don’t know if that was intentional that it breaks down silos or it was an unintended consequence but talk to us about how is it helping to break down. How does it usher in what – digital transformation is something everyone talks about. How does it help with that?
Chapman: Yeah. Well, I think when you look at – and I’m gonna sort of just geek out a little bit and share that one of the things that I have – now, we’re starting to see some maturity in what I would call sort of the modern reference architecture for how you run your IT service today or business services today, is that when you look back at sort of the technology that we invested in in the ’90s and in the ’80s, which at the time, was the most modern advanced technology I should say, at the time. Unfortunately, it was architected in such a way that it was very siloed and you ended up sort of building up these sort of monolithic sort of application stacks where you went to sort of the oracles for everything or the SAPs for everything and it was very hard to integrate.
And unfortunately, what happened is is you sacrificed a lot of what I would call best of breed capability and innovation in that model and because that architecture doesn’t, in a sort of step wise, linear way evolve to support today’s way of working – today, we have a new style of employee. One that has grown up digital that’s been educated by consumer experiences that expects a certain set of things when they enter the workplace. You have a new style of workplace. It’s very open and social and collaborative.
You have this new style of IT that’s evolved to support that. And when you look at what’s actually happening, the modern architecture is built around this best of breed ecosystem and there’s innovation going on everywhere. But the innovation is generally organizations are picking the area that they’re gonna be the most innovative in and they maniacally focus on the innovation in that space. So they’re the best at what they do and Box is absolutely the best at managing unstructured content for the enterprise with the best compliance and regulatory and so on. And security and risk and all those things.
And then there are other companies that are the best at messaging and then there are others that are the best at certain types of collaboration. And what we get to do now is to start to bring all these services together because they’re built to interoperate which the architectures of yesteryear weren’t and we get to bring and curate that into the experiences we bring to our employers, partners, suppliers, and customers and therefore, we’re starting to create more automation, more digitization across this best of breed ecosystem. Long winded answer but there’s a lot going on in there today.
Shimel: Oh, absolutely. So let’s pick into a couple of those areas. Number one is lock-in or the – we used to suffer with these large stacks. You were locked in though. Right? I was talking to someone earlier today, a start up who managed Macs for enterprises. Right? And why haven’t we seen that before? Well, maybe there were enough Macs. But we’ve all suffered from lock in in our technologies.
Chapman: Yeah. And we don’t like it. We don’t like the way it makes us feel. We don’t like to feel captive or without choice and really today, freedom of choice is extremely important. And so what I do know, though, and this is something that I’ve certainly come to appreciate is especially if you are, as you think about a much more heavily cloud based footprint, you have the freedom from infrastructure. You have the freedom from direct sort of operational responsibilities that in our past lives, we have significant operational responsibilities managing a lot of technology debt and infrastructure and so on and we had to manage service levels, capacity, performance, backups, failover, all the things that are just really keeping the lights on.
We’re not adding any material value necessarily directly to the businesses. It’s the things, the services that run on top. And so what I’ve seen is is that now as you start to move to a cloud based model which is much more subscription based. The on ramps are a little smaller. The off ramps are a little smaller. The ability to change out services for services that may not be as innovating as fast as others as is a little easier but I think the key thing is this. Every one of those services that you subscribe to, it’s a mutually beneficial outcome. They have to earn your business on an ongoing basis.
Shimel: Yep. On a minute by minute. Not even daily.
Chapman: Because if not, the subscription stops and you move it somewhere else and if they are not maniacally focused on innovating in their space, having highly reliable, scalable services, having services that meet your compliance and regulatory requirements and having services that meet your trust and risk criteria for that particular service, they don’t have a business. And so the beauty of this model or this new emerging model, I should say, is that in the old ways, they’d sell you software and then you sort of … you get locked in and it’s hard to move. Now, we think about it as all the time. It’s about the customer all the time. How are we making our customers get more value from their investment in us and how can we make sure we’re providing reliable services and innovative capabilities that they’re subscribing to to maintain them as a customer. And that’s good for us.
Shimel: Absolutely. So I would even go further, Paul. I’d say the hardware vendors locked you in more than the software guys even did. Right? When you made decisions, whether you were on Solaris or this flavor of Unix or Microsoft or what have you or HP, I mean, HP, and I know you used to be at HP, no disrespect to them, but they were really good at talking about freedom and best of breed while locking you in tighter than a drum. But today, so to me, this all refers to something that I call portability. We have, as consumers, as customers, we have such portability today where we can take our stuff up and go. It’ll run anywhere we want, just about.
Chapman: That’s a great point. Actually, a really good point. And the infrastructure, I was focusing on the services piece but you’re absolutely right. As I mentioned, the freedom from infrastructure in the cloud world is like no offense to the infrastructure providers. It’s just I add more value to my organization when I’m not focused on those things. It’s a give back of time and one of the most precious things we can give anybody is time. And I’m able to use that time to do what I would consider much higher value, contribute at a much higher value to the organization because of that.
Shimel: Absolutely. Paul, I wanna transition to another topic that we spoke of but I think you gave me a great segue here and that is that when we talk about things like transformation and thriving and leveraging what we have, what the technology world has to offer us today, so much of it, frankly, isn’t hardware or even software. The success or failure of organizations is often based on their culture and their people. Right? And some, we live in a gee whiz world where we take things, we were talking off hand about answering the doorbell from China, but organizations can’t really thrive, compete, and transform without that cultural sort of happening. Curious on your thoughts on that.
Chapman: Yeah. Probably … at Box, we talk a lot about the future of work and also, we talk a lot about it’s not about rewriting your software. It’s about rewriting your company. And when we look at companies that … if you look at just some of the statistics, 52 percent of the Fortune 500 have disappeared since the year 2000. And if you do the math, that’s more than one per month. And a lot of people get into this techno-panic thinking it’s technology that’s disrupting these companies but the reality of it is is technology is enabling that disruption but the technology that’s doing this is available to everybody including the incumbents that have built their business models and their operating models on sort of a sort of legacy way of working. And you have – if you look at the companies that are disrupting, it’s because they have a different operating model. They have different business models.
They operate culturally very different and the cultural shifts are really, really hard, especially in organizations that have become very established in their markets because that’s what got them there. And the reality of it is we used to coin this term sort of this frozen middle part of an organization that didn’t necessarily have – doesn’t necessarily have aligned incentives to change. Because it’s like what’s in it for me on the other side of this change? And they say at any given point, a third of an organization, and this changes, varies from company to company but a third or an organization knows where you’re going and how you’re gonna get there. A third doesn’t but they’re willing to learn and invest and figure it out and a third doesn’t, never will, isn’t interesting, da, da, da, da. And that’s resistance.
And the resistance to change has to be resolved. A lot of companies will figure out maybe what they need to do and even why they need to do it but it’s the execution of the how and sometimes, that how is where it really holds people back because they realize that they may not be able to get there with the culture and the people they have. And they switch out leaders and so on and I can share some more in what I’m seeing companies that are able to do this successfully. Most of them, it does take sort of what I would call sort of breaking the company somewhat in order to ______ them.
Shimel: No, no, no. You gotta burn it down. Right? And I don’t wanna burn it down. But there has to be fundamental change. Last question because I know we’re running low on time but how important is leadership? Right? As a CIO, this is kinda your … this is a big part of your role, right? But how important is leadership in making that transformation?
Chapman: It’s absolutely key. You can’t do it without the right leadership mindsets and call it coalesced support. So I’ll give you a recent example that I was talking to a CIO of a very, very large company that had been struggling with, call it, reinventing itself and reestablishing itself in a new modern digital world and was starting to make some, is starting to make some success there. And one of the questions that we often get asked as CIOs is, “What’s your partnership model look like or your engagement model look like with the business and how do you think about change management?” And like I said, I went to school to learn how to do programming and COBALT and RPG. I never went and got a PhD in change management.
[Crosstalk]
Yeah. I’m not an expert at trying to change the mindsets of thousands of people across an organization when at least a third of them, based on our numbers, don’t even want to. Right? I’m jesting a little bit. The CIO is one of the best partners to enable change but isn’t necessarily always in the best position to drive that change across the organization. You have to have leadership alignment and so on and the company I was talking to, I was talking to the CIO and he was asked that question. “What’s your – people are interested because they’re starting to see meaningful change.”
And they asked him about his relationship model with the business and he said, “Here’s what happened. When we got our new CEO, he went to all of the heads of all the different business functions that we have and said, ‘If we are going to transform, if we are going to think like a digital company, if we’re gonna think like a technology company and we’re gonna disrupt, it’s incumbent upon every single one of you business leaders not for IT to come to you and say, ‘Here’s what our model for working with you is.’ It’s what is your model for working with IT? What is your model for working with the CIO?’”
And he said that once that accountability came into the business owners, he said it just greased the runway for the CIO and organization to enable accelerated change across the organization because it wasn’t incumbent upon the CIO then to drive that change. It was to enable the change. And I actually think that that’s something that is key here is we have to rethink the model of how we engage and operate, not just it’s a technology solve.
Shimel: Right. I agree with you. First of all, I wanna apologize for taking up much more time than I think we booked but it was a great conversation. It was kind of fascinating. Maybe we can have you on on another chat at some point or through the other vehicles we use. But look, continued success with Box. We’re customers.
[Crosstalk]
devopsy.kinsta.cloud only for our most important data. But anyway, thanks for being our guest on DevOps Chat today and we’ll speak to you soon.
Chapman: Appreciate it. It’s fun hanging out.
Shimel: All righty. This is Alan Shimel for devopsy.kinsta.cloud. Thanks for listening. This is another DevOps Chat. See you soon everyone. Take care.
Many years ago when I was interning as a software engineer, I made a mistake that could have ended my career just as it was getting started …
It was my birthday, and I was given the task of loading queries to a database. The task was straightforward—even then, when I was only an intern learning my craft. I was using a program to upload the queries automatically. It looked like the program was running well, so I decided to go home and leave the program running overnight.
The next day at the office I met any intern’s worst nightmare: Everything had stopped. And it was all my fault. The queries were loading with a paralyzing level of inefficiency. In fact, it took up all of the processing power that we had in that company server, making it crash. All operations were brought to screeching halt … just for my measly query project.
It took several hours for the operations team to have everything running again. Luckily, I didn’t lose my job. In fact, the company ended up handling the situation graciously, and it turned out to be a good learning opportunity for me.
These days, something like this is entirely avoidable, and it’s all thanks to the magic of autoscaling. As recently as 10 years ago, obtaining more server space was a slow and rigid process owned by a few gatekeepers. Now anyone can acquire more server space with just a few clicks.
The democratization of processing servers has completely transformed the internet. It has given any organization virtually unlimited processing power at a moment’s notice. When done strategically, autoscaling allows for a more flexible and agile processing infrastructure. If implemented in a way that’s ad hoc or rushed, your organization will end up spending exorbitant sums of money without seeing the full potential of autoscaling.
Here are three questions you need to answer to develop a winning autoscaling deployment strategy.
What Should You Autoscale?
Before the advent of autoscaling, projecting server needs took a lot of guesswork. Imagine, for example, that a timely and effective advertisement suddenly brings a 1000x spike in traffic to a company that sells sunglasses. If the company is not ready for this spike in traffic, everything comes to a screeching halt. Orders are not filled—and some orders can’t even be made because the server space is not ready to handle the unexpected surge in volume.
With autoscaling, processing power can be offloaded to a new server automatically, based on the conditions set by the system’s administrators. This is valuable not only in the case of spikes in traffic, but also for lag periods. This is a way to avoid paying for processing power that you’re not using.
In any organization’s computing architecture, there are a number of components that can be autoscaled, whether it’s central processing units (CPU), random-access memory (RAM) or bandwidth. The first step is to collect as much data as possible to understand the current operational loads of each of these items. Any organization using Amazon Web Services (AWS) has access to some data monitoring services, but they tend to be very basic. It’s better than nothing, but there are other options out there.
One option is black box testing, which allows you to measure various components within your infrastructure through monitoring. Among many things, it can help you determine where bottlenecks are occurring. Elastisearch is another great resource for word searching and metadata collection within a document or a digital framework.
Another effective tool to assess autoscaling needs is something called an ‘agent.’ It works by sending pulses to measure processing data throughout an entire system. Collecting data like this will give you crucial insight on the state of your processing infrastructure.
Once you collect all of this data, a thorough analysis is in order. At present, analyzing this data must be done manually, but before long machine learning will take care of this part of the process. It’s important that you consider all layers of your applications (front end, back end, database) to autoscale effectively. It can be tempting to scale at will, given how easily it can be done. Be vigilant against abusing this capability. Otherwise, it won’t be long before you spend outrageous sums of money on something you don’t need.
How Should You Autoscale?
An important concept to understand before you decide on your autoscaling approach is containers. Containers are small pieces of code that are meant to perform a specific function. In this way, a single machine can contain several apps used to perform a variety of functions: CPU, RAM, bandwidth, disk space, etc.
Containers are a sophisticated way to get creative with the server space you’re working with, and should be considered even before you start building on a vertical or horizontal autoscaling strategy. Better algorithms in existing hardware can help you to avoid making new hardware purchases. In any case, once you do decide to autoscale, containers should most definitely be considered as you strategize and build.
However, while containers can be great for scaling, it should be taken into account that they have the potential to add a great deal of complexity to your server. For example, if your server can run 20 containers, but suddenly you need three more, buying a new server with more containers is an expensive solution, since the server would be mostly empty.
In principle, there are two ways to autoscale: vertically or horizontally. Scaling vertically means adding more physical machines with more gigabytes for more processing power. The biggest AWS machine server that’s currently available (at least as far as I’m aware) is 128 CPUs. That’s the most processing growth you can make from just adding one server. The cost of a new machine like this will vary: anywhere from 80 cents to $7 an hour is what you can expect to pay.
The other approach is to scale horizontally, which would involve a cloud-based solution. For organizations not ready to make the commitment that comes with adding a new physical server, this is the approach worth considering. Plus, with a horizontal approach to autoscaling, it’s possible to adjust processing needs on the fly.
It’s important to understand that not all parts of an application can scale the same way. Databases, for example, can scale up with ease. But scaling in (horizontally) would take a lot of time. If your data is stored in 30 front-end containers and you want to scale in to 15 containers, you can’t do it in one step. You’d first need to scale from 30 containers to 28, wait for the data to replicate, then scale in from 28 containers to 26, again wait for the data to replicate, then from 26 containers to 24, etc. Otherwise you risk losing all of that data.
When Should You Autoscale?
Consider the above example of the sunglasses vendor. While it’s impossible to predict exactly when your business or customer base will reach a tipping point, something such as publishing out an advertisement with a broad reach is certainly an instance where your organization ought to be ready for a sudden spike in traffic. Your business decisions should be made mindfully, so that the potential effects of actions like these—and the strain they create on your organizational architecture—are understood ahead of time.
Anticipating peaks and troughs in demand is something businesses have been doing for centuries. In Colombia, for example, utility companies have long understood the power of “arepa hour”: This is when Colombians are at home eating breakfast (specifically arepas, a traditional breakfast food in Colombia). Knowing that the majority of the country is likely to be in their homes sometime between 7:30 and 9:30 a.m. consuming power, that is a crucial data point that can allow for utility companies to be ready for this disproportionately high level of demand.
The dilemma to determine “the arepa hour” of your business’s autoscaling needs often comes down to CPU and memory. Should you be ready to autoscale based on when your site crosses a predetermined CPU load? Or should it be based on the memory running on your servers?
The answer is really neither. There is no “golden metric” that will point the way for your autoscaling strategy, and you should never decide to scale based on a single data point in isolation. For example, it may make sense for processing data to be offloaded to a new server once the processing capacity is at 90 percent, but it’s also entirely plausible that your application continues to run completely normally with zero issues at this processing load.
You should be aiming to strike the right balance between efficiency and not compromising the user experience for those who visit your site or use your application. And the only way to do this is to consider all of your data points holistically and how they interact with one another—not in isolation.
Conclusion
We are past the time in which a query loading faux pas like the one I made as an intern can completely shut down a company’s processing activities. Given how accessible processing servers have become, some may find it tempting to “brute force” their way through every problem: Make a few clicks and pay for more server space at will. Don’t be goaded into taking the easy way out. Think about autoscaling strategically by answering these deceptively basic questions, and you will in fact make it easier (and more cost efficient) for your organization in the long run.
