devopsy.kinsta.cloud

Tag: public cloud

  • Understanding and Implementing the APM Puzzle – Performance and Availability

    Understanding and Implementing the APM Puzzle – Performance and Availability

    While every organization is unique, there’s one universal goal everyone can get behind—creating and delivering an exceptional end-user experience. For both classic IT organizations and companies built in the age of cloud, delivering continuous availability and appropriate application performance is key.

    Although digital transformation has become a buzzword some would say is meaningless, the idea of digitizing a traditional company and building new businesses whose foundations are established upon the latest web technologies is critical in today’s IT and non-IT organizations. Think about the prevalence of cloud migration efforts during the past several years. Traditional organizations typically prioritize on-premises applications with increasing quantities of hybrid applications. But, because they aren’t yet fully transitioned to cloud—whether IaaS, PaaS or SaaS—their IT organizations need to live in both universes.

    Monitoring these environments at both the infrastructure and application tiers and across each stage in this transition requires support of a full stack of tools to cover web uptime, user experience, transaction monitoring, application tracing, log analysis and appropriate infrastructure monitoring. That said, modern organizations that have fully embraced cloud and leverage modern microservice architectures have similar challenges, but the exact same level of monitoring visibility requirement.

    Whether public, private, hybrid or multi-cloud, nearly every organization has moved to the cloud in some capacity. Regardless of what the company produces or the services IT provides, it must maintain uptime, availability and seamless performance to create and continuously deliver a user experience rivaling competitors and exceeding user expectations.

    Because let’s face it, even though every business is different and virtually every market is saturated, consumers and B2B companies have more options than ever. Without delivering a solid end-user experience, there’s an ever-increasing plethora of alternative vendors and providers from which customers can choose.

    Public Cloud Platforms for Classic IT Organizations and the Cloud Generation

    Let’s explore a few differences between how traditional IT organizations and cloud-era companies interact with the cloud. At a high level, Azure is a natural choice for IT companies with significant investments in apps traditionally acquired (not built) by the company and managed by their IT team, that are now tasked with digitizing their business by leveraging public cloud platforms. These types of companies often have Microsoft Office, Microsoft Exchange and other Microsoft products already implemented on-premises, and Infrastructure-as-a-Service (IaaS) or Platform-as-a-Service (PaaS) solutions as well. If they’re now adding a public cloud platform to their mix, staff expertise and solution familiarity make adopting Microsoft Azure a natural next step. Microsoft has long recognized the need to make these types of transitions and mixtures easier, demonstrated by its Office365 suite (as an example).

    On the flip side, companies that were born fully in the cloud, (i.e., relatively newer organizations without a legacy anything on-premises, with respect to IT infrastructure) will likely have a higher propensity to choose a provider such as AWS or Google Cloud even though Microsoft, Oracle and others are certainly competing there as well. So, it’s not a matter of one being intrinsically better or more fully-featured, but about which is the best fit for the organization and its priorities.

    At the end of the day, however—regardless of organization type or public cloud platform implemented—every company needs monitoring tools and solutions that keep applications up, running and performing seamlessly. With an integrated set of simple, easy-to-use, quick to value and powerful solutions, organizations can achieve truly proactive lifecycle assurance of application availability and performance.

    Uptime and Seamless Performance: Putting Together the APM Puzzle Using All the Pieces

    Each and every piece of the APM puzzle must be incorporated to achieve full-stack visibility and ultimately top performance: user experience, metrics, traces and logs. Having a log management solution monitoring availability, enables proactivity and enables IT to react quickly to help identify and remediate problems is paramount—it’s table stakes that everything be available. After all, if a website or page isn’t accessible, business transactions cannot be processed and that business is effectively down.

    Creating and maintaining a healthy application involves not only ensuring it’s functionally available to users; it must also be performing appropriately—continuously delivering both a reasonably responsive user experience and supporting the required volume of transactional work. This means a complete view into all aspects of performance—from the website and webpages all the way into and across all relevant application components and their associated infrastructure—be proactively visible and monitored.

    There is thus a natural and increasingly urgent workflow affinity between logs and APM—across the development lifecycle. When errors occur, it’s critical to understand, precisely and immediately, where are the impacts on service performance, or if there even are any impacts. Conversely, when application performance issues arise, it’s critical to immediately know exactly which errors—across the complex and constantly changing infrastructure and application stack—are directly related to the problem at hand.

    Without this bi-directional workflow integration, not only can application performance and availability not be managed proactively, but any issues with application service availability and performance will take far longer to diagnose and repair—seriously impacting the business bottom line.

    Just because everything is available doesn’t necessarily mean it’s appropriately performing for the end user trying to use the application.

    With all this in mind, it’s paramount to implement powerful, easy-to-use solutions that’ll achieve integrated application performance and availability management for any type of organization operating on-premises and/or cloud-based applications across all cloud environments.

    Conclusion

    While traditional IT organizations and cloud-only companies are different in many ways, providing exceptional end-user experience and achieving seamless availability and performance (no matter what) is a common and increasingly critical business goal. To achieve business success, organizations should choose a complete suite of simple, powerful and integrated solutions to optimize and manage their cloud-hosted applications and platforms. Anything less leaves the APM puzzle unsolved.

    — David Wagner

  • DevOps Chat: Data Protection in the Cloud With Druva

    DevOps Chat: Data Protection in the Cloud With Druva

    As any CIO knows, a data management architecture is essential to caring for one of the organization’s most valuable assets: data. Data management sounds simple but it’s not. Backup and recovery, disaster recovery, data retention, governance and e-discovery are all parts of any effective data protection strategy.

    As businesses move and grow their presence in the cloud, the same capabilities apply in this very dynamic and elastic environment.

    Recently, Druva received another investment round of $130 million led by Viking Global. Mike Palmer, CPO at Druva, joins us on this episode of DevOps Chat where we talk about Druva’s planned use of those funds and Druva’s approach to providing a SaaS-based data protection platform.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today, I’m joined by Mike Palmer, chief product officer at Druva. Our topic today is how data protection is in your future as you move to the cloud. Mike, welcome to DevOps Chat.

    Mike Palmer: Thank you, Mitch. I have been looking forward to this conversation with you.

    Ashley: Me, too. Glad we’re together online and talking about data protection. Would you first start, though, just let’s introduce yourself, tell us a little bit about what you do as chief product officer and tell us a little bit about Druva.

    Palmer: Well, as you say, running product is my primary job as chief product officer, as you would imagine. I run engineering and pricing and strategy for the company. And Druva is an amazing organization, one that I came to around about a year ago, coming from various positions in the industry, watching the evolution of both storage as well as data protection software and how it’s been evolving in a world of public cloud adoption.

    Druva is the first and only SaaS player in the data protection space, which made it super intriguing for me because, as we look at enterprises and how they adopt SaaS, whether that’s been from CRM to ERP to collaboration platforms, what stood out as the next big wave of SaaS adoption was, in fact, data protection. So, we’re focused on that space and growing rapidly.

    Ashley: You guys are doing real well. As a matter of fact, I know a little bit of old news here, about 30 days, but you had a $130 million investment. I think Viking Global led that for you—congratulations, by the way. That’s awesome.

    Palmer: Thank you very much. I don’t think $130 million ever feels old.

    Ashley: [Laughter]

    Palmer: But it was a great validation of the strategy and Viking is a tremendous investor. Folks in the industry know kinda their origins dating back a couple of decades and we were glad to have them aboard. They were a great partner in talking through the evolution both of our business as well as the industry over the next, what we consider to be, kind of 5 to 10 years and they saw what we saw, you know, which was kind of a relentless pace of adoption in cloud and cloud storage, the desire for enterprises to find ways to simplify their current environments and move technical skills from some of the legacy areas like data protection into some of the more advanced ones or next generation ones like machine learning and analytics, and we stood at the intersection of these trends, and they got on board with us and have been a great partner.

    Ashley: Clearly see a good future for you. Let’s talk about—so, this 130 million and the other investments that you’ve had, they didn’t give that to you to put it in the bank, they want you to put it to work. So, what does the future of Druva look like as you start to invest those kinds of funds?

    Palmer: Yeah, it’s a great discussion, because we are competing, frankly, on a few different _______ looking forward. You know, number one and the basic one is, how do we drive the best possible TCO for enterprises who have increasing amounts of data to manage, increasing amounts of risk and liability related to the regulatory environment surrounding that data. How do we expand the applications and the workload coverage for them as they support current apps, but also the ones that they’re adopting in the native cloud?

    And then, as we look a little bit forward, as I was told by an investor, you know, this is either a good investment or a great investment and it’s gonna hinge around how we can expand the data protection definition into one in which customers get more value from that copied and stored data. If we can find ways to help them use that data and extend it into the ecosystem, get better visibility into it, it will effectively not only change the architecture, but we will have changed the entire value proposition, and that’s what we’re shooting for.

    Ashley: Now, you’re operating in AWS in an elastic environment, you know, storage is right there, you want some more, you just grab it, use it. I would imagine that’s very easy to get out of control proliferation. You need some data management capabilities around what you have as well as the whole process of backing up data, restoring, recovering, making sure you’re not losing information—all of that.

    What are some of the key things that are different about operating in a cloud environment versus maybe a traditional on premise data center?

    Palmer: Great question, and let me start by giving us some background statistics to frame up how big this challenge is and the opportunity is. You know, the thing that we keep most in mind is, if you look at Amazon’s five year Kager of cloud storage pricing, it’s negative 60%.

    Ashley: Hmm. Going down.

    Palmer: In a world like that—it’s going down and if you talk to Amazon, they will say that that is the path that they will continue on over the next five years, enough to challenge customers with, “Tell me, when you buy an appliance today, how much cheaper is it gonna be next year or the year after or the after?” It’s likely it’s gonna be the same, so they’ll lose out on the innovation in the industry, even just around the ability to create better price and performance. ESC did a very nice survey and asked CIOs in their premises data centers what were the top five opportunities for cost efficiency? And number one on that list was storage management.

    Ashley: Mm-hmm.

    Palmer: You know, enterprises are dealing with this long history, and you had talked earlier about your history as a CIO. All CIOs deal with generations of proliferation of storage vendors as well as the hardware itself, the incompatibility, and just even the ability to track it, tough challenges, and the third stat, which is the one that I find most interesting because it indicates the tipping point is, in 2021, so just about a year and a half from now, we’ll see the crossover point where there are more dollars spent in cloud storage than premises storage. So, this will be the point at which companies can effectively be viewed as cloud first from a storage point of view to the point where they’ll have to defend [Audio skip]. And in that world, your first question about what do we have to help them with—we have to help them realize the promise of those savings.

    You made a good point—it can easily get out of control, especially where you have very distributed teams who are building applications and taking advantage of that infrastructure in a more decentralized way. We have to help them manage the tiering, we have to help them manage the data classification, we have to help them manage not just the cost side, but give them advice on performance. Much of this can be automated through machine learning. So, we’re gonna give them a new model for what old terminology like ILM or information life cycle management or HSM or hierarchical storage management, we’re gonna turn that, really, into a background automation process and let them focus on what matters, which is, do I have the right policy, can I leverage this data for other uses, and can I do that without special skills?

    Ashley: Well, you know, you bring up some really good points and that is, it’s one thing for a startup or someone who’s building a Cloud Native application as a young company or maybe a one-off kind of application. Enterprises look for environments that are gonna de-risk or things that are gonna de-risk their move to the cloud. And I would imagine having a structure that it’s at least familiar so you have not only data protection and data management, but you’ve got governance and things like that. You’re able to do some forensics, predictive analysis things like that. The kind of functionality that they’re used to having on prem, but maybe in a cloud environment or possibly even a hybrid model.

    Palmer: I think—yeah, you nailed it. And one of the words that I used frequently is convergence. You know, one of the benefits of cloud is that because you’re provided with all of the substrate, you’re provided with compute, you’re provided with storage and networking and policy and all kinds of things surrounding you that the job of a company like Druva and even the job of enterprise developers is just to focus on the application logic.

    And what that means in our space is, as you mentioned, you have a lot of surround in your data center. Customers that, in my former world, they would buy a product for backup, they would buy a product for disaster recovery, they’d have an e-discovery product, they’d have an archived product, they’d buy a data classification product.

    Ashley: Been there, done that. [Laughter] Been there on all those.

    Palmer: Painful, right? You know, and all you see is your costs going up and complexity going up and different skill sets and your training and conferences to go to and it’s slowing you down, right? And when you look at Druva in a single platform, we simply consider those ancillary value propositions like e-discovery or data classification as on demand, callable apps.

    Ashley: Mm-hmm.

    Palmer: So, when you back up with us, you simply tack on the DR service, you simply tack on the data classification. You have a work flow for legal hold. And by the way, you pay for this on demand, you don’t have to pre-pay capacity licenses, you don’t have to pre-pay storage.

    Ashley: Mm-hmm.

    Palmer: So, you have a world in which not only do we replace your current apps, but we have a world in which, as customers come to us and we release new capabilities every other week, that they can put something on a roadmap and see a functional working application in weeks and months over their existing data set, paid for only when they use it. You know, and this is revolutionary for most CIOs.

    Ashley: Usually you have to buy the amount of storage you think is sort of your top capacity and use it as you go and then potentially upgrade where, of course, in the cloud, it’s elastic, it’s fungible. We can change, add more as we need it.

    Palmer: And of course, you have to do the same with your software licenses, you have to then integrate those products, you have to find space in your collocation facility. You know, when you really analyze how much cost and effort it is to adopt a new piece of software or a piece of hardware, it’s far beyond the obvious.

    And not to mention, by the way—and this is a stat that I quote to a lot of people. It has been in my experience that, when a software vendor releases a new version of a product, it can be 12 to 18 months before even half the install base adopt that version. So, if agility and speed matter to you as a business in adopting capabilities, you have to take risk and wait a long time before you get a capability, whereas in SaaS, you get it on demand with no effort—again, completely changes the model.

    Ashley: I mentioned hybrid earlier, something we haven’t talked about yet. Most enterprises can’t lift and shift everything into the cloud. As a matter of fact, they may not want to. They may want to keep some hybrid either multi-cloud or on premise. Is that an area that you can readily address with them?

    Palmer: Yes, and thanks for asking that, because one of the difficulties in being unique in your space is just explaining your model in the first place. And from a SaaS point of view, while our application architecture is Cloud Native, we service customers in their premises data centers and into the public cloud, including their SaaS providers as well.

    So, what that means is, as you said, you know, if you have a customer that has—and most are within this category—have a data center where they have workloads where their recovery times are perfectly fine to take advantage of all of those benefits I mentioned earlier in the cloud, we help move them to the cloud and manage and restore and so forth for them to their local premises facility. However, if there is an application that has an RPO or a recovery point or a recovery time objective that requires it to be within the data center, we also kind of provide them with a virtual appliance that solves that problems as well.

    And then, as they build native cloud applications and, for example, Amazon, AWS, or they adopt Office 365 or Salesforce, we also help data protection and classification, both natively in Amazon as well as with SaaS providers as well. So, we really are in them with their current architecture. We’re also with them in their future architecture.

    Ashley: Yeah, that’s the key to transition, I think. Again, I mentioned earlier about de-risking and part of that is familiarity as well as giving me a path to get where I want to go, and I may not fully go to the cloud, right? I may always have to have something, as you mentioned, because of RTO or RPO requirements to be on prem.

    Palmer: That’s right.

    Ashley: Tell us a little bit about some of your, some of the capabilities that are just kinda native with the product that, you know, what does it take to get started using Druva? Can I just plunk down the credit card and, like Amazon, I’ve got all of it available to me, or is there a set of planning processes that are helpful to go through to properly architect a data protection architecture like this?

    Palmer: Oh, and we love this question, because again, coming from the industry where the idea of even doing a POC is a very complicated process of governance rules and software installation and finding hardware and disk capacity. With Druva, you literally sign up on the online platform. We give you access to the agent type that you’re looking for, so let’s use VMware as an example. You can download a proxy, provide it your credentials, put a policy in, do all of this within about 10 to 15 minutes and you can initiate your first backup.

    And you can do this and we have tested this over and over without having specialist backup administrator skills, which means you can also start delegating privileges to application teams and such where you as an administrator have the ability to enforce and create overall policy. So, now you have a world where your most mundane, everyday tasks are made so simple that you can delegate them to your customers, your customers from an IT provider point of view—which increases their satisfaction, because they can do these things on demand, and they free up your time to do more important things.

    Ashley: That’s awesome. I was breaking out in a cold sweat as I was asking that question. Having done this at multiple companies, it is not a fun process to stitch it all together with different products and really have to architect it properly because you have to worry about things like how are you gonna monitor compliance. You mentioned e-discovery. Now, we have things like GDPR compliance as well as our own internal processes about data retention and making sure we’re protecting ourselves against ransomware, things like that.

    Palmer: And we deal with some of the largest companies in the world, and one particular use case we saw super well is on the end point side where customers are dealing with either legal situations or they’re concerned about end user behaviors and what they’re doing with data. So, we not only provide them the ability to protect devices, but we also provide them with ideas about what’s going on with the data, and then certainly with workflows as you mentioned to go and pull that data aside or do a hold where they need to—again, this is all just workflow in an interface. It’s not trying to get one product to work with another product or making sure that versions are upgraded or compatible or patched sufficiently. We take care of all of that.

    Ashley: Mm-hmm. As I’m kind of peeling the onion here remembering things that you have to address with a data management approach, you know, you have to deal with things—you mentioned tiered storage earlier. Does de-duping apply in a cloud environment? Does caching apply in a cloud environment? Are those things that you also address?

    Palmer: All of the above. So—and you asked a very good question earlier. If I’m a customer adopting the cloud, where are some of my risk areas or things that aren’t provided to me? And one of those examples is de-duplication. So, if I’m going to adopt S3 or I’m going to move data into Glacier, how am I gonna do that efficiently in a world where I’m potentially coming from a de-duplication position on premises, I’m gonna lose that in the cloud.

    With Druva, we de-duplicate that data for you, so we are minimizing your cloud storage footprint. And once we’ve done that, we’re also taking care of the tiering into whether it’s S3IA or Glacier or Glacier Deep Archive, we’re not only replacing your HSM strategy, we’re doing that, as I said earlier, in an automated way.

    Ashley: Mm-hmm. What’s nice, too, is—again, I’m kinda having the negative flashbacks here. The old process of which are tier one, how much in memory storage are you gonna have at your first tier of storage, how much are you gonna have at this speed, how much are you gonna have at your third tier of speed. With the different services that Amazon provides, that’s what you’re bringing together in one collection in a full data management architecture so you can, of course, dynamically allocate any of those as you need them, but that’s what you’re tying into is those services as well as what you provide.

    Palmer: Capacity planning is a non-issue in a cloud and SaaS world. Of course, we will work with you where, as you mentioned earlier, where you have RPOs and RTOs on site where you know that the use case doesn’t apply to a remote data center. We’ll work with you with a virtual appliance. We also have customers that will always want a land speed restore, but they’re increasingly collocating in places like Equinix, for example, where they can cross connect into the cloud provider.

    So, even where you have requirements that look like data center requirements, it’s almost as if your data center is moving into clouds in certain collocation facilities where you can take advantage of SaaS even without the virtual appliance.

    Ashley: What do you feel like is the biggest pain point that you address for customers? Obviously, they have needs, functionality, capabilities—all of that kinda thing. But when someone comes knocking on your website and says, “I think I need this,” what is most commonly the biggest problem that they’re trying to address?

    Palmer: There are two problems, but the compelling events are different. So, the two problems we’re solving for them are cost. They look into the future and they realize their costs are going up, not down, and cloud can help solve that problem.

    And the second one is complexity. They don’t want to have to hire more storage administrators or backup administrators, track different vendors. They don’t have to do hardware refreshes. They don’t want to have siloed systems. These are problems that are all solved with SaaS provides like Druva and the public cloud. So, those are the core problems.

    The compelling events, on the other hand, can be very different. We have Fortune 100 companies that have data centers all over the world, and their cloud journey can include moving the satellite facilities to the cloud today and increasingly looking at cloud and the core data centers tomorrow. So, they have a more data center migration consolidation use case.

    We have others that have remote offices or field facilities where supporting an appliance is expensive if not impossible because they don’t have the staff. So, it’s refreshes and getting equipment out of facilities is the second one.

    Ashley: Mm-hmm, mm-hmm.

    Palmer: We have customers that come to us because of ransomware, where we offer an air gap solution where the only alternative on premises was tape. So, we give them the same security via encryption and have an air gap solution to their primary network that no one else can. So, they’re adopting the cloud with the safety of tape.

    And then, of course, your basic use cases of backup and disaster recovery, where customers want not just to be able to restore data but move an application, have the full orchestration capability, and instantiate that without the additional costs of owning their own architecture or frankly even managing their own cloud integrations.

    So, we integrate a DR capability in our platform. So, customers come, they get native backup, they get native disaster recovery all in one interface without other products or other training. So, we have compelling events and we’re saving our customers 50% based on their existing legacy architecture and we have proven that a few different times via external experts as well as customers themselves and complexity and really taking all of the legacy mundane tasks and skills and making them obsolete.

    Ashley: Well, Mike, I’d like to thank you for joining us. We’ve run out of time, here. I think we could talk another hour about data management [Laughter]—

    Palmer: [Laughter] For sure.

    Ashley: —share some war stories along the way, too. So, I’d like to thank you, Mike—Mike Palmer, chief product officer at Druva, for joining us.

    Palmer: Mitch, thank you very much. It was a great time talking to you. Maybe we’ll do it again.

    Ashley: That would be great. We’d love to hear an update as things kinda move forward and you spend some of that money. [Laughter] And I’d, of course, like to thank you—you, our listeners—for joining us today. This is Mitch Ashley with devopsy.kinsta.cloud, and you’ve listened to another DevOps Chat podcast. Be careful out there.

    — Mitchell Ashley

  • DevOps Chat: Hybrid, Multi-Cloud Management for DevOps With CloudBolt

    DevOps Chat: Hybrid, Multi-Cloud Management for DevOps With CloudBolt

    Agile, DevOps, multiple cloud providers, serverless, contemporary cloud native apps, shadow IT using a credit card–it can be daunting for any IT organization to be responsive to the internal customer needs. It’s even tougher to be proactive and get ahead of the curve. Enter Cloud Management Platforms (CMP).

    On this episode of DevOps Chat, we talk with Bernard Sanders–no, not the presidential candidate–CTO of CloudBolt. Our conversation explores how IT can use a CMP to provide the IT and self-service capabilities so DevOps and Agile teams won’t feel the pain and slowdown of IT past.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. Today, I’m joined by CTO Bernard Sanders from CloudBolt. Our topic today is DevOps for hybrid and multi-cloud environments. Bernard, welcome to DevOps Chat.

    Bernard Sanders: Thanks, Mitch. Great to be here.

    Ashley: Super good to have you on the podcast. Would you start out by introducing yourself? Tell us a little bit about you and also a little bit about CloudBolt.

    Sanders: Sure. So, I’m Bernard, CTO and co-founder of CloudBolt, and spent most of my career in the data center automation/DevOps/cloud management space as it’s been known throughout various eras, you know, since 2000 when I started in the industry. And I’ve gotten to see a lot of different enterprise IT shops and how they work—what goes well, what goes wrong. I spent a lot of time in IT shops and the financial services industry and in retail and government, and it’s been interesting seeing the commonalities between these.

    And it was through an observation of those commonalities and talking with a colleague of mine, Augie, that we decided to start CloudBolt software in 2011 and started working on the software, put out first GA release that year. And CloudBolt is a hybrid cloud management platform, sometimes called a CMP. It essentially helps large organizations who have a data center presence and also want to use the public cloud. It helps them consume both from a single interface.

    Ashley: Cool. Well, first of all, I have to compliment you—anybody with a buddy named Augie has to be a good person.

    Sanders: [Laughter]

    Ashley: Augie sounds like a great guy. [Laughter] So, this was like, sort of from your garage, kinda from the basement type of startup that you two started it originally and got this company rolling?

    Sanders: That was basically it. We did have a leg up in that we were working for a consulting company at the time based out of Washington, D.C. named August Schell, and we spun this company out of August Schell. So, starting up, the two of us and the others we brought on were able to focus on the technology and the solution and going and getting some customers, and we didn’t have to worry so much about the back office stuff—the HR, payroll. We piggybacked on our parent company’s infrastructure for a lot of that, so—

    Ashley: Nice. So, kind of a spinout or bootstrap off of the parent.

    Sanders: Exactly, yep.

    Ashley: Very nice. Alright, well, you know, not everybody gets to start out doing cloud applications in a cloud-native format, or even just within one cloud. You know, oftentimes, we’re saddled with complexity, we’re saddled with the existing applications—I don’t mean it in a bad way, it’s just the reality of the business—or we’re part of a digital transformation moving to the cloud, which is sort of where we get these hybrid and multi-cloud environments.

    What are some of the challenges that you saw that set you about, “Let’s go start this company, build this cloud management platform.”

    Sanders: Yeah, yeah. So, there’s a couple areas to discuss there, you know, the first of which you touched on is that not everybody gets the luxury of starting greenfield. You know, if you’re starting a brand new software company with a few people and a few needs for servers, you could start entirely containerized, serverless with the latest technology. But so many companies we talk with, still ask us, “Oh, do you support mainframes? Do you support AIX, do you support HQX?” [Laughter] The answers to those questions are no, in our case, you know? We gotta draw the line somewhere. But there’s a very long tail of technology at these enterprise companies. There is a need for stability and continuity and they can’t just drop everything and move to the latest platform with all their applications.

    Ashley: You know, I like to say generations of technologies never go away.

    Sanders: That’s correct.

    Ashley: There’s probably code I wrote when I got out of college, scary enough, if we think about that. But yeah, it is. You’ve got to adapt to what you have. We can’t rewrite everything.

    Sanders: You know, your other part of your question was what problem did we see that inspired us to start CloudBolt, and what we were seeing, Augie and I, any time we went out to these large institutions is a breakdown in the communication between the IT department at a large company or organization and their end users.

    Ashley: I’ve heard that happens, yeah. I’ve never seen that, but I’ve heard.

    Sanders: [Laughter] Yeah. It just happens so much it’s almost a joke, but you know, end users will have to put in a ticket with IT if they wanna get a VM, either in their private data center or sometimes even in the public cloud. And then that IT group has to go do a lot of manual steps and there’s oftentimes a lot of hand offs between groups to eventually build that environment or that individual VM or network storage out and then get it back to the user. And it’s a black box and everybody gets frustrated with each other.

    Ashley: Mm-hmm.

    Sanders: We thought—there’s gotta be a better way to do this.

    Ashley: So, how organizations can work together more effectively, let’s also tease apart both the multi-cloud and the hybrid aspect of this. So, multi-cloud, do you find that’s because things sort of start up on their own and different groups go out and somebody starts on Azure because they’re Microsoft code, and somebody starts out on Amazon or Google or wherever and then some day, somebody else says, “Why don’t we pull this together?” Or is it more of a conscious, “No, we need these different cloud environments for these good reasons, so let’s put together a cohesive strategy or architecture for how we go about that?” How do those things evolve?

    Sanders: I think it’s both. I’d say it’s probably 60/40. A lot of times, it’ll be a company that says, “Oh, we’re all on Amazon.” Then they go acquire another company that does Azure and then another one that does something else. So, a lot of times, it’s that unintentional growth.

    Ashley: Mm-hmm.

    Sanders: But I think more and more, you’re seeing each of the cloud providers, they are diverging over time and they each have their own strengths and weaknesses. So it’s totally valid to say, “Hey, we’re gonna use Google Cloud for this one thing and we’re gonna use AWS for this other use case, because that’s where these platforms are strong.”

    Ashley: Interesting, interesting. So, talk about the hybrid part of this. You know, whenever there’s a new generation or a new shiny object, it’s like, everything is gonna go there. So, everything is gonna go to the cloud, and of course, many things have and some things have fully and some things have just partially.

    Sanders: Mm-hmm.

    Ashley: And there’s always that need. You talked about mainframes, for example. There’s that need of having a hybrid environment. How do you define a hybrid environment?

    Sanders: Yeah, so I define hybrid as using both a private cloud and a public cloud. So, private cloud meaning your own data center or, you know, maybe one that you colo or somehow rent out. But basically, your own gear, your own hardware and infrastructure, but also using the public cloud at the same time. And that—you know, I think that’s been one of the biggest about faces in the last five or six years is, if you went back five or six years ago, listening to a lot of the industry analysts, you would’ve thought that we’d be off public cloud, you know, a year or two later. Or we’d be off private—

    Ashley: Of private, yeah.

    Sanders: –and that would go totally extinct.

    Ashley: Of course.

    Sanders: And that we’d be completely on public cloud. But I think that was an unrealistic expectation. There was a Goldman Sachs study earlier this year that said 26% of workloads are in the public cloud. So, that leaves a huge portion that’s still a huge majority in the private environments.

    There’s a growing need for hybrid, you know? People don’t want to use both totally disjointedly, they wanna use them together in some way. That’s what I would call hybrid cloud.

    Ashley: That kinda sets the context for talking about this cloud management platform. Define that for us—what is a cloud management platform?

    Sanders: Yeah, cloud management platform is a tool which can provide a single pane of glass or interface as well as API to manage disparate IT environments, and that can be public cloud, private cloud, containers, virtual machines, physical. It can include serverless, it can include public cloud services as well as basically any request you would make of IT. That’s what I would call a cloud management platform or CMP. The other main defining characteristic is, it needs to provide self-service to end users. It has to be a tool where end users can go and get what they want on their own rather than asking IT to do it for them.

    Ashley: So, really, the ability for IT to manage its environment but also to provide portal access, if you will, if you thought about it that way, anywhere for people to do self-service, APIs, STKs, so DevOps groups can tie into the platform.

    Sanders: Exactly. Yep.

    Ashley: Okay. So, let’s go there. Let’s talk about how does DevOps fit into—IT wants to organize things, right? IT wants to get its hands around and be able to at least manage—you know, not necessarily control or constrain, but you know, they’re responsible for what’s happening in the organization with the technology and so, they want some form or semblance of management control over that.

    How do you do that and, at the same time, let the DevOps groups thrive and flourish?

    Sanders: That’s right. Yeah, so, really what it’s about is having the IT team, giving them a tool where they can set their policies, set the standards, but expose an interface, or multiple interfaces, to different departments or groups for those teams and DevOps folks within those teams to go get what they need when they need it and automate what they want on top of that base that IT has given to them.

    So, in the case of most cloud management platforms, the IT group would bring it on, purchase it, get it installed and set up and then expose that UI and API out to these departments who can code against it. So, the DevOps team in a particular department, for example, might have a CI/CD pipeline that calls into the cloud management platform that’s exposed by IT.

    So, both groups are empowered. IT gets to set the quotas and the approval process and the host name standards—like, everything policy related within that cloud management platform, and those end users and DevOps folks get to provision what they want when they want it.

    Ashley: I don’t want to make it sound like a panacea, but it sounds like this would be a path for an IT organization who maybe has lost control or doesn’t feel like they have their arms around everything that’s happening in the organization and they can become sort of the provider of the cloud computing infrastructure resources.

    Sanders: Absolutely.

    Ashley: And this could be a path to them kind of getting in front of what’s happening instead of just being reactionary.

    Sanders: Absolutely, yeah. The other effect that it has that we’ve seen, you know, CMPs have, whether it’s CloudBolt or a different one is that, you know, the private data center has lagged behind public cloud in terms of its interface. Like, going to the Amazon console or Google cloud, ordering a VM—it’s a pretty easy, fairly self-service process. It’s not that way with OpenStack or VMware and Nutanix. I know it was one of private virtualization systems people use.

    Having a cloud management platform levels up your private data center to a level that’s similar to public cloud where people can do self-service in the private data center, not just the public cloud any more.

    Ashley: Okay, so at risk of me sounding like I’m just happy ears and throwing you softballs—

    Sanders: [Laughter]

    Ashley: –what’s the hard part about implementing a CMP?

    Sanders: [Laughter]

    Ashley: It can’t be all roses and, “Here’s a credit card and download and—poof, your NPS scores shoot from 10 to 56.”

    Sanders: Oh, can I only name one thing?

    Ashley: Yeah. [Laughter] What are the challenges of getting from “life’s pretty tough” to, “we’re in a much better position now that we’ve gotten this platform in place”?

    Sanders: Yeah. So, for folks who bring in a cloud management platform, I think the challenges are oftentimes less technical and more organizational, because it affects so many different groups. You have to get people’s buy-in that, “Hey, we’re gonna do this thing. We’re gonna use this as our common interface to IT.” And that can be challenging. I think it’s made easier if the interface is good and you make it attractive and appealing and powerful and give them what they need in that interface, people will come to it. But it can still be hard enacting any kind of a change within these large organizations.

    Ashley: So, is it easier in a smaller organization, mid-sized or large? I can imagine maybe it’s easier in a large enterprise where you can sort of lay out standards and how you have a process for doing that or maybe mid-sized is tougher because it’s harder to herd the cats? I don’t know, what’s your experience?

    Sanders: That’s a good question. I think, just generally, any time you get more humans together, it’s harder to make change and to get them all doing the same thing. And, yes, if you have executive buy-in in a large organization, that can make it easier, but I think it’s still basically always harder to move more people than it is to move fewer people to a new way of doing things.

    Ashley: Okay. So, another, tougher question—why is CloudBolt better than the rest of the CMPs on the market? Why is it a better mouse trap?

    Sanders: [Laughter] Good question. You know, I don’t do a lot of the competitive analysis myself, but what I’ve heard from our customers is that they can get CloudBolt installed and running and get value out of it much more quickly than other platforms. You know, we’ve done demos before where, by the end of the demo, you know, where in a GoToMeeting or a Zoom meeting or we’re showing the product to a customer and by the end, they’re asking us, “Oh, where’s the button for X?” We have to pause and say, “Wait, what’s going on over there?” In the time it’s taken it to demo it to them in half an hour—so, they’ve downloaded it, installed it, connected it to their VMware environment, they’re, like, provisioning VMs. It’s a pretty nice bar, I think, we’ve hit in terms of consumability. They also talk about the extensibility being very powerful.

    We try to build all kinds of hook points and extension points, capabilities to let them extend it to do whatever they need it to. A lot of these IT shops have very specific requirements and systems and standards and ways they need things to operate, so they like that extensibility.

    Ashley: Well, teams doing agile, DevOps teams, you know, they build an environment around them where they can execute quickly because they can get access to resources quickly, you can go to Amazon credit card—poof, I’ve got whatever. And it’s great to build that kind of process and that environment around them. Of course, we run into things that are gonna slow them down, like quote-unquote IT, or some new thing, the shiny new CMP kind of thing.

    How do you get a DevOps team on board that’s been just fine going to Amazon and Google and Azure and whoever else to get what they need and now they need to go through this IT thingy that’s been placed in front of them?

    Sanders: Yeah. I think, really, what you need to do is solve problems that they’ve got, and CMPs do that. You know, when they see one interface, one API that is consumable and up to their standards and allows them to deploy to any different cloud as well as the private data center, usually, it doesn’t take much more convincing than that. They say, “Wow, we were gonna have to build that,” because we were just given the mandate to do two different clouds or to do VMware plus Azure.

    And now, we don’t have to do that ourselves—that’s great. We can build more functionality and just build on top of that great platform there.

    Ashley: Can you come in and work with an existing either homegrown automation, or if they’ve built around some build and automation tools for deploying both code and to the cloud? Can you work with tools that a DevOps team has already set up, or do you have to kinda retool parts of what they’re doing?

    Sanders: There’s usually very little retooling. There’s bidirectional integration that usually happens. You know, if somebody’s gotta build pipeline in Jenkins or Concourse, they’ll oftentimes have that start calling into CloudBolt to do those builds, and they can sometimes remove some of the complexity from their systems if they were calling directly into infrastructure to do that themselves.

    And then, similarly, CloudBolt can call out into existing tools they’ve got. A lot of our customers have homegrown cloud management or change management databases, CMDBs, and they want CloudBolt at some point in the provisioning process to call into those to register a new server or CI in there.

    Ashley: Mm-hmm. What’s the next thing that’s happening for you with your product and where you’re taking this?

    Sanders: There’s a lot of things that are happening in the cloud management space, you know? The space is not too old, so it’s still defining itself, and the industry is still figuring out what they need in this kind of a tool. So, we have been introducing increased cost management and analytics to show you what your public cloud spend is and what you could save and give you recommendations on how to save. So, that’s a common area of improvement, continuing to make it more and more extensible and consumable, of course, doubling down on our containerization support, we started supporting Kubernetes in 2015 and we’re leveling that up in each of our releases.

    And then, as well as supporting all of the different directions the public cloud providers are going, they’re trying to diverge; we’re trying to provide a common platform. So, that’s a constant challenge for us to solve and it’s a fun one and it’s one the engineering teams thrive off of to give people a common interface with increasingly different underlying technologies.

    Ashley: I can imagine, obviously the more tool support that you provide for Kubernetes to Jenkins or whatever the next iteration of fantastic tools that are used in a DevOps environment, all the better. But also, if you’re solving, “Here’s log aggregation, here’s audit trails, here’s access controls that DevOps teams don’t have to set up,” that would be—and you can do it through a programmatic interface—that’s gonna be a win for them.

    Sanders: Absolutely. Yeah, and security is an increasing focus for us as well, providing tools and integrations through which they can make sure every VM that’s deployed is up to standards and compliance and they can get security reports across their entire enterprise.

    Ashley: Yeah, if you can help them sell through a security audit or review, they’re gonna love you, I would think.

    Sanders: Yes.

    Ashley: We could talk about this for another 40 minutes if we wanted to. Unfortunately, we don’t have that much time. I’d like to thank you, Bernard Sanders from CloudBolt, for joining us.

    Sanders: Thanks, Mitch. It’s been really fun.

    Ashley: And I’d like to thank our listeners, of course, for being with us today. This is Mitch Ashley with devopsy.kinsta.cloud. You’ve listened to another DevOps Chat, and be careful out there.

    — Mitchell Ashley

  • Out of Date: Cloud Anti-Trends of 2019

    Out of Date: Cloud Anti-Trends of 2019

    Let’s talk about 2019 cloud tech anti-trends. A lot of those anti-trends are nothing but cloud myths, which are still widely spread and talked about.

    In this article, we’ll take a look at some anti-trends and try to understand where’s the myth part in them.

    Private Cloud Is Not Popular Anymore

    According to the IDC research organization, by 2021, companies from around the world will spend more than half a billion dollars on cloud tech. This is two times more than in 2016. At the same time, there is a clear tendency: The public cloud workloads are growing, while the situation is exactly opposite for the private cloud. Cisco said, after three years 73% of the company’s applications will move to the public cloud, while just a year ago this figure was around 50%.

    Such a shift is partly caused by cloud providers, who offer users of public clouds more and more features that simplify the development of services. For example, tools that allow creating machine learning models, manage software deployment processes and analyze logs.

    That’s why many felt the active development of a public cloud meant the final decline of the private cloud niche. During his speech at a conference last year, Anthony Lai, senior vice president of the cloud-based division of NetApp said private clouds would “die out.”

    I wouldn’t expect the private cloud to disappear completely. The statement that it will cease to exist is only partly true. The private cloud may disappear as an independent unit, but it will remain part of the hybrid IT infrastructure.

    As noted in Gartner, 72% of companies in the world use a hybrid strategy. And it is expected this figure will grow. According to analysts from Market Research, the market for cloud object storage systems will reach six billion dollars by 2023. This will definitely spur further development of the hybrid IT infrastructure.

    One reason to use private clouds is predictable business processes.

    Yes, the public cloud offers flexible scaling. The company does not have to urgently buy new hardware if the load on the services has increased dramatically (for example, the application became a success and tons of new users came in). In this case, you can simply rent additional computing power. But if the load can be predicted (say, it is static) a private cloud is often enough.

    Also, a private cloud is often used to create object storage. Sure, object storage services are also available in the public cloud and a number of companies use them. Just a couple of years ago, Netflix migrated its entire infrastructure to the public cloud.

    However, not all companies need to build a distributed infrastructure as Netflix did. There are cases when companies have object storage in a private cloud because it already has an appropriate low-cost automated infrastructure. CERN is one of them. All the data generated by the Large Hadron Collider is stored in a private cloud.

    The IT-Infrastructure Market Is Oversaturated with Specialists

    Today, companies are actively abandoning their own infrastructure in favor of cloud solutions. That is why the need for in-house specialists engaged in its maintenance, gradually disappears. The IaaS provider takes on many issues related to the management and administration of the infrastructure, and the specialists need only ensure that everything is set up correctly.

    That is also why companies often abandon their own IT departments. At first glance, it seems this will gradually lead to instability in the labor market. Experts in the field of IT infrastructure, system administrators and data center operators will have to compete fiercely for the remaining jobs. But will they really have to?

    This does not mean such specialists have become completely unnecessary. In reality, in-house specialists simply change the focus of their activities. Most of the IT infrastructure administration tasks are now on IaaS and IaaS providers are now the ones who need and are actively fighting for such specialists.

    Even when you do use third-party solution, that is what happens sometimes:

    When customers start using intermix.io for the first time, they can see the set-up and configuration of their Amazon Redshift cluster in the context of their queries and workflows.

    At that point, customers experience one common reaction: Knowing what we know now, how would we set up our Redshift cluster had we do it all over again?

    The answer is not that straightforward.

    However, sometimes it happens that migration to the cloud stimulates the company to strengthen and develop the IT department more, attracting new cloud employees. These people study the cloud ecosystem, analyze the trends and evaluate how the transfer of an application to the cloud will affect the company’s work. If the company is already working with the cloud, then cloud specialists are also exploring new features of the current platform and exploring services that will benefit a business in the future.

    Clouds Don’t Hold New Technologies

    Four years ago, IBM listed seven cloud technologies to help businesses. The list includes IaaS/PaaS, private/hybrid cloud, test and application development environments, big data processing and disaster recovery solutions, along with data storage and backup systems.

    These technologies served as a standard technology stack for a long time. It was all developed with the efforts of IaaS providers. Therefore, there was an opinion that the cloud ecosystem was finally formed and no other use cases will be foreseen in the near future.

    New Directions Still Appear: The blockchain technology was born in 2008, gained worldwide popularity 8 years later and came to the cloud about a year ago. Based on the virtual infrastructure, a new direction has started to develop: Blockchain-as-a-Service (BaaS). In this case, the client receives a virtual environment prepared for working with blockchain technology. Users can create their own applications on the blockchain (for example, for processing financial transactions) and run them in a secure virtual environment.

    Quantum Computers: the most recent, fresh cloud trend. The quantum machine uses qubits instead of bits we are used to. Unlike bits, qubits can take the value of zero and one at the same time. Thus, a quantum machine can be in several states at once. The more qubits a quantum computer has, the faster it performs the tasks assigned to it. Imagine how much faster it can brute-force passwords or simulate chemical reactions.

    Quantum computers already have practical applications. IBM has recently opened access to its 20-qubit developments via the IBM Q cloud platform. And this year, Alibaba partnered with the Chinese Academy of Sciences to deploy an 11-qubit quantum device in the cloud. Quantum Computing as-a-service (QCaaS) opened the way for experiments on the infrastructure of providers and proved the cloud industry can adapt to the emergence of new technologies.

    Another Trend Is Fogging or Fog Computing: IT-industry experts are trying to solve the problem of delay in transferring data to the cloud. In the case of fogging, server components are not located in the centralized data center, but on the periphery, close to the end-user devices.

    In such an architecture, a main, large cloud still remains but has additional networking and computing layers: cloud, fog, edge. The Cloud (cloud layer, primary) solves expensive tasks for periphery machines (edge layer, end-user): data analysis, system logging, etc. The fog layer does local calculations in real-time to generate reports and send them to the central hub. This architecture improves the performance of the entire infrastructure by significantly (up to 10 times) reducing the delay in data exchange.

    To Sum It Up on Cloud Myth-Busting

    So, as with everything, rushing into extremes doesn’t help. That’s what we figured out:

    1. Even though public cloud seems to be a mainstream option, private cloud infrastructure still makes sense for certain use cases.
    2. Most of the IT infrastructure administration tasks are now on IaaS and IaaS providers are now the ones who need and are actively fighting for Engineers i.e. market is more than far away from being oversaturated with specialists.
    3. Clouds don’t hold new technologies. Did anyone really believe this one anyway?

    — Dmitrii Borodin

  • DevOps and Networking: Working to Achieve Nirvana

    DevOps and Networking: Working to Achieve Nirvana

    Enterprises don’t own the cloud, just the experience. If there’s an outage or even a hiccup in the performance or network service, it’s on the network and application teams (not the cloud provider) to fix it. But often with problems in the cloud comes finger-pointing, as network and application teams look to determine liability and responsibility when it comes down to crisis situations.

    There is no question that applications are growing more complex, as is the infrastructure to power those applications. According to the 2019 “Stack Overflow Developers Survey,” comparing year to year the amount of time it takes for software engineers to become effective on teams is increasing. This trend could be, in part, due to the amount of tooling and rise of an operate-what-you-build model (e.g., the dawn of the full lifecycle developer). As our applications become more distributed—and to cope with scaling and availability demands—skill sets that traditionally were infrastructure and development-based have begun to blur.

    Networking Shifting Left

    Networking-centric capabilities such as service meshes (e.g., Istio and Linkerd) and container networking interfaces (e.g., Calico and WeaveNet) are gaining a lot of traction and visibility in the cloud-native application development world. Though SDN infrastructure has been shifting left toward development pipelines, application development teams are learning networking concepts. However, the core networking infrastructure is still maintained by the networking teams.

    This paradigm of software-defined infrastructure is creating a learning curve for multiple teams. With the dawn of NetOps, software development lifecycle (SDLC) principles are being introduced to the networking teams. Traditionally, ticket- or service-driven networking teams can participate in iterations and they are now building the next generation of cloud-agnostic networking capabilities. With NetOps, there is more interaction and innovation. But change is not without resistance; the dynamic nature of modern infrastructure and the short-lived nature of pre-emptible/ephemeral infrastructure do not bode well for static networking rules.

    Networking Team Pressures Today

    With infrastructure headed to the trifecta of clustering, replication and load balancing, networking teams have to deal with many more dynamic addresses than before. Static ports and addresses that used to be loaded into networking tickets are now being replaced with wide ranges and service addresses powering the networking stack of our cloud-native and containerized workloads. Adding to the complexity, the networking team is leveraging public cloud infrastructure. The infrastructure that the networking teams are responsible for is leaving the data center and distributed web infrastructure technologies, such as distributed domain name systems (DNS) and content delivery networks (CDNs), are common for the networking teams to be involved with. Like the application development teams, the amount of new technology hitting the networking teams is increasing.

    One of the great advantages of using public cloud infrastructure is the ability to quickly expand or contract available resources. With this ability, we can spin up and down test environments to try out changes and even the infrastructure level that was not even possible. With this, we have the ability to integrate more. The application development team, which has been iterating for some time, now has common ground with the networking team to try out iterations.

    All in a CIDR

    To illustrate the needed collaboration between networking and application teams, let’s look at an example:

    In one of the biggest incidents of my career, my company was migrating pieces of our platform to a public cloud vendor. Because the organization was early on in this transformation, a lot of minutiae was yet to be flushed out. We were packaging classless inter-domain routing (CIDR) calculations into our first-time VPC configurations—a task the development team was responsible for at the onset of our cloud migration. It was my job to enter this for my service. Instead of a /8 CIDR, I implemented a /16 CIDR. A seemingly small miscalculation on several configs turned into an incident, blocking a majority of traffic hitting the service. The mistake was quickly corrected, but the incident response framework was well on its way to being executed.

    So, who was responsible? I was the one who created the configuration and pushed to production, so this was my fault. But in retrospect, the best solution would have been to involve the networking team immediately in these types of changes. Because the application development teams were vanguarding into the public cloud, however, we cut out the networking team. This was an organizational problem that needed to be addressed. If we were going to deploy into a data center successfully, the networking teams needed to look over my requests and catch my CIDR error.

    Silver Lining, Right?

    During this CIDR incident, I was trying to look at the silver lining: Since we can’t access our application, that means the instances/app must have a lot of extra capacity, right? Untimely, yes, but the analogy did make sense with the silver lining. We were getting billed by a public cloud vendor and because there was a drop in usage, our bill technically should have been lower. But it still impacted the customer. The money saved in our incident was outweighed by external measures (revenue and brand damage). A cloud outage and a subsequent dip in digital experience can have a massive impact on revenue, averaging $38,855 per hour of outage. During high impact or busier times, this number can be much more.

    DevNetOps Nirvana

    The adage of bringing everyone to the table early proves to be valuable. Each team in an organization can have a strategic impact and improve architecture in multiple parts of the stack. Different teams have different sets of skills and can have mutual respect for the vertical skills of each of the teams. In the end, a transparent and positive relationship between network and application teams will help with network remediation and outage prevention. And, the closer the teams can work together and the more they can communicate, the more their investments and efforts can contribute to the business bottom line and quicker remediation long term.

    — Ravi Lachhman

  • DevOps Isn’t Getting Cloud Security Right (and What Can Done About It)

    DevOps Isn’t Getting Cloud Security Right (and What Can Done About It)

    The song title of Billy Joel’s 1977 hit, “Get it Right the First Time” aptly describes what DevOps teams need to take to heart when making the jump to the public cloud. But in most cases, security is lacking in the pipeline when making the shift—and a change in DevOps culture and mindset is necessary to “get it right the first time” for cloud security, according to McKinsey analysts.

    It seems commonsensical to make sure adequate security protocols are extended to the DevOps pipeline before migrating to the public cloud. But in practice, statistics tell a different story.

    According to analyst firm McKinsey, a full 78 percent of more than 100 firms recently surveyed are not reconfiguring their security tools when migrating to the cloud. Similarly, most security standards in place among the enterprises listed in the survey are non-sustainable for cloud networks.

    “There are, of course, many security configurations for a public cloud for DevOps that need to be in place,” said James Kaplan, a McKinsey partner and cybersecurity lead. “Creating your server image in a public cloud can create many vulnerabilities [compared to in-house data centers].”

    Security improvements will obviously need to be made on an industrywide scale in the near future as more companies embrace the cloud for DevOps. Currently, only 10 percent enterprises McKinsey surveyed have ported more than 40 percent of their workloads to the cloud. Within three years, that percentage is expected to jump, as 80 percent plan to migrate more than 40 percent of their workloads to a public cloud platform.

    Culture Lag

    At issue is how DevOps teams’ cloud projects very often remain straddled by security practices dating back to the 1990s.

    “Most DevOps security is still with a ticket-based model,” Kaplan said. “They just called Joe in security and asked him to do x, y and z.”

    Porting legacy infrastructure and operations to the cloud can also be problematic. A startup, for example, might make the jump more easily than a 30-year-old Fortune 500 company with data centers located in the North America, Europe and Asia. “It’s one thing if you’re building greenfield, such as creating a video streaming service from the ground up,” Kaplan said. “It’s much tougher to go to an agile DevOps cloud platform world if you are dealing with 30 years of legacy applications, eight mergers and all that type of good stuff.”

    The Big Meld

    Cloud servers can be fired up and ready to use in minutes. But it can take weeks before standalone security departments catch up. This might mean waiting on a security team to manually complete vulnerability scanning, penetration tests and firewall installation before cloud deployment. Or, in the worst case, these steps remain incomplete, leading to relatively easy-to-exploit vulnerabilities on the public cloud.

    “DevOps cloud security requires particularly high levels of sophistication and capabilities, as well as agility,” Kaplan said. “There has also traditionally been less focus on speed.”

    The solution is to converge security with the DevOps from the outset ahead of cloud development. In this way, security controls, reviews and deployment are integrated with DevOps, to create DevSecOps for the cloud migration process. This heavily involves developing APIs for security automation, Kaplan said.

    Implementing DevSecOps also requires investments in training and culture. This consists of ensuring DevOps teams receive additional training  to ensure security is integrated into every step of development work and for developers to maintain security support once the switch to the cloud is complete, Kaplan said. Training ensures DevOps team members have working knowledge of security APIs and orchestration technologies, and are eventually able to create new security APIs for DevSecOps, Kaplan said.

    The end result is that cloud deployment remains agile, as it should be. Cloud migration is done more swiftly and efficiently compared to traditional security models, once the DevOps team has acquired the security knowledge it needs, Kaplan said.

    “DevOps teams really need to expand their purview to think more about security. They must create more rigor around the security services and how the developer journeys should involve security,” Kaplan said. “And they then need to think about how to incorporate those security APIs into the application scripts for infrastructure APIs, capacity scaling and what have you.”

    — B. Cameron Gain

  • VMware Adds More DevOps Depth to Cloud Strategy

    VMware Adds More DevOps Depth to Cloud Strategy

    VMware today added a bevy of services to its portfolio that make managing instances of its software running in a public cloud easier.

    Additions to the VMware cloud portfolio include VMware Hybrid Cloud Extension Service for Private Cloud, a software-as-a-service (SaaS) offering that enables IT organizations to employ a self-service portal to move workloads between public clouds in a way the minimizes downtime.

    Other additions include the ability to apply both the Wavefront by VMware monitoring service and VMware Cost Insight Service against VMware Cloud on AWS, in addition to other public clouds. VMware has also added 45 integrations to expand the number of processes this SaaS application can monitor.

    Finally, VMware has added a VMware Log Intelligence Service to provides access to real-time analytics of log data.

    The arrival of these services coincides with the availability of VMware Cloud for AWS in the London region of AWS, which will be followed by the VMware stack becoming available in the Frankfurt, Germany, region of the AWS cloud.

    Chris Wolf, vice president and CTO of global field and industry for VMware, said that as more organizations look to move out of data centers they operate, they are asking VMware to extend the mobility capabilities the company developed for its hypervisor into the public cloud. In fact, many of them are now moving to deploy VMware software across multiple clouds, he said.

    Naturally, that expansion of VMware beyond the local data center is creating a raft of DevOps challenges, many of which the company is trying to address by making available a series of SaaS applications under the moniker VMware Cloud Services.

    VMware is making a concerted effort to expand its DevOps portfolio at a time when many developers have shifted application workloads onto public clouds based on virtual machines other than VMware. Obviously, VMware has developed many partnerships with cloud service providers. But the number of VMware hypervisors running in public clouds remains a relatively small percentage of the total number of hypervisors. VMware is hoping that will change, however, once developers discover VMware hypervisors on public clouds, which in many cases can reduce the headaches associated with trying to manage hybrid clouds based on disparate hypervisors. As part of that overall strategy, VMware also has been aggressively embracing emerging cloud-native platforms such as Kubernetes that can run anywhere.

    Most enterprise IT organizations are still learning to walk when it comes to hybrid cloud computing. The most common use case is still disaster recovery. Most IT organizations are a long way from dynamically moving workloads between clouds. Most of the use cases for moving workloads initially will be confined to simply lifting a workload from an on-premises environment. But as IT organizations also gain more experience, soon there may be as many workloads moving between clouds as there are workloads moving between servers in a local data center.

    — Mike Vizard

  • Case Study: DevOps Key to AWS Cloud Migration

    Case Study: DevOps Key to AWS Cloud Migration

    DevOps practices halved the time it took this SaaS company to migrate to AWS

    Australia-based Whispir provides communications applications as a service. One of its goals is to get products into users’ hands as soon as possible, and that requires development, operations and product staff working together, said Jonathan Swift, head of product and operations.

    Whispir formerly operated from a co-location data center, a model that continued as it expanded internationally. But the time and effort needed to run the systems was not cost-effective for an organization of its size, so the company switched largely to AWS. Governance issues faced by some clients force the company to still run its own systems in certain geographical locations.

    In the process, “[We] learned a lot about our app and how to make it cloud-ready,” said Swift. “Cloud isn’t free, cloud isn’t cheap” and is potentially more expensive than placing servers in co-lo data centers if the software isn’t built correctly.

    For example, it makes sense to make the most use of a physical server as it is largely a fixed cost. But in the cloud, the fewer resources you use, the less it costs.

    Crises Lead to Spikes

    On the other hand, a lot of Whispir’s business is crisis-focused (eg, Caltex uses Whispir at its Kurnell oil refinery in Sydney to send emergency messages), and it takes a certain amount of time to spin up additional instances to deal with spikes in message volumes. “That was a real challenge,” said Swift.

    Those crisis-communications clients expect Whispir to have sufficient resources to meet sudden demand. That’s an expensive proposition with co-located hardware, whereas Ccloud resources can be scaled up quickly, though not instantaneously.

    Seasonality has to be taken into account. One relatively predictable driver of crisis messaging is Australia’s bushfire season. Unfortunately, this overlaps Christmas, which is a peak time for marketing messages. So Whispir plans to run at around 200 percent of its normal baseline capacity during the fire season. This is easily done with cloud infrastructure, which Swift describes as “a beautiful thing” in this context.

    Some organizations are scared by multi-tenant environments. The ability to stand up dedicated instances on AWS is a great feature that Whispir can offer, especially to its clients in the financial sector.

    The company’s DevOps practices had been focused on the co-location environment, so even though kanban still “works very well for us” after moving to AWS, some previous choices had to be revisited. In particular, new monitoring tools were needed to cope with the elasticity inherent in the cloud and because the focus expanded from the hardware to the whole stack.

    Selected tools included the Zabbix open-source monitoring platform and Sumo Logic for trend analysis.

    Earlier this year, Whispir moved all of its Australian customers into a “pure AWS” environment. There are some great tools for migration, Swift said, “but it’s not as straightforward or seamless as you might think.” He recommends doubling any initial estimate of the time required for such projects.

    There were some nasty surprises: The first month’s bill from AWS was “huge” and it took a couple of months to determine the cause, even with the aid of AWS’s “really beneficial” premium support service.

    Security requirements in various jurisdictions had to be taken into consideration, with updates to various procedures to ensure the new cloud environment was compliant. For instance, the procedures required when a physical system was decommissioned were well-understood, but how could they be translated into a cloud environment? One specific example is that an in-house storage device can be destroyed when it reaches the end of its life, but in the cloud it’s largely a matter of relying on the provider’s controls.

    Avoid Lock-In

    But Swift warned that “the challenge is to not lock yourself in” to a particular cloud provider. For Whispir, which operates a mixed cloud/on-premises environment (no cloud provider offers the right onshore facilities in New Zealand and the hardware that Whispir has installed in Singapore is too new to be replaced by IaaS), that is a requirement rather than a strategic choice.

    Whispir would have avoided AWS-specific features that would make the platform easier to use and more difficult to get out of, despite the fact that using provider-specific toolsets can yield significant cost savings.

    Whispir is keeping an eye on Azure—”great tools, great partnerships, great ecosystems,” observed Swift—and may migrate to Microsoft’s cloud in around 18 months.

    A particular advantage is that Azure Stack can be run on private hardware, which could be useful if Whispir decided to enter a market where Microsoft Azure wasn’t present.

    Either way, the intention is to go 100 percent cloud to the extent that is possible.

    ‘Infrastructure as Code’ Means More Power for Ops

    The move to the cloud has made Whispir’s operations people more powerful, he observed. The infrastructure-as-code concept allows for rapid deployment, enables operations to contribute more to product development, reduces risk and enables the company to be more responsive to the market.

    Furthermore, dealing with on-premises hardware still very much a waterfall situation, thanks to the delays and dependencies involved. But moving to IaaS means infrastructure issues sit comfortably within the kanban model.

    IaaS also means that various network-related tasks simply disappear, but other issues arise in their place. For example, some external services used by Whispir (eg, SMS gateways) require the use of VPN connections from a known IP address, but with cloud infrastructure the best you may be able to do is specify a range of addresses. Whispir’s short-term fix was to route such connections through a specific server in the co-lo center, and in the longer term it found providers with less-restrictive arrangements.

    Moving from co-lo to cloud would have been “incredibly hard” without DevOps, Swift said. It would have taken at least twice as long, he noted, and the company wouldn’t be able to run the mixed environment as well as it does.

    “There’s no way we would have been so successful” without DevOps, he said.

    — Stephen Withers

  • IBM Adds Cloud Foundry PaaS to Cloud Private

    IBM Adds Cloud Foundry PaaS to Cloud Private

    As IT organizations are being asked to provision and support an ever-growing number of platforms, they increasingly require an approach that abstracts away as much of that complexity as possible. With that goal in mind, IBM created IBM Cloud Private, an on-premises platform that automates the deployment of a private cloud based on Kubernetes container orchestration software. This week, IBM announced it is extending the reach of IBM Cloud Private to include support for the open-source Cloud Foundry platform-as-a-service (PaaS) environment.

    IBM Cloud Private is designed to be deployed on top of virtual or physical servers, including instances of hyperconverged infrastructure based on software from Nutanix. Michael Elder, an IBM distinguished engineer for IBM Cloud Private, says IBM has developed tools for automating the provisioning of not only underlying infrastructure, but also containerized and legacy versions of IBM middleware such as IBM DB2 databases, the Open Liberty Edition of WebSphere application server and IBM Microservice Builder. In addition, Elder says IBM Cloud Private can be employed to deploy a variety of open-source databases, including MongoDB and PostgreSQL.

    Also available for an additional fee with IBM Cloud Private are a plethora of DevOps tools including APM, Netcool, UrbanCode, Cloud Brokerage, Jenkins, Prometheus, Grafana and ElasticSearch. Regardless of the platform employed, IT organizations should be able to apply a consistent set of DevOps processes to manage either Kubernetes or Cloud Foundry, Elder says.

    IBM is also addressing security issues by including Security Vulnerability Advisor to scan containers and an ability to encrypt all data in flight.

    Elder says many IT organizations will deploy 12-factor applications on a PaaS environment alongside containerized applications running on Kubernetes. IBM Cloud Private is designed to provide a common management framework for managing both environments. However, IBM is not extending that management framework to the IBM public cloud. Elder says the company determined that attempting to create a common management framework for both would result in too much complexity, especially for administrators of public cloud services that require a much more streamlined user interface experience.

    Most IT organizations could, of course, craft their own private cloud. IBM Cloud Private simplifies that process by providing all the pieces required under a common management framework. Rather than spending months putting those elements together, IBM is making a case for an integrated platform that enables IT organizations to concentrate more of their efforts on deploying applications instead of managing infrastructure.

    The debate occurring across the IT industry now is to what degree will container-as-a-service (CaaS) environments based on Kubernetes supplant traditional PaaS environments. IBM, as a major supporter of both Kubernetes and Cloud Foundry, is clearly betting there will be room for both. The Cloud Foundry Foundation recently furthered that goal by adding support for a Kubernetes runtime.

    Thus far, however, the complexity of standing up a PaaS environment has limited the adoption of Cloud Foundry to Fortune 1000-class IT organizations or cloud service providers. Meanwhile, CaaS environments based on Kubernetes increasingly are being stood up by developers and then handed over to IT operations to manage. Given that disparity in deployment models, it may not be too long before the number of CaaS environments simply overwhelm the number of instances of traditional PaaS environments.

    — Mike Vizard

  • Research Firms Agree, Cloud Growth Continues to Soar

    Research Firms Agree, Cloud Growth Continues to Soar

    Investment in all the various types of cloud computing has gone gangbusters in recent years, and for good reason: Cloud computing helps to cut cost, shorting time to market, provides unrivaled scale and reliability.

    This will only continue as enterprises continue to divest from on-premises infrastructure over the long term while seeking to scale the tremendous data generated by AI, IoT, machine learning and digital transformation. It looks as though for the few years ahead, it’s nothing but tailwinds for cloud investment and growth.

    According to research firm Gartner Inc., the software-as-a-service (SaaS) market reached $48.2 billion in 2016, growing more rapidly than the firm previously projected—although Gartner didn’t say in the release what its previous revenue prediction had been.

    Currently, Gartner expects the total worldwide public cloud services market to grow 18.5 percent in 2017 to reach $260.2 billion. That’s up considerably from $219.6 billion in 2016.

    According to Gartner, SaaS revenue is expected to grow 21 percent this year to reach $58.6 billion. “The acceleration in SaaS adoption can be explained by providers delivering nearly all application functional extensions and add-ons as a service. This appeals to users because SaaS solutions are engineered to be more purpose-built and are delivering better business outcomes than traditional software is,” Gartner writes.

    The highest-revenue growth will come from cloud system infrastructure-as-a-service, which Gartner says is projected to grow 36.6 percent in 2017 and hit $34.7 billion.

    Cloud Mojo Keeps on Rising

    Cloud is only going to continue to take a greater share of overall spend in the years ahead. According to Sid Nag, research director at Gartner, last year about 17 percent of the total market for infrastructure, middleware, application and business process services shifted to cloud, and by 2021, that will rise to 28 percent.

    Those looking at trends outside of cloud won’t be surprised to see cloud growing so much greater than that of the overall global economy. According to research firm IDC, worldwide information and communications technology (ICT) is set to rise over the next five years, fueled by IoT, robotics, augmented and virtual reality, cognitive computing and artificial intelligence.

    Total ICT spending on traditional IT, telecom services and new technologies will grow to $5.6 trillion by 2021 from $4.3 trillion in 2016, representing a compound annual growth rate of 6 percent in constant currency terms. Annual growth will accelerate through the forecast period, to 6.5 percent in 2021 from 5.5 percent in 2017, as new technologies account for a growing proportion of the overall market, IDC writes.

    Of course, all of this will increase investment in security and, according to a recent report from Markets and Markets, the cloud security marketing, including identity, data leak prevention, intrusion detection/prevention systems, encryption and security information and event management systems. According to Markets and Markets, the cloud security market will grow to $12.73 billion by 2022, up from only $4.09 billion in 2017.

    — George V. Hulme