devopsy.kinsta.cloud

Category: DevOps Chats

  • DevOps Pro Tips on Cloud Management

    DevOps Pro Tips on Cloud Management

    In this DevOps Chat we are joined by Andy Richman of Park My Cloud and Samir Mehra of Cloud Health Technologies. Both of these cloud experts give sound advice on how you can best and most efficiently manage your cloud infrastructure, whether it is on AWS, Azure, Google or anywhere else. If you are interested in optimizing your cloud deployments and cloud management in general (who isn’t?), then have a listen.

    As usual, the streaming audio of our conversation is immediately below, followed by the transcript of our conversation so you can read along. Enjoy!

    BTW, we have recorded two other podcasts with Park My Cloud. You can listen to them here and here.

    Alan Shimel: Hey, everyone, this is Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another episode of DevOps Chat. We’ve got a great chat lined up today. We’re gonna be talking a little bit about managing your cloud infrastructure, cost, and how much do you really know about or how much insight do you really have into your cloud infrastructure.

    We’ve got two great guests lined up to join us and help us today. First, I have Andy Richman. Andy is with ParkMyCloud, and Andy, welcome to DevOps Chat.

    Andy Richman: Thank you, Alan. Looking forward to this.

    Shimel: Okay, and you know what, Andy, before we introduce Samir, why don’t we just, for anyone who may not be familiar with ParkMyCloud, why don’t you give us a little background on what your role is there?

    Richman: Sure. So, I’m the Product Manager over here at ParkMyCloud, and I think about all things automation and saving money in the cloud. The way we describe it is cost optimization. We launched the company in September, 2015, so we’ve been around just over a couple of years now, and we work with all manner of companies, from large Fortune 500 type companies, right, all the way through to smaller startups born in the cloud, and we’re helping all of those people effectively manage their infrastructure and get the most bang for the buck out of it.

    Shimel: Excellent. Okay, we’ll come back to you in a second, Andy, but first let us introduce Samir Mehra—and I messed that up Samir, I apologize. Why don’t you correct me?

    Samir Mehra: It’s Samir Mehra.

    Shimel: Oh, I wasn’t too bad, then. And Samir, why don’t you tell us your company and background

    Mehra: Yeah, so I’m Director of Product Management here at CloudHealth Technologies. CloudHealth Technologies is a SaaS platform that provides visibility into all your cloud environments, so customers can really look at areas for optimization, and this can include cost, usage, configuration, security, and performance. And then we also have ways that you can automatically remediate some of these optimizations with our policy engine.

    Shimel: Got it. So, Andy, Samir—not to boil it down to dollars and cents, but let’s boil it down to dollars and cents. Is cost sort of the biggest factor in what companies are looking at in terms of managing their cloud infrastructure? I mean, you guys do similar but different tasks for organizations using cloud. What’s your opinion? Is cost the number one priority there? Andy, let’s go with you first.

    Richman: Sure. So, it depend where you are in terms of your company’s cloud evolution. So, if you’re moving from an on prem data center type operation into the cloud, then no, it probably isn’t. You’re thinking about migration, getting workloads moved over, you know, getting performance in the cloud.

    Once you’re in the cloud, you know, obviously, you’re looking to optimize the performance, you know, you’re probably looking to scale up what you’re doing there. And again, cost is moving up, but it probably isn’t number one or number two. But once you’ve been in the cloud for a couple of years and, you know, month on month, you’ve seen those cloud bills climbing, then typically, yeah, it begins to get a lot of attention. And, for most of the companies we’re working with, they’re all at different stages, but probably the majority are pretty focused, and it probably is in that top three, top five for the company.

    Shimel: Excellent. Samir, how about you?

    Mehra: So, I look at it in a different way. It depends on what type of company you are. If you’re a born in the cloud company, you know about how cloud is supposed to be managed. I think cost is not the important thing, it’s the efficiency with which you want to execute things and how quickly you want to get up with your production infrastructure or your test dev infrastructure. And really, I think those companies are very apt at managing some of their costs and making sure their cost is optimized, because they only have been dealing with opex.

    But, when it comes to enterprise companies, I feel some of them, even though they’ve migrated a bunch to the cloud, they still think of cloud as something which is capex. They generally overprovision when they don’t need to overprovision. You know, cost is secondary for them, too, but they have so much of extra spend in what you call cloud sprawl that they really need somebody to help them visualize it and give them ways to optimize that cost.

    Shimel: Yep. So, you know, I always like to say that the costs are almost symptomatic of something else, though, right? Especially when we talk about things like cloud, guys. You know, the costs are relatively set. You can look up on a price chart and see, you know, what a gigabit of storage is, how much CPU time costs or what have you.

    I think when we talk about the cost of cloud and stuff like that, really, the underlying thing is a question of control. Do we really have insight into what we’re using and how we’re using it, right? And, because we don’t, it translates into a cost we may not understand or expect or even know what the heck’s going on. But really, the underlying issue is control and more management of what this is.

    And both of your organizations kind of play a role in that. Why don’t you—and not to create a knock-down, drag out, but can you give me a contrast, Samir, let’s say, in what your company does versus what Andy and the folks over at ParkMyCloud do?

    Mehra: So, I’ll start with the fact that, you know, from what we’ve seen our customers spend, IAS and virtual machines or EC2 instances are really still the biggest spend. People haven’t really transitioned into a lot of TAS services, though TAS services definitely are picking up.

    The way we do it is, you know, to help prevent cloud sprawl, we do have a policy engine where you can set up a policy which tells you if there’s an instances that is started that is more expensive than what you want an individual to set up. So, if they go spin up a really huge instances, you can get alerted, you can actually set up a policy which turns that instance down, even before it gets to run. And the other thing is, you know, the way we prevent sprawl also is, we give you—we, again, use a policy engine and tell you, “You know what? There’s somebody who spun up an instance,” and they have them tag those instances. So, there’s no way to know who’s actually spun that instance up, and if you don’t know who spun that up, you can actually now go and ask them, “Why did you spin it up, and what’s the purpose of this instance?” if it is an expensive instance that they’ve spun up.

    So, we definitely rely on our policy engine a lot, on making sure we have these governance policies implemented to help prevent cloud sprawl from the beginning.

    Shimel: Excellent. Andy, your take on it?

    Richman: Yeah, sure, so we do have a slightly different viewpoint on this. So, when that cloud bill drops in at the end of the month, typically about 70 percent of it is being spent on compute, just as Samir said. If you actually look within that compute, what you’ll quickly discover is, about half of it is for production systems—these are things that typically need to run 24/7—but the other half is your dev, test, QA, data analytics type environments. And there’s at least the potential to turn those instances off when they’re not being used, and we’re really focused on that half of the infrastructure.

    So, what we do is, we give DevOps, IT Ops, you know, engineers an easier way to get those instances turned off. So, in a week, there’s 168 hours, so if you didn’t know, that’s about 730 hours in a month. And even if your team’s working 60 hours a week, which is a pretty hard working team, it still means you can save 65 percent by just turning things off when they’re not being used.

    And so, what we’ve really focused on is the automation piece of this. So, to ensure that when things aren’t being used and there’s no value being added, they’re just simply turned off. And we have semi-similar kind of policy engines that automate this, but an ability, effectively, to create a simple schedule, have it applied across your various different teams, and throwing off some pretty significant savings pretty quickly.

    Shimel: Got it. So, automation is something that, in DevOps, we understand very well, right? It’s key to what a lot of what DevOps is about. And so, the idea of turning down, shutting off, closing up unused instances, unused resources—I mean, just, it’s a no-brainer, you would think, right?

    But why do you think more people or more organizations aren’t doing that kind of thing? Andy, why don’t you go first, if you’d like?

    Richman: Oh, sure. I mean, some of them are doing it, and they’re writing on scripts, but they tend not to do it in a particularly systematic manner. And when you actually talk to the people—I spend quite a lot of time talking to our customers—a lot of it is because, you know, they don’t want to take away control from the teams who are out there, you know, delivering code, delivering value to the company. And often, they just don’t have the insights in terms of how those various instances, VMs are being utilized. And it really requires buy-in from the teams themselves.

    And so, what we’ve seen in the companies that are doing this well is that they create a lot of buy-in. The thing that people quickly realize is that oftentimes it’s not just about stripping costs out and spending them elsewhere in the enterprise, this is just about efficiency, and that money often will get recycled back into other R&D projects, you know, other things that people wanna be doing.

    So, I think the more progressive organizations get it. I think what they’re able to do with a tool like ours is to do it in a scalable way. So, you’ve got multiple teams, lots of individuals, lots of different workloads. We give them a way to do that very, very quickly, simply. It adds a couple of minutes of work to their week and they can kinda schedule it in place and reap the benefits.

    The other thing that’s awesome is visibility. This may be something that Samir can talk about a bit more. It’s, once you begin to give back the DevOps teams, you know, the actual visibility on the savings they’re getting, it becomes a kind of a, somewhat of a self-fulfilling prophecy or a kinda closed loop. They see the savings they’re getting, they do more of it; they see more savings, they do more of it—that kind of positive affirmation.

    Mehra: Yeah, actually, we—sorry, go ahead, Alan.

    Shimel: No, no. I was gonna say, Samir, what about you? [Laughter] I’m sorry.

    Mehra: No, that’s fine. So, we’ve seen, we use our product, our DevOps engineers within CloudHealth also use CloudHealth to manage the AWS environment that we have for production for our SaaS environment. And our DevOps engineer is definitely not a financial person, but like Andy said, when he sees those cost savings, his eyes do light up. He’s like, “I can really save this much money if I actually right size some of these instances that are being underutilized.” And he does, he’ll do, over the weekends, or when we can, do a right size of some of our instances to drop our costs down.

    So, we actually use our platform ourselves to do it, and our DevOp engineers are using it. And we do have plenty of customers doing similar things, and it’s not just right sizing a particular instance. You can also, you know, look at it from an automatic scaling group perspective. You know, you want to right size the automatic scaling group or the next automatic scaling group that you spin up. You want it to be right sized because you’re gonna have a cost savings associated with it.

    Richman: One of the things, Alan, that’s interesting is that people would often quote you the cost of their infrastructure or say, “This is an instance that’s 25 cents now, or 50 cents now” or something like that. What they haven’t done is the mental math—well, okay, what is that gonna mean in a month? And so, a 50 cents an hour instance is actually $365.00 a month. And if you can save 50, 60 percent of that, it’s a significant amount of money, particularly for customers who have tens, hundreds, or even thousands of instances.

    Mehra: Yeah, I completely agree. I think once you multiply the one instance which costs 50 cents into 100 and 200, that’s where you actually see the huge amount of cost savings that you can have.

    Shimel: Excellent. Good. Guys, as I think I mentioned to you when we got on here, time goes really quick, and we only have time for another question or two here, and I want to focus in on a couple of things. Obviously, looking at both of your company services is a great way to get started along this, getting a handle on what you have in the cloud. But if you had to give, let’s say, a top three things that our listeners should do, beyond just signing up—signing up with your company is obvious. But top three things together between the two of you—what would they be?

    Richman: Samir, do you want to go first, or…

    Mehra: Yeah, I would say, make sure you have visibility into your spend so that you can make sure that you’re not spending where you don’t need to spend.

    Shimel: Excellent.

    Mehra: So, cost visibility, for sure.

    Shimel: Andy, how about you?

    Richman: Yeah, so for us, it’s really—you know, focus where the largest amount of potential savings are. And, based on our experience, that’s in your non-production workloads. So, when we focus on that and then start to look, you know, talk to your QA team, talk to your development team, talk to people running analytics workloads, and really understand, you know, how many hours a week those instances truly need to be running. I mean, even if you’re conservative and you only turn things off in the really small hours of the night, the savings really quickly mount up. And then, to really start to do it systematically, kinda team by team and rolling it across, rather than just more of an ad hoc kind of effort.

    Mehra: Yeah, and I can add a third one along with that is, you can also do it on your production infrastructure, and there’s two ways to really do it. One is, you know, right size your production infrastructure and you can do it during a down time where you can right size during a down time, but also look at reservations, because reservations is a huge cost saver compared to running on demand.

    Richman: Yeah, absolutely. I mean, one of the things that we have recently added to the platform and we’re gonna be adding a lot more in the next couple of months is bringing in the utilization data. So, you know, looking at your CPU, your network, your disk usage and really using that to drive these changes.

    We’ve recently introduced these kind of simple heat maps where people can come in, look at the platform, and see the hours when their instances are actually being used. Rather than just asking people when they’re using, you know, what’s being used when, you can actually see the actual usage and, post-Christmas, we’re gonna be moving into the right sizing space in a similar way using similar kinds of metrics.

    So, there’s definitely a lot of tools out there that can help you do this in a more informed way.

    Shimel: Yep. Agreed. Well, guys, we’re just about out of time, here. Samir, Andy—thanks so much for being our guests on this episode of DevOps Chat. You know what I didn’t realize—Andy, for people who want to get more information on ParkMyCloud, is it ParkMyCloud.com?

    Richman: Yep, that would be ParkMyCloud.com.

    Shimel: Yep, and Samir, URL for your company?

    Mehra: Yeah, it’s www.CloudHealthTech.com.

    Shimel: So, that’s Cloud Health Deck, D-E-C-K.

    Mehra: T-E-C-H.

    Shimel: Okay. Got it. Alright, T-E-C-H. We’ll put it in the show notes as well, in case anyone is not clear.

    Andy, Samir, thanks for being our guests. This is Alan Shimel for devopsy.kinsta.cloud, and we’ll see you on another DevOps Chat very soon, everyone. Thanks for listening. Have a great day.

    — Alan Shimel

  • DevOps Chat: IBM’s Cloud Private with Mike Kaczmarski, CTO DevOps and Service Management

    DevOps Chat: IBM’s Cloud Private with Mike Kaczmarski, CTO DevOps and Service Management

    In this DevOps Chat we discuss IBM’s Cloud Private. Cloud Private is the new offering from IBM that encompasses much of the different cloud offerings the company has developed over the years. Our guest on this DevOps Chat is Michael Kaczmarksi, an IBM Fellow and CTO of DevOps and Service Management. In his distinguished career at IBM, he has obviously seen technologies come and go. Michael gives us an inside look at IBM’s Cloud Private offering and how it relates to DevOps, service management, hybrid cloud and so many other contemporary technologies in the market today.

    As usual the streaming audio of our discussion follows immediately below. Below that is the transcript of our conversation to follow along.

    Alan Shimel: Hey, everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. We’ve got a very interesting chat lined up for today, with a great guest. Let me introduce him. We have today Mr. Mike Kaczmarski of IBM. Mike, welcome to DevOps Chat.

    Mike Kaczmarski: Thank you, Alan. Glad to be here.

    Shimel: So, Mike, you have a CV résumé full of accomplishments, and I don’t mean to embarrass you, but why don’t you just share with our audience kinda what you’re doing now with IBM and a little bit of your background?

    Kaczmarski: Alright, I’d be glad to. Well, my title is IBM Fellow, CTO of DevOps and Service Management, in a unit inside IBM called Hybrid Management—Hybrid Cloud Management. I’ve been with IBM—well, in February, it’ll be 35 years.

    Did a lot of work in the storage portfolio—a lot of the Spectrum products, Spectrum Protects, Spectrum Control. I came into a business unit that was called Tivoli at the time, became very familiar with all of the service and systems management products, and then into products like Maximo and others that we acquired or built out organically. And then became involved in a lot of the cloud technology—IBM Cloud and now I’m working very hard on DevOps technology and service management technology in a new offering that we have out called IBM Cloud Private.

    Shimel: Excellent. Excellent, excellent, excellent. And we’re gonna try to get to as many of these as we can, Mike. But in order to do that, I think we need to lay down a little sort of baseline, if you will. And how I wanted to do it was, you know, let’s talk about really what I think is one of the primary drivers behind the whole hybrid cloud transition that we see, and that is that traditional applications—you know, everyone would just like to move everything to the cloud, perhaps, make it easy, and be done. But that’s not the way the world works, right?

    Kaczmarski: [Laughter] Right.

    Shimel: You know, customers are beginning that migration, but that migration will probably take—Mike, you said you were with IBM 35 years. I’ve been around awhile myself. We’ll be long gone by the time that migration is over, I suspect. [Laughter]

    You know, so in the meantime, customers are faced with sort of this hybrid period where, you know, some of the applications and some of the application infrastructure may still be on-prem, may still be maybe a private cloud on-prem, maybe on a public cloud, maybe on a private cloud in a data center. You know, truly, a multi-cloud and hybrid environment. Is that what you’re seeing at IBM?

    Kaczmarski: Certainly what I’m seeing. If you consider things like the importance of customer data, data sovereignty, customers’ concerns over security, proper connectivity—all those need to be addressed, and customers’ businesses are running on their IT, and they’re comfortable at different stages with different aspects of those things being on the cloud, whether public or dedicated, and you often find something, like private cloud is a wonderful way to get started on understanding what operational architecture programming changes and even organizational changes they need to make to best start to utilize the cloud. Whether they take all those capabilities to public or dedicated cloud or not, the private cloud helps them.

    Shimel: No doubt about it. And so, Mike, when we talk about—you know, they may be IBM customers. But when we talk about organizations transforming their traditional—and by traditional, I mean non-cloud, they’re not cloud native apps—and trying to migrate them to the cloud. You know, I think you highlighted some of the things that we run into, but anything else that I think listeners may be interested in or at least gotchas that they should be looking for in transforming these apps and trying to move them to the cloud?

    Kaczmarski: Yeah. Well, I think there’s three patterns that we see pervasively. The ones you hear a lot about are those customers who are building out new cloud services, new cloud native programming models, and extending their business into the cloud using some of these new microservice-oriented programming models—polyglot technologies—and perhaps connecting them to their enterprise from the cloud; hence, the hybrid sense through APIs and API management so they can take advantage of their back office systems and the data and expose other application pieces on the cloud.

    There’s a few other patterns. We see customers who have applications that would fairly easily be optimized by cloud technology, such as container technologies for rapid deployment, rapid scale out, resiliency. There are applications that can be, we call it, lifted and shifted fairly easily. And then there’s a whole area of applications that are built out in monolithic fashion that have very tight interdependencies that have been around for years and years, and customers are making decisions on what they do with those applications, and it kinda depends on how much those applications are changing, how critical they are to the business—I don’t wanna touch it, it’s not changed in years, it’s doing its job. That might be some of their criteria. In other hands, parts of those applications might be changing fairly rapidly. And if that’s the case, then they start looking at refactoring or factoring out the portion of application code that is changing fairly rapidly into, say, microservices and placing those in the cloud.

    So, I think there’s a number of approaches that they’re taking, and obviously, it’s a business issue. What makes sense for me to refactor? What am I gonna gain by the refactoring of these applications and putting them in the cloud? How am I gonna lower operational costs? How is DevOps gonna help me, and what kind of return am I gonna get from that versus continuing to do what I do or even leaving some of my monolithic applications alone?

    So, I think, you know, there’s a broad range of considerations that customers have when they look at moving their applications to the cloud, and hopefully I’ve kind of listed a few, there.

    Shimel: Absolutely. So, Mike, you know, this is all—it’s all happening, and I think we’ve kinda described it well. But now, so IBM has recently come out with this Cloud Private, as they’re calling it, and you know, frankly, I think there might be a little misunderstanding in the market where, hey, you’re talking about hybrid cloud, but now, why are you doing Cloud Private? Right? Is it private cloud, is it hybrid cloud? Cloud Private—where does that fit in? Can you help clear it up for our listeners?

    Kaczmarski: Yeah, sure. IBM Cloud Private is an on premise cloud offering based on Kubernetes technology, optimized by some of our systems group. It does run and allow you to deploy private clouds, what I’ll call on premise clouds, on Intel, on Power, on zLinux, so you can take advantage of the systems you might already have deployed, but it has full connectivity to any services in—cloud services that are running in the cloud public or dedicated cloud. It’s a way for customers to start to either write new microservices or write new capabilities or refactor, as I mentioned, some of their applications into a cloud that is running on premise, allow them to experiment without concerns of data privacy, data governance, data sovereignty, and allow them to figure out what’s the best approach that they want to take for refactoring their applications or moving them to the cloud.

    It is Kubernetes-based. It’s based on the same technology that runs on our public cloud and what we call the container services. So, much of the workloads that can run in the private cloud if the customer chooses can be transparently moved to the public cloud or a dedicated cloud for them. The Kubernetes technology overall, the Docker containerization—everything’s running in Docker. They’re fairly pervasive in the industry, so it does give the customer a choice. I think it gives them a foundation that they can start with. They can move workloads at their most comfortable, keeping them on-prem, and then they can start to extend those workloads to the public or dedicated clouds as well.

    You know, another big difference with IBM Cloud Private is, we’re moving our middleware, our traditional offerings like the WebSphere and message queue and IIB and DB2 and the data science experience—traditional middleware that customers have running on-prem that they’re writing their applications against are now available in IBM Cloud Private in containerized versions, the same service, the same APIs, a lot of the same capabilities. And so, they can actually take advantage of the same middleware that they’ve been running with, now containerized in clouds so that they can take advantage of simpler deployment, AB deployment, a simpler upgrade, resiliency, scale out, and so on.

    So, that’s what I think we bring to the Cloud Private arena is the ability for customers to leverage the investment they have in our middleware and middleware services now in a cloud deliverable. And that also makes it easier for them to move their applications into a cloud architecture, whether it’s on-prem in Cloud Private or in public in IBM cloud or a dedicated cloud, an IBM cloud.

    Shimel: You know what, Mike, that was fantastic, and it kinda made a lightbulb go off for me, here. What really, what Cloud Private is offering customers, or anyone—you know, eventually, I guess, their customers—but it’s portability. It is giving you a cloud ready infrastructure, whether that cloud be on prem, at an IBM data center, maybe in a public private type of situation, hybrid situation or not. And not only is it allowing that infrastructure through containers and Kubernetes to be portable like that, you’re now portable-izing, if that’s a word, [Laughter] that whole layer of middleware that so many have been using for so long—

    Kaczmarski: Yeah.

    Shimel: – and making that also portable to whatever or however and in what combination you want to post your applications going forward.

    Kaczmarski: Yes, that’s sort of a—

    Shimel: And that’s really powerful.

    Kaczmarski: – yeah, that’s a good description, yes.

    Shimel: Yep. And, you know, I think, you know, Mike, it really goes to the heart of what we’re seeing in technology today, which is the—and a lot of it is via Kubernetes; well, microservices and containerization, let’s call it that.

    Kaczmarski: Yes.

    Shimel: It’s fundamentally shifting the IT infrastructure or the preferred IT infrastructure in a way that we haven’t seen maybe since when we moved from client server, or maybe when virtualization and hypervisors first became sort of dominant or started really asserting influence in the market. This is really a shift of that magnitude. Do you agree?

    Kaczmarski: Yes, I do, and actually, those are two perfect storms that kinda come together, the virtualization and the client server. I think containerization is an interesting supporting feature of microservices. It really allows a full stack deployment, and actually changes some of the roles and responsibilities of what developers have traditionally done.

    Shimel: Absolutely on that. So, this is DevOps Chat, though. Let’s talk a little bit about DevOps. I always like to tell people that—hey, DevOps is an umbrella term today. It’s more than just the breaking down of the silos between dev and ops, and there’s obviously a cultural aspect to it. But, you know, the thing about DevOps is, it’s about how things get done today. And so, when we talk about things like microservices and containerization and cloud, right, I don’t know if DevOps is a byproduct of those things or if DevOps is a precursor or an accelerator to those things, but they seem intrinsically linked.

    Kaczmarski: Yes.

    Shimel: What’s your view on that?

    Kaczmarski: Yeah, I think they are tightly linked. I think DevOps enables the kind of automation you really want to be able to understand how your applications are being developed, are integrated, how they’re performing, whether or not you’re ready to go into production with what you currently have, whether or not you’re really ready to support, operationally, the applications you already have—all of those things, as early in the life cycle as possible.

    And, you know, when you talk about containerization and microservices, as compared to monolithic applications, there’s a lot more moving parts. There’s many more moving parts in a microservice container organization of your code. You know, you have business function decomposed into various different microservices. It’s actually an organizational construct as well as an architectural construct. You have teams around those microservices building them. They all have API contracts with each other on their behavior, and now they’re deploying at different times into what should be a running system that’s providing a service. At every point along that line, you wanna understand. You wanna be able to herd all of these microservice teams that are together building out the application in a consistent way and always know where you stand in terms of the application being built.

    By the way, the same applies to monolithic apps, because they undergo change if they’re interacting with these cloud native apps and microservices. So, you need to be able to automate and make sure nobody breaks the builds, you need to be able to do continuous deployment of the microservices in, say, a test cluster or a cloud where they interact with each other. You want to vary the versions that are out there to understand if the API contracts have been violated. You want to have a real good idea of the performance and scale of the microservices that are out there—and there are so many of them in a traditional application now, manual tasks would never be able to keep up. So, automation through DevOps in terms of build, deploy, the manageability, the testing of the microservices is paramount in being able to scale a true microservice architecture to a product that has reliability and can be easily updated and can actually iterate very quickly to meet customers’ needs.

    Shimel: Absolutely. Mike, let me introduce another term into our term goulash today, and that’s service management, alright? So, service management is something that has certainly been on the IT agenda, right, on the IT menu for longer than the word DevOps has been used, right?

    Kaczmarski: Yes.

    Shimel: Service management is an important piece of it, right? And there’s been somewhat, frankly, of a disconnect on the nexus or connection of DevOps and service management. Are they compatible? Do they work together? Is there a continuum with them?

    What’s your view? I mean, you have both under your portfolio, here. What’s your view on that?

    Kaczmarski: Well, I think that integration of DevOps into Ops, if you will, relative to DevOps I just talked about in terms of automating, you know, build, deployment, test—integration into Ops is more about making sure you can monitor the application, making sure that if there are outages, you have procedures, run books that can quickly bring things back, supporting things like site reliability engineers, understanding who’s going to be notified if there’s a problem with a particular service—all of those kinds of things push back operational characteristics on the DevOps and even the development team.

    You know, as our teams were building out IBM Cloud, a lot of the services in IBM Cloud are managed by, developed by individual squads, so a logging service or a particular database service or whatever we make available in that cloud, there’s a squad behind it. Obviously, they develop it, they go through regular DevOps deployment and testing, they determine when it goes into production a lot of times based on analytics from a lot of the automation that they have.

    When it goes into production, they’re immediately also responsible for the ongoing operation of those services. This is the development team. They’re now responsible for the ongoing operation of those services. Now, they have site reliability engineers as well, but if you take some development teams and make them responsible for the operational characteristics of their services, a lot of interesting things start to happen. You start seeing development not around the application itself, but also around, “Oh, hey, I have some Selenium scripts that I’m gonna use to automatically monitor response time of the application. This needs to get deployed with the code.” “Oh, I have some run books for the SREs or automated scripts that, should certain events show up, will automatically restart or scale out services appropriately.”

    You start seeing operational artifacts as part of the development of the particular microservices or applications that are being built. And of course, as that starts to occur, you start seeing standards for handoff of these kind of artifacts between DevOps and Operations teams so that not only are they deploying application, they’re deploying the management around that application and they’re ready to go when that application goes to production.

    So, I think that’s the biggest cultural change I’m seeing with DevOps going into Ops is really integrating the management capabilities and intelligence into the application and the awareness that now the developer and the DevOps team have in providing what is needed to properly manage the application and operations.

    Shimel: Mike, that was one of the best answers to that particular subject that I’ve heard. Thank you very much for that.

    Unfortunately, Mike, as I told you before we started, the time here goes really, really fast. I promised you we’d be done in 15 minutes and we’re on 20 minutes already, so I know we had a lot of other topics we wanted to cover. Maybe we’ll have you back on at another time and we can do a part two on this, but we’re about out of time right now.

    Thank you so much for being our guest on DevOps Chat today, Mike.

    Kaczmarski: Oh, thank you, Alan, I really enjoyed it.

    Shimel: Not a problem at all, and as I said, I’d love to have you back on here soon, and we can continue the discussion. But before we go, for anyone who wants to get more information on IBM Cloud Private, where can they go?

    Kaczmarski: Well, actually, the easiest thing you can do is Google IBM Cloud Private, and there’s a number of references out there, and tools. There’s cloud reference architectures, developers, communities. If you want to engage with some teaming with IBM on what’s the best way to move some of your applications to the cloud, there’s IBM Cloud Garage, and there’s some wonderful blogs out there where developers will take you through how they actually move some of the applications to the cloud. So, I would just Google IBM Cloud Private, and you should see, in the first two things that come up, some really good references.

    Shimel: Fantastic. Mike Kaczmarski, IBM Fellow and CTO for IBM Private Cloud, DevOps Service Management—thanks so much for being our guest on this episode of DevOps Chat. To everyone listening, hope you enjoyed it, and we’ll see you soon on another DevOps Chat. Have a great day.

    — Alan Shimel

  • DevOps Chat: Chef Habitat Project Continues to Mature

    DevOps Chat: Chef Habitat Project Continues to Mature

    When Chef Habitat was released a little more than two years ago (good devopsy.kinsta.cloud story on Habitat here and a good DevOps Chat on it here), many people in the DevOps community wondered where Chef was going with Habitat. It was a really ambitious vision, but could it meet its lofty goals?

    Habitat is now made up of many different packages that provide different functionalities. There is, of course, much more to Chef Habitat. This DevOps Chat will tell you all about it.

    Joining us on this DevOps Chat are Jaime Winsor, a co-author of Habitat, and Tasha Drew, senior product manager of Habitat, and Blake Irvin of SmartB, a Habitat user. Between the three of them, there is a ton of great information here. If you are interested in Chef Habitat or would like to find out more, this is a great place to start!

    As usual, the streaming audio of our conversation is below, followed by the transcript of our conversation. Enjoy!

    Audio: Chef Habitat

    Transcript: Chef Habitat

    [su_pullquote][su_custom_gallery source=”media: 118811″ target=”blank” width=”250″ height=”250″]Blake Irvin
    Blake has spent over a decade working in systems and software engineering automation, on platforms ranging from on-premise metal to containers-as-a-service, BSD and Solaris systems to embedded Linux sensor platforms. He lives and works in Berlin, Germany, where he’s Senior Operations Engineer at smartB, GmbH, an energy-efficiency and sustainability startup.[/su_pullquote]

    Alan Shimel: Hey, everyone, this is Alan Shimel from devopsy.kinsta.cloud here for DevOps Chat, and we’ve got a really great chat lined up for this episode. It is all about Chef Habitat, and we’ve got some really-we actually have three guests on the show today, so we’re gonna move it along to keep it moving.

    Let me first of all introduce my two Chef guests or from Chef. One is Tasha Drew, who’s a Product Manager for Chef Habitat, and one is Jamie Winsor, an engineer with Chef and one of the co-authors of Habitat. Jamie, Tasha-welcome to DevOps Chat.

    Tasha Drew: Thanks so much. Happy to be here.

    Shimel: Thank you.

    Jamie Winsor: Yeah, thanks for having us, man.

    Shimel: Hey, it’s my pleasure. And then, joining Jamie, Tasha, and myself today, all the way from Berlin, is Blake—and I hope I’m not gonna mess this up—Irvin. Blake, are you there?

    Blake Irvin: Yep.

    Shimel: Okay. Thanks. Hey, Blake, I appreciate you taking the time to join us. Blake, you’re with, is it BusyBee, is that what’s the name of the company?

    Irvin: It’s called smartB.

    Shimel: smartB.

    Irvin: Yeah, and the B essentially stands for business, because we are building, because we focus on providing energy efficiency analysis for businesses—small to very large businesses and the buildings that they operate.

    [su_pullquote align=”right”][su_custom_gallery source=”media: 118812,118813″ target=”blank” width=”200″ height=”200″]Tasha Drew
    Tasha Drew is the Product Manager for Habitat. She previously launched Chef Automate’s analytics capabilities, was Head of Product at Rentlytics, was responsible for product for Engine Yard Cloud, and spent several years in various data center endeavors on behalf of Lockheed Martin. Tasha holds an M.S. in Management from RPI and a B.S. in Computer Science from USC. When not in front of a computer, Tasha prefers to be cruising around Napa or at the beach with her dogs—all while having intensely deep thoughts about how to save application developers from tedious rework via the wonders of application automation with Habitat.
    Jamie Winsor
    Jamie Winsor is a lead engineer at Chef Software and the coauthor of Habitat, an open source project built upon distributed system protocol Butterfly to provide a self-healing, self-configuring, stack-agnostic, frictionless abstraction for running applications—regardless of their complexity—to software developers. Jamie has been a software engineer in the video game industry for 10 years, focusing on networked application servers on such titles as League of Legends, Lord of the Rings Online, and Dungeons and Dragons Online. One of Jamie’s responsibilities in his game development tenure was to bring what we today know as DevOps into the daily lives of the other developers on his team, which Jamie accomplished by building, evangelizing, and teaching methods to his peers. He draws on that experience today in building Habitat, as he helps enable all software developers, regardless of their experience, bring their ideas to life without investing in the details of operationalizing an application.[/su_pullquote]

    Shimel: Wow, that’s really good stuff. Jamie and Tasha, you both work for Chef, no need to introduce who Chef is to our DevOps audience, but Blake-welcome. And we have you on today because we’re gonna talk a little bit about Habitat, and Blake, if you don’t mind, I wanted to start with Tasha and Jamie and talk a little bit-just a little background. Maybe we have some folks on here who are not familiar with Habitat.

    So, Jamie, you’re a co-author. Why don’t you take that one, right? What’s Habitat, for people who may not be familiar?

    Winsor: Yeah, sure. So, the Habitat project is a number of pieces of software, and the overall goal is to, I hope, redefine what application automation means by focusing on the application instead of the infrastructure. In the past, we have been building computers in service to host our applications, and what Habitat does instead is it says, “Let’s focus on what the applications are that we need to run our application, what software we need down to a bare minimum, and then provision the machine around it,” and Habitat runs exactly the stuff that you need.

    It does so with a packaging system and a build system and then a process supervisor. The process supervisor runs the packages, the process supervisor communicates to other supervisors using a gossip protocol to share and spread information about what software they’re running, which allows us to enable things for free, just by packaging your software with Habitat, such as service discovery, leader—follower failover, rolling configuration updates, rolling software updates—and it’s all coordinated using the gossip ring.

    Finally, the piece that we built, the last piece of Habitat is simply called Builder, which we can also chat about today, which we just launched about a month ago. And Builder is a hosted Software as a Service build service that’s currently in preview mode. It’s free, and it builds your software that’s packaged with Habitat and hosts it as well in release channels, which allows you to have supervisors subscribe and then receive automatic updates using a complete CI/CD process of promotion between an unstable branch to a stable branch and so on. That’s basically it.

    Shimel: Yeah, no, more than basically-that was a great description and great stuff there, Jamie.

    Tasha, now, Habitat was released as an open source project by Chef, and it’s probably about two years ago now, right?

    Drew: Yeah, yeah.

    Shimel: And so, Tasha, tell us kinda what’s been the latest with Habitat.

    Drew: Sure. So, basically, we have been working nonstop since the initial reveal of Habitat about two years ago, just adding functionality and capability across both that Builder system and that process supervision system that Jamie’s been talking about.

    We’ve also been working on the deployment piece of the overall integration between those two which, in Habitat’s world, is highly agnostic. So, the idea is, use whatever tools you need to deploy those artifacts and supervisors that you’re using with Habitat. And so, we’ve been working with partners to have a really great cloud foundry ecosystem story, Kubernetes ecosystem story, AWS and Azure story, and then Terraform as well.

    So, the idea is, we have been making it easier and easier for people to come get started using Builder, and that’s why we’ve released Builder in free preview mode, it’s because we just want to have this very low friction way for people to get started with Habitat and start rebuilding all of their workloads, and then deploy using your favorite technologies and then start seeing the benefit of the supervisor in Builder, communicating between the two.

    Shimel: Absolutely. And as you mentioned, Builder is in free preview right now. You know, some people look at things like Habitat and some of the other great open source projects Chef is kind of shepherding, and they say, “This is all great, great, great, but how—you know, where does, how do they stay in business? Where do they make money?”

    So, Tasha, can you talk a little bit about sort of where Chef sees commercial opportunity with Habitat?

    Drew: Yeah, absolutely. So, for our enterprise customers, having on prem solutions that integrate with all of our open source products in order to have that really strong control plane management layer is super important. And so that’s, with Chef Automate, something that we continue to invest and see incredible success with.

    And then, with Habitat, Habitat becomes the technology that continues to support enterprises while they want to move fast to deliver applications to production at scale. And so, having this part of the technology suite be adopted by our core set of enterprise customers will continue to be part of our strategy, but then we also have Builder to enable smaller shops to just get started—really low friction point, really easy to use. And how we monetize Builder itself over time will be something that we investigate as we see how people adopt and use the technology.

    Shimel: Excellent answer. Great, Tasha. So, Blake, I didn’t wanna leave you too long in the cold. Let’s bring you into this now. First of all, Blake, I don’t think I asked you, and I don’t know if our audience is savvy to it, but what exactly is your role at smartB?

    Irvin: So, I—we have a team of sort of multidisciplinary, polyglot engineers at smartB. We do all kinds of things because we’re a small shop and a small company, still. But the focus that I have—and this has been my focus and my interest, sort of, for the last, it’s actually been a little bit over 10 years is on all manner of operations engineering.

    So, what I used to do would be something more like traditional systems administration, and now I would say I’m more of a software engineering operations specialist. In other words, I’m focusing on improving and streamlining the process of software engineering and less focused on things like racking servers.

    Shimel: Sounds like you’re doing DevOps to me, man. [Laughter] But Blake-

    Irvin: Yeah, you could say that.

    Shimel: Yep. So, Blake, smartB is using Habitat?

    Irvin: Yeah, that’s true.

    Shimel: Let’s talk about why.

    Irvin: I would say, for the moment, about-sorry?

    Shimel: I was gonna say—why? What are you using it for?

    Irvin: Well, we’re using it for almost everything. We have one—yeah, we have one production service that is not yet what I called Habitized or wrapped up in the Habitat package, and we’re just waiting for some work in the community to be finished before we can do that. But yeah, like, 85, 90 percent of our services are running under Habitat, using the Habitat supervisor, and we’re using as much of that workflow that Jamie and Tasha were describing as we can at the moment.

    And that’s actually been a pretty good process for us, I think, because the way that Habitat’s designed, the adoption process can be incremental. So, when Tasha was referring to giving Habitat users the freedom to pick and choose how they want to deploy or how they want to go about adopting, that’s definitely something we’ve leveraged a lot to make our story good for ourselves.

    Shimel: Got it. And so, Blake, I mean, when did you guys adopt, start with Habitat?

    Irvin: So, when I saw the launch video, which was, I think, Adam speaking in Seattle-Adam Jacob, the Chef CTO. I was sort of, I was watching it originally just out of curiosity and then, as the talk went on, my mouth started hanging open further and further, I think. Not because I thought it was the most groundbreaking thing that I’d ever seen, but more because I couldn’t-I was surprised to see how many of the check boxes that I had in my mind from 10 years ago around this kinda stuff, all this kind of operations work, how many of those check boxes were getting checked off by the project as the architecture and the behavior of the tools was described.

    So, we really started looking at the feasibility of adoption within weeks, I think, of that launch video.

    Shimel: Wow.

    Irvin: And started doing the rollout probably a few months afterwards. So, we’ve been doing it like a very slow, incremental rollout to, you know, ensure that we don’t disrupt any of our current software development process or our current production environments. That’s what’s been going on. It’s been a continual process for, I’d say, like eight months now, but just done very slowly.

    Shimel: Very cool. So, Blake, you know, I have a long history of working in open source-related projects and stuff, and I’m always curious, you know, as a consumer, if you will, or a user of this open source project, have you guys—what have you contributed back? Have you contributed any code, bug findings, suggestions for new features? How are you giving back to the community, here?

    Irvin: Yeah, sure. I think the thing, since we’re a really small team, what we’ve really focused on is trying to use the tool in the way that it’s been designed to be used and then giving feedback to the community and/or to the core maintainers whenever we find bugs or places we want to see improvements, and just keeping that conversation really open and fluid, which has been great. It’s one of the reasons that I’ve continued to use Chef software stuff for—not quite my whole career, but a sizable majority of my career. It’s just that, I think, as an organization, Chef believes that community is very important, and in fact, perhaps, one of the most important things about building a company. And compared to other experiences I’ve had with other open source groups, I think that’s been a real win for everybody involved.

    So, in terms of code contributions, I think mostly what we’ve done is put out a lot of plans, which are the sort of blueprints for packages that Habitat uses when you set up like a set of dependencies for your application. So, we’ve definitely published a lot of those back into the community. I think you can find them if you look on the Builder site under the smartB origin, you’ll see all the packages we’ve published. And we’ve also published some stuff back into the core plan side, just the ones that the Habitat team maintains.

    Shimel: Got it, got it. So, Jamie and Tasha, does Blake and smartB sort of represent the kind of prototypical Habitat user?

    Winsor: I’d say so. You know, with Habitat, you can pick and choose what pieces you want, and it’s been great to watch Blake and smartB grow from—back to the packaging manager and the supervisor and gradually moving toward adopting Builder. They’ve been great to work with in the community. They gave us a lot of great feedback. You know, they’re users, so one of the things that is really important is, you build a feature or you build—you know, you fix a bug, and then you immediately get the feedback from the community, and they’ve been unbelievable to work with.

    I’d say that Habitat would work for any team that ships software, though. So, they’re absolutely a typical user, because they ship software, right?

    Shimel: Mm-hmm. Cool beans. Guys, we’re almost at the end of time, unfortunately, leading up here over 15 minutes. But Tasha, for people who may want to, you know, go explore Habitat themselves, where can they go to get information?

    Drew: Yeah, we would love them to visit our website, Habitat.sh. There, you can see, we have a quick, you know, just kick the tires, 10-minute kinda step-through demo. We have more in-depth tutorials if you want to start just playing around with both the run and build time features of Habitat, and our community is very active on our Slack channel, which you can find a link there to join, but it’s just at Slack.Habitat.sh. So, as you get started, if you wanna just jump in that Slack channel, the entire core team is there, our community is there, and we seek to be super welcoming to new people. New questions are very welcome in the general channel. So, you can just hop in and get support and ask whatever questions you have.

    Shimel: So, it’s Habitat.sh?

    Drew: Yeah, yeah.

    Shimel: Excellent, excellent. Guys, listen. Thank you very—all three of you, Blake Irvin, Jamie Winsor, Tasha Drew—thank you so much for being our guests in this episode of DevOps Chat. Continued success, Blake, with you guys in your bigger business as well as using Habitat, and Jamie and Tasha, with you guys continuing to develop and make Habitat, you know, an indispensable tool for companies around the world. Alright?

    Drew: Thanks so much.

    Winsor: Thank you.

    Shimel: Alright, everyone, have a great day. This is Alan Shimel for DevOps Chat. We’ll hopefully see you soon on the next chat. Until then, take care, everyone.

    — Alan Shimel

  • DevOps Chat: ParkMyCloud Saves Wolfpack App Over 50 Percent on Cloud

    DevOps Chat: ParkMyCloud Saves Wolfpack App Over 50 Percent on Cloud

    In this DevOps Chat, find out how the developers of Wolfpack (wolfpack.run) are saving over 50 percent in cloud costs by using ParkMyCloud’s cloud management platform.

    Though the cloud offers many benefits to app developers and others, controlling the costs can sometimes elude organizations big and small. Cloud sprawl can quickly lead to large cloud bills. In this chat, we speak with Jonathan Chashper, CEO of Wolfpack, about how his company got a handle on the cloud and is spending less than 50 percent of what it was paying before.

    As usual, the streaming audio of our conversation is below, followed by the transcript of our conversation.

    Audio

    Transcript

    [su_pullquote align=”right”] [su_custom_gallery source=”media: 118815″ target=”blank” width=”200″ height=”200″]Chris Parlette
    Chris Parlette is the Director of Cloud Solutions at ParkMyCloud. Chris helps customers reduce their cloud waste and manage their hybrid infrastructures by drawing on his years of experience working at various software startups. From SaaS to on-prem, virtualization to cloud, monitoring tools to cloud management platforms, and small businesses to large enterprises, Chris has seen it all and loves helping drive improvements to IT management.[/su_pullquote]

    Alan Shimel: Hey, everyone, it’s Alan Shimel of devopsy.kinsta.cloud, and we’re here for another DevOps Chat. Today’s chat is actually a great behind-the-scenes look at ParkMyCloud and how an app provider named WolfPack is using Park My Cloud to really kinda get a handle on their cloud costs, their cloud usage and optimizing the cloud for their applications.

    That being said, let me introduce you to our guest for today’s podcast. Our first guest is the CEO and founder of WolfPack, and his name is Jonathan Chashper. Jonathan, welcome to DevOps Chat.

    Jonathan Chashper: Thank you, Alan. Thank you for having me.

    Shimel: Thank you. And then our second guest is from ParkMyCloud, it’s Chris Parlette, and Chris, I hope I don’t mess up the title, but it’s director of Cloud Solutions?

    Chris Parlette: Correct, yes. And thanks, Alan, for having us—appreciate being here.

    Shimel: Not a problem. So, guys, this is gonna be a fun one, I think. Jonathan, why don’t you give our audience a little background—what exactly is this WolfPack app?

    Chashper: Sure. So, WolfPack, we are—you can look at us as a connected vehicle technology focusing specifically on helping individuals move together. Think about how much fun it is to move as a group. Imagine you are, you know, maybe several families moving in a few cars. It’s a lot of fun, but there are two major problems—one, keeping everybody together, making sure people don’t get separated, don’t get lost. And the second one is actually communicating with each other while traveling and doing it safely.

    Now imagine you are now doing it on model cycles, which is the first niche we’re focusing on. So, we’re solving those two problems with WolfPack, and by the way, people can take a look at WolfPack.run-R-U-N-to see how it works. But generally, what we do is, we provide, with a turn-by-turn navigation, to every one of the people driving or riding, while providing a radar showing the location of everybody else.

    So, we actually make sure that everybody gets there without being killed, without being lost, without getting into a ditch, and making the experience much less stressful, and much more fun.

    Shimel: And that’s a good thing. But Jonathan, well, you know, WolfPack represents sort of this new—it’s not really new. I’ve seen them now for a couple years, but I call them these instant group type of applications where you can create a group on the fly and that group can have two people or 200 people or more, right? And this group then does something. In the case of WolfPack, you know, they’re on a run or a trip of some sort, and you have Google Maps built into it or underlying it and, you know, all of these things, and they’re great.

    The problem is, as you probably are well aware, is you never know from moment to moment, you know, what your load is gonna be, right? How many groups, how big a group, you know, what’s going on?

    And so, in resource planning, that can be a problem. Talk to me—

    Chashper: Correct.

    Shimel: —yep. Talk to us a little bit about what you’re doing with ParkMyCloud and how that’s helping you.

    Chashper: Mm-hmm. Yeah, so you’re absolutely correct. What you said at the beginning, it’s true. We can—our users can immediately generate groups, it can be, they can generate any number of groups. It can generate—any group can have any number of people. They can plan a ride which can be as short as 10 miles or as long as 1,000 miles, so you’re absolutely right.

    And so, when we’re building the system, obviously, it’s a cloud based system. We are on top of AWS. What’s important for us is that the development costs, being a startup, is being-you know, kept to the minimum. And so, what we did, we actually looked into ParkMyCloud, and with ParkMyCloud, we got to the point that we control our development costs very, very tightly. Generally speaking, when you’re using a server in the cloud, as long as you don’t turn it off, it still works and you’re still being billed for it. And if your development teams go home-say, they work 9 to 7 every day, they go home at 7. If the servers are off, you’re still paying for them.

    So, what we did with ParkMyCloud, we actually manage the up time of those servers, so we only pay for the time we’re actually using those servers for our development for our staging.

    The other good thing we’re having is that we actually have quite good visibility. True—a nice supporting feature, which I think is pretty new, where we can see very, very quickly, you know, when is our load high, what is actually being used, what is not being used, if anything needs to shut down. So, we can manage that very, very easily. And specifically, on the park management, it’s almost a file and forget. Once you set it up once, you never need to go back to it, and it just does it for you.

    Shimel: Got it. So, Jonathan, let me just be clear, I want to make sure we’re talking apples to apples. So, when you say, you know, you’re not using the server overnight or something like that, are you saying that, with ParkMyCloud, you can schedule your instances to close? So, in other words, I’m only gonna use this instance from 9 to 5, and at 5:00, retire that instance, close it out, don’t keep it on my billing account, and when I start up again tomorrow at 9, I’ll start a new instance with that same configuration and kind of pick up where I left off, or is it something else?

    Chashper: So, first, we need to understand that you can use ParkMyCloud on your non-production instances. Production instances obviously need to be up 24/7, so you don’t want to shut them down.

    Shimel: Yeah, you’re right—I should have said that. [Laughter] No one wants to shut down their production.

    Chashper: Yes. So, what I was referring to, I was talking about our development and staging nonproduction systems.

    Shimel: Mm-hmm.

    Chashper: Yeah, so that’s for that.

    Shimel: But are you actually—I guess what I’m asking, are you retiring those instances and firing up new ones the next day, or is it—

    Chashper: Oh, this is—yeah, so this is done automatically for us. That’s the beautiful thing about ParkMyCloud. We actually, once you set up the parking schedule, it will shut down the server, say, 5 p.m. today, and it will turn it on back tomorrow morning, say, 9 a.m. We don’t need to do anything. This is done for us automatically, and so you know, where the developers come back into the office, as far as they’re concerned, nothing changed, they just keep working.

    Shimel: That’s fantastic. Chris, I just want to confirm with you—those instances are actually, you know, it’s sort of like immutable infrastructure, right? Those instances are retired, done, and then re-fired up the next day?

    Parlette: Yeah, so what this is—and it’s not terminating the instances, it’s actually just stopping them and then turning the same ones back on. And the nice thing is about that, is that now, you know, as people are working on these servers, you know, if you haven’t fully gone with data, you know, not being stored on those servers, you know, as you’re working on these same servers over and over and over again, you know, we stop them and then start them right back up again. So, even if people like your finance department or your QA department maybe have some things, you know, spun up, we can actually turn them off to save on the CPU and memory costs, and then we just turn them back on again.

    Shimel: I got it. Okay, because I was gonna say, you know, I mean, the immutable infrastructure angle is great, but in this sort of circumstance, it’s not just a configuration of the server, it was the state of the data that you were working on, and how the heck would you re-create that every morning? [Laughter] But that explains it—got it.

    So, Jonathan, you know, so ParkMyCloud is making you guys a lot more efficient here in the cloud and your use of cloud resources. Have you been able to quantify a percentage savings or anything like that?

    Chashper: Oh, absolutely—most definitely. If I want to be, you know, conservative, I would probably say that we’re saving around 55 [percent]to 60 percent.

    Shimel: Really?

    Chashper: At least. Frankly, I think we probably save close to 70 percent. We’re really very tight. We really manage to do that in a very tight manner.

    Shimel: So, beyond just closing down and firing up the instances the next day, what are some of the other things you’re doing with ParkMyCloud?

    Chashper: Oh, so—well, for us, the first thing is reporting and notification. If something happens, we get notified, we can actually control what we get to be notified about. So, then we’ll get notification about maybe a server that needs to be shut down or something like that.

    Reporting is very important to us so you can see your usage, you can see what is up and what is down.

    Shimel: Got it.

    Chashper: One of the cool things we just started using is actually a mobile app. So, there is a mobile app on iOS for ParkMyCloud. People can download it at the iOS store. And it actually happened with me, I was at dinner, and we—my guys had to stay a little later, and we had to, what we call, snooze one of the parking, you know, schedules, and I just did that through the mobile app, I didn’t even have to go online.

    Shimel: Fantastic. Very cool. So, guys—look, 50 percent or better savings, all these other benefits is certainly nothing to sneeze at. I think anyone would jump at it.

    But, Chris, let’s dive in with you now a little bit, if we can. You know, we have an audience here of DevOps sort of folks from all over the world, right, who would love to save 50 percent on their cloud bill. How is ParkMyCloud working with these DevOps engineers and DevOps teams to realize those kinds of savings and efficiencies.

    Parlette: Sure, yeah. So, we work, you know, plug into a lot of DevOps processes and toolkits straightaway. I mean, that’s something that we do internally, and then, you know, we certainly have all of our customers, from large enterprises down to small startups, all doing this same kind of thing.

    And so, there’s a few different ways to do that. One of the big things about DevOps is automation, and so, we have a full policy engine so you can set up the policies to look at tags, to look at locations to different regions that these servers were in, naming schemes—all these different kind of factors, and you can automatically apply schedules and assign to teams and decide on different actions to take based on these policies for any new instances that you spun up. So, that’s one automated way.

    We also have a full API, so you could plug into DevOps tools like Atlassian, Bamboo or Jenkins so that these tools can kind of reach into our API, temporarily override schedules, change schedules—all this as a part of an automated thing.

    One of the other things about, you know, DevOps—not just automation, but also just thinking whole business, kind of a holistic approach to this thing—and so, we also have a full, a very easy to use UI. So, if your users are, maybe are a little less technical or you don’t want them to necessarily have direct access to a database console or Azure or Google consoles, this can kind of be sort of a proxy for that console so that other users can come in and manage their end servers. They don’t have to ask IT to turn things on for them. They can kinda manage their own servers and do all this, but you can decide who has access to what through this approach.

    So, there’s a couple different ways we plug into these teams and these processes so that users can get their best bang for their buck.

    Shimel: Got it, got it, got it. Now, so, Chris, when you talk about plugging into things like Bamboo or Jenkins, what you’re really—I mean, I just wanna make sure I’m clear. You can build sort of the ParkMyCloud—not sort of, but you can build the ParkMyCloud policies right into your CI/CD pipeline. Is that what you’re saying?

    Parlette: Yes, so if you do things like using tags, especially as instances are spun up or spun down, a lot of people say, you know, “Hey, I put a million tags on every instance.” And so, that’s what the ParkMyCloud policies can read off those tags and kinda decide on the actions based on those tags.

    And then, as far as things like—you know, specifically, if we were to talk about Jenkins or some other build processes, you know, you can have these environments built out and shut down. And then Jenkins, at the beginning of some sort of build process, can call into ParkMyCloud and say, “Hey, I need these servers up, and so I wanna override these schedules.” Spin ’em up, the servers are up, it does all it needs to do, and at the end, it just either shuts them down or, if you don’t shut them down, then basically, if you only did an override, let’s say, for two hours? At the end of the two hours, we’ll revert back to our original schedule, so that way you don’t have to remember to turn the lights out when you leave the room, right? So, we actually can shut things down for you.

    Shimel: Excellent. And Chris, give me an idea of—so, when you guys are engaged, I mean, with that kind of API, with that kind of integration into the DevOps toolset, you’re probably getting a decent view of how DevOps teams are using ParkMyCloud. And you’ve given us some examples, here.

    Are there any—I guess what I’m trying to ask, are there any special things that the DevOps people are doing that perhaps waterfall or other types of technologies are not doing? Is there commonality? Is it very easy to say, “Oh, these guys must be doing DevOps, because here’s, you know, what they’re doing?” Have you noticed any of that?

    Parlette: Yeah.

    Chashper: Alan, before Chris answers, maybe I can jump in and give a quick lead-in to this.

    Parlette: Sure.

    Chashper: Sorry, Chris—hope you don’t mind.

    Parlette: Sure.

    Chashper: Actually, one of the things I wanted to say is that we don’t have DevOps people at all, and one of the reasons we’re using ParkMyCloud is because we’re actually—we’re not DevOps people. We don’t know what needs to be done. We don’t speak, you know, Chef or any one of those guys.

    So—but the fact that we could actually use ParkMyCloud and then the user interface they provided us with, the web client, at least it did for us, went away, and everything is handled at that level. So, for us, that was a big, big move forward.

    Shimel: Yeah, no—and that actually is a great point, Jonathan. And Chris, I’m sorry we hijacked you here for a second.

    Parlette: No problem.

    Shimel: But the problem is, in today’s market, frankly, hiring people who have Chef or Puppet or Jenkins or, you know, DevOps skills sets is (a) hard to find, and (b) very expensive, right? And so, if you can deliver benefits like 50 percent savings on cloud usage or allowing some automation like this, it certainly is a lot easier. I don’t know of any engineering team that wouldn’t like that.

    But, you know, Chris, what do you see? I mean, are most of the people using the UI, or are they just you know, kinda going to APIs and stuff like that?

    Parlette: Yeah, and you know, as we get into larger customers, you know, it definitely is a mix of the UI and the API. The other change that we see, the mindset shift that we see from people is, especially as they’re moving from on-prem to cloud, a lot of people, when they’re used to—a lot of people, when they’re coming from a [Dog barks].

    Shimel: Hey, Jonathan, I think Chris might have shut off there for a second. Chris, you there?

    Parlette: Here we go. I’m back—I’m back. So, one of the things that, you know, as people are moving from on-prem to cloud, you get a lot of people who are used to leaving things on 24/7, because you know, on-prem, as things are left—you know, if you spin things up, it’s not an additional cost, really, to leave them on 24/7.

    And so, in this shift to the cloud, they have this shift of mindset where, if they’re being charged by the hour or it’s something that, you know, every hour that it’s left on is additional cost.

    Chashper: Yeah.

    Parlette: And so, when people are used to leaving things on 24/7, now the shift we have with people using ParkMyCloud, especially in an automated way, is to leave things off 24/7 and then kind of spin them up on demand. And so, that’s really where I think, you know, Jonathan and his team use that a lot, where things can be off, you know, turn on a schedule, but also just turn on on demand so, as the users have access to the UI and the API, they can turn things on on demand, as opposed to having to turn things off on demand.

    Shimel: Fantastic. Well, guys, you know, as I told you off mic, the time here goes really quick. [Laughter] We’ve already been on 18 minutes, beyond the 15 minutes I told you we would do, so we’re gonna need to wrap this one up right here. But first of all, Jonathan Chashper, CEO, Founder of WolfPack, it’s WolfPack.run is the URL, correct?

    Chashper: Yes, absolutely—thank you.

    Shimel: Thank you. Thank you for sharing with us today and giving us a little peek behind the kimono of how you guys are using ParkMyCloud. Continued success for any of our motorcycle riding friends or any group out there who—you know, I encourage you to check WolfPack out, it’s a pretty cool app.

    And Chris, thanks so much for sharing a little bit more, you know, a little behind-the-scenes look and a little bit more of a peek at, you know, ParkMyCloud and how it’s helping companies. Thanks for being on today’s DevOps Chat.

    Parlette: Yeah, Alan. Thanks for having me. I really appreciate it.

    Chashper: Alan—thank you very, very much. It was a pleasure.

    Shimel: Thank you, guys. You know what? We’re recording this right before Thanksgiving—have a happy holiday. Happy holidays to everyone listening! You’re probably listening to this after Thanksgiving, but it’s the start of the holiday season, so happy holidays, everyone.

    This is Alan Shimel for devopsy.kinsta.cloud and DevOps Chat. Hope to see you soon on another chat, and have a great day.

    — Alan Shimel

  • DevOps Chat: Microservices & Containers – Mastering the Replatforming of the IT Infrastructure

    DevOps Chat: Microservices & Containers – Mastering the Replatforming of the IT Infrastructure

    The replatforming of the enterprise IT infrastructure utilizing microservices and containers is no small undertaking and is usually provoked by a shifting set of key business drivers. That is precisely the case today. The term “digital transformation” is in the hearts and minds and on the lips of top-level business executives and IT leaders alike. The underlying traditional or legacy infrastructures that have dominated enterprise IT for nearly 30 years simply cannot handle the workloads or power the applications that will drive business decisively forward.

    Microservices & Containers

    Both microservices and containers are destined to play a major role in this enterprise replatforming, for different reasons. Containers, for their part, hold significant benefits both for developers and the development effort as well as for the organization itself. Understanding this range of benefits is instrumental to securing IT budget for containers going forward.

    In this DevOps Chat, Jim Scott, director, enterprise strategy and architecture at MapR, discusses the most promising and compelling tools, technologies and solutions destined to play a major role in this replatforming of the enterprise. Jim highlights how to stay one clear step ahead as IT undertakes what is arguably its most mission-critical task in a generation.

    You can also download a great ebook on this topic by Jim at here.

    As usual, the streaming audio of our chat is immediately below, with the transcript of our conversation below that.

    Audio

    Transcript

    [su_pullquote align=”right”][su_custom_gallery source=”media: 118809″ target=”blank” width=”300″ height=”300″]Jim Scott
    Jim Scott is Enterprise Strategy & Architecture at MapR Technologies and is very active in the Hadoop community. Jim helped build the Hadoop community in Chicago as cofounder of the Chicago Hadoop Users Group. He has implemented Hadoop at three different companies, supporting a variety of enterprise use cases from managing Points of Interest for mapping applications, to Online Transactional Processing in advertising, as well as full data center monitoring and general data processing. Jim also was the SVP of Information Technology and Operations at SPINS, the leading provider of retail consumer insights, analytics reporting and consulting services for the Natural and Organic Products industry. Additionally, Jim served as Lead Engineer/Architect for Conversant (formerly Dotomi), one of the world’s largest and most diversified digital marketing companies, and also held software architect positions at several companies including Aircell, NAVTEQ, and Dow Chemical. Jim speaks at many industry events around the world on big data technologies and enterprise architecture. When he’s not solving business problems with technology, Jim enjoys cooking, watching-and-quoting movies and spending time with his wife and kids. Jim is on Twitter as @kingmesal.[/su_pullquote]

    Alan Shimel: Hey everyone, this is Alan Shimel devopsy.kinsta.cloud, and we’re here for another DevOps chat. We’ve got a nice chat lined up today. Lucky enough to be joined by Jim Scott of MapR. Jim, welcome to DevOps Chat.

    Jim Scott: Thanks, Alan.

    Shimel: Thanks for having – thanks for being with us today, Jim. So Jim – well, let’s start with this. Our audience may or may not be familiar with MapR, but let’s assume for the moment they’re not. Can you give us a little bit of background about MapR?

    Scott: Sure. So MapR is a converged data platform. The operable words being “data platform.” So MapR provides a data platform for all data. All types of data, all locations, cloud on premise, and it can run in all of those locations simultaneously. The premise of the platform is to deliver a standard open API architecture on top of our data platform, so that customers can reap the benefits of our platform, but not have to worry about being locked in to the vendor, MapR, for the software that we provide through those open APIs, or even the infrastructure that you choose to run on.

    So if you wanna run on your hardware in your own data center, or if you wanna run your software in the cloud or multiple clouds, you can run your software on top of MapR on any of that infrastructure, anywhere you choose, any time.

    Shimel: Great. So Jim – and just for our audience information, what’s your role with MapR?

    Scott: Oh, so for MapR I’m in charge of enterprise architecture, both internally and externally.

    Shimel: Excellent.

    Scott: So what that means is internally, I’m basically lining up and showing how we’re using MapR to solve our own problems, and then externally, it basically means I work with our customers a pretty substantial amount, helping them work through how to be successful with a big data platform. So what is the right way to do something. I’ve personally worked in a large number of different industries, and so my experience comes to the table quite often.

    Shimel: Got it, got it, got it. We got a little vibration there, but I think it went away. And Jim, as we were talking off-mic before we started today, I was telling you I had covered MapR back in Network World when you guys launched, and I’m gonna guess that was around 2008 or so. Correct? 2009, maybe?

    Scott: Yeah, the company went in – or was founded in stealth mode in 2009.

    Shimel: Yep, I remember it. So certainly there’s been a lot of water under the bridge in terms of in 2009, cloud itself was quite the rage. It was relatively new, still, and people were still figuring that out. The whole big data market – there wasn’t a market for big data, frankly. Hadoop and these things had just come out.

    But Jim, probably one of the biggest innovations or changes in infrastructure over the last two or three years, maybe four, has been the big move to containerization of microservices. I would imagine you’re seeing this at MapR as well.

    Scott: Yeah, absolutely. So if we look historically, just say maybe eight, ten years back, message-driven architecture started becoming popular; service-oriented architecture, things of that nature. And that popularity was great – eh, maybe even 15 years back. But the problem was scaling out the messaging platform itself was difficult, it was expensive, and it was really just a pain from an administration perspective, to be able to scale out.

    So we’ve seen this transition over the last handful of years, moving more towards a microservices model. And the term – a lot of people like to throw it around loosely and call something a microservice when it’s not, but the less monolithic in nature, the more difficult to handle the large scale.

    Which is why the Kafka API and what MapR provides with MapR Streams helps enable the customers, because they have the ability to push through massive volumes at a linearly scaling cost model, so they know exactly where they’re going. So it’s pretty easy to push through something like 1.5 trillion events per day on a platform like MapR with as little as five servers.

    And the more micro in nature, the higher the volume you’re going to need to be able to handle. And so those messaging systems of the past – just not there. Just couldn’t handle it. I mean, I remember being excited when we were getting 50,000 to 60,000 messages per second through the system.

    Shimel: For good reason, Jim. You’re talking to someone who was excited when the 386 came out. So yeah, I hear you. But Jim, fundamentally what my – and I just had this discussion, because we actually have a little microservices journal, Microservices Neighborhood, within devopsy.kinsta.cloud. And I was talking to JP Morgenthal, who’s a CTO at DXE, actually, about this just yesterday.

    Fundamentally, fundamentally – and we’re talking about real microservices now, not nonsense, right? But fundamentally, it has such promise to change the way we do development and to change the way our infrastructure runs, right? Especially when you marry it to containers.

    Scott: Yep. Yeah, and I think what you’re alluding to, and this is one of the things that I have the biggest problem with when talking with people about microservices, is the fundamentals are that a microservice is really more about the deployment model than anything else.

    And without a container to be able to put it in and deploy it, it becomes a very difficult and unwieldy thing, even though it’s so small.

    Shimel: Got it. So Jim, you mentioned before you were working on both the internal and external aspects of this. Internally, if you can – let’s not give away any company secrets – but internally, is MapR utilizing your container-based infrastructure with microservices?

    Scott: Uh-huh, absolutely. Yeah, matter of fact, we talk about it pretty regularly. So for our own website, we have implemented customer 360 capabilities. We run our software inside of containers, and we run it currently on one cloud, but we have on-premise hardware, and we will be in the not-too-distant future running multicloud and on-premise all simultaneously. And in order to do that you need a technology like a container to get that type of isolation from your infrastructure.

    Shimel: Yep. An abstract, if you will; an abstraction – I had an interesting conversation, Jim, with I think it was at _____ recently, or another site, another large organization who had totally moved to a containerized infrastructure, and he was talking about what a profound change it was.

    And I asked him about patching containers versus immutable infrastructure, just standing up new ones, replacing the old, rather than actually patching the existing ones. Where are you on that question, if you don’t –

    Scott: Well, I love the concept of being able to go back to my base, default image and be able to run it through my entire devops process, continuous integration process. So I can go patch one instance in my dockerfile, I can test the core out, make sure it’s doing what it’s supposed to, and then I can let all of my downstream builds run, pick up the new container, and run them through their integration tests.

    Shimel: Excellent. So Jim, I don’t know if everyone in our audience understands this whole idea of eating your own dog food, and what it means. But can you maybe give us an example or two of how things you’ve learned by using them internally at MapR have translated into better service, better product, better results for end users?

    So how you’ve taken it out, how you’ve taken certain processes or waves of technology that you use yourself, and translated that into a customer experience.

    Scott: Yeah, so there’s a couple that pop to mind, and number one is actually containers. We only introduced support for containers around the beginning of this year.

    Shimel: Really?

    Scott: But we’ve been using – yeah, it was – I think our press release that we did on it when we officially released support was around February time frame this year. But technically speaking, we have a number of customers who’ve been using containers with MapR for two, three years.

    And little secret here: We use it internally, and have been for two, three years as well. So making sure that we had it fully vetted and very well baked from a support infrastructure perspective was really critical, but getting that out there for customers, really important.

    Because it solves such a major problem with containers, which is you look at something so simple as security, right. If I am responsible for the web server for my company and you are responsible for the financial systems of your company, or of my – or our company, that software in a container has the ability to run on the same server.

    But if they run on the same server and we are utilizing local storage on that server – because you can’t save data inside a container; it blows up. It’s gone when the container dies. So if you save data locally on that server, what happens? Well, suddenly I run these security risks of my web server having access to financial information – probably not a good idea.

    Shimel: No.

    Scott: So we implemented, within our MapR persistent application client container, the ability to talk to MapR from within the docker container. And because MapR offers NFS support, it’s almost invisible and there’s hardly anything to do. And you give it a ticket from your authentication environment, and you’re good to go.

    Whatever’s running inside that container has a user ID, and when it talks to MapR, the data is saved and protected. You don’t have to worry about oh, well, if I’m gonna save data, I’ll save it to the local server. You get rid of that whole problem. It isolates where the data goes, already being protected and highly available, if your server physically dies, just like if your container dies, you don’t care. You don’t have to restore the server to try and get the data off of it.

    Shimel: Got it, got it. That’s 100 percent – that’s the beauty of that whole abstraction layer, if you will.

    Scott: Exactly.

    Shimel: Jim, here’s an interesting question, and I don’t know if you have this info, but I figured I’d ask anyway. We’ve been doing a bunch of surveys about container adoption, and it’s funny, we’ve done several of these surveys now over, let’s say, the last two years, and when we first started, a lot of people had plans of looking at docker and containers.

    Other people, a decent amount of people, had containers in their – they were in the testing dev environment kind of thing, not production. Few had them in production. Then recently, like our most recent surveys, let’s say, over the summer, that number shot up. Probably 40, 50 percent of respondents have containers in production.

    That may be because it’s devopsy.kinsta.cloud, we’re in a devops bubble, and so you get people using this kind of new stuff. But I’m curious, out of the MapR customer base, a gut check if you don’t have the real number, what do you think the percentages are that are using containers in production these days?

    Scott: It’s a pretty big number. I don’t have a hard-and-fast number, but if I just go by the customers that I talk with, I feel like it’s gotta be over 50 percent.

    Shimel: Really? Wow.

    Scott: Yeah. Yeah, we’ve got one extremely advanced customer in Australia, their name is Quantium. And they talk about how they use MapR all the time, and they’ve basically reinvented their entire operating model, and they’ve done it on top of MapR. There’s an enterprise architecture I documented that they talk about periodically; it’s called the zeta architecture.

    They basically use an orchestration layer, they use MapR for persistence, and then they use containers to launch and deploy anything anywhere arbitrarily across their cluster of hardware for their customers. And they turned their entire internal platform to an as-a-service model for internal and external customer consumption.

    Shimel: Got it, excellent, excellent. We’re hearing stories like this all over now, and the interesting thing is it’s almost wherever you see containers, you’re seeing microservices. But you’re also seeing microservices in places where you don’t necessarily also see containers.

    And it reminds me of when I was in school, we used to learn all pasta is macaroni, but not all macaroni is pasta, or something to that effect. But microservices has a place outside of containers, though they seem to just go really well together.

    Scott: Yeah, absolutely, and I think when you start looking at the different types of use cases that you would fit into a microservice, this is where you start seeing how people are going to choose to deploy and manage. If you’re looking at event-driven, message-driven, where it’s taking a message in, it’s doing some work and it’s dropping a message back out, versus it’s a restful service endpoint, where it’s a synchronous response.

    Yeah, people are choosing to do things in different ways, but nearly – everyone that I talked to, they’re using containers in some fashion.

    Shimel: Yep, yep. It’s true in my case too, but like I said, I always figure well, I’m in a bubble, so of course I’m gonna see that. Jim, what’s next for MapR?

    Scott: Oh, that’s an in intriguing question. Recently, we announced our 6.0 launch, and we have greatly enhanced a number of features around our database capabilities. I seem to have this conversation a lot with people, is they start looking at big data technologies, and they say well, this is great and all, but I’ve built this software to work with mySQL or Postgresql or Oracle or whatever RDBMS.

    So that’s fine, you can containerize those applications if you wanna start changing your infrastructure and how you orchestrate your software, and if it’s important enough to you, you could migrate it over onto a no SQL database like MapRDB document database, or you can just leave it running in a container so you have a more flexible infrastructure.

    And the database technology enhancements that we’ve been making are really critical to kind of driving the future of getting people off of those super-expensive to scale RDBMSes that are being used for non-relational purposes.

    Shimel: Excellent. Just time frames – when might we see some more on this stuff?

    Scott: Well, it’s been announced in the last few weeks, and so we’ll start our 6.0 general availability very soon. I forget the exact date. Then we’ll just continue plugging along, adding more new, fantastic features to help enable the customer base.

    Cloud integration with our platform is another really big one, making sure that we can help enable customers to benefit from the concept of cloud neutrality. Cloud providers are never just gonna get together as a consortium and say hey, why don’t we standardize our APIs to the benefit of our customers?

    It just doesn’t happen. It’s not gonna happen. Google, Microsoft, and Amazon are never just gonna call each other up and say hey, what do you think about this?

    Shimel: I agree. Agreed. Well, not – you know what – well, I was just, I’ve seen a lot of changes over the years, but that’s probably a pretty safe bet, Jim.

    Scott: Yeah.

    Shimel: Unfortunately. Anyway, Jim, believe it or not, as I told you when we started, the time goes really quick, and we’re about out of time for this version of DevOps Chat. But we’d like to maybe have you back on in the future. We’d love to keep up with what’s happening with MapR, especially as it relates to microservices and containers and some of the new stuff we have going on.

    Jim Scott: Yeah, sounds great.

    Shimel: Great. But in the meantime, if people wanna get more information about MapR, where can they go?

    Scott: Well, I would recommend they go to MapR.com.

    Shimel: Okay.

    Scott: Or of course, you can always follow MapR on Twitter and/or any of the other social networks that you join and work with. But the website is always my favorite, because there’s a lot of great material there, including the MapR blog, and all of the ebooks, including the “Microservices and Containers” ebook that I wrote is out there, for free.

    Shimel: Got it. Got it, got it, got it. All righty, well, we’ll have to maybe get some links to those we’ll put in the show notes of this as well.

    Scott: Yeah, that’d be great.

    Shimel: Fantastic. Hey, Jim, thanks for being our guest on this episode of DevOps Chat. Look forward to having you back soon and finding out more about MapR. Continued success, and keep us posted about what you guys are doing.

    Scott: Thanks, Alan. Appreciate it.

    Shimel: All right. Hey, everyone, this is Alan Shimel for DevOps Chat on devopsy.kinsta.cloud. Thanks for joining us, and we’ll see you soon on another chat. Take care, everyone. Bye-bye.

    — Alan Shimel

  • DevOps Chat: Aqua Security Seeks To Secure Containers

    DevOps Chat: Aqua Security Seeks To Secure Containers

    The migration to container-based application infrastructures is accelerating at a faster pace even than we saw with hypervisor-based infrastructure 18 or so years ago. Hand in hand with this is the race to provide better security solutions that are native to and purpose-built for containerized environments. Aqua Security has established itself in the container security arena, recently raising $25 million in capital funding as it seeks to lock down the container security market.

    In this chat, we sat down with Rani Osnat of Aqua Security to discuss the recent funding round and the state of the container security market. Below is the streaming audio of our discussion. Below that is the transcript of our discussion.

    Audio

    Transcript

    [su_pullquote align=”right”]Rani Osnat[su_custom_gallery source=”media: 1739530″ width=”100″ height=”100″ align=”center”]
    Rani Osnat has more than 20 years of enterprise software industry experience, in project management, product management and marketing, including a decade as VP of marketing for innovative tech startups in the IT security and cloud arenas. Previously Rani was a management consultant in the London office of Booz & Co. He holds an MBA from INSEAD in Fontainebleau, France. Rani is an avid wine geek, and a slightly less avid painter.[/su_pullquote]

    Alan Shimel: Hi, everyone. This is Alan Shimel for devopsy.kinsta.cloud and Security Boulevard, and welcome to another chat.

    Today’s guest is Rani Osnat, VP of Marketing for Aqua Security, one of the leaders in the container security market, and Rani, welcome to our DevOps Chat.

    Rani Osnat: Thank you, Alan. Thanks for having me.

    Shimel: Thank you. Thank you for being here. So, Rani, the big news out of Aqua recently is, by anyone’s yardstick, a large raise of capital to fuel Aqua’s continued growth.

    Osnat: Yep, that’s correct. We raised $25,000,000.00 in a series B round led by Lightspeed Venture Partners, which is a very well-known tier one VC.

    Shimel: Yep.

    Osnat: And, it’s—well, to the best of my knowledge, it’s the biggest funding round in our space in continuous security to date, and it also brings total funding for us to $38.5 million.

    Shimel: That’s fantastic.

    Osnat: So, we’re, I think, in a pretty good position to capture what we think is a tremendous opportunity in this market.

    Shimel: Sure. And so, Rani, we were talking off mic prior to recording today—you know, beyond the sheer magnitude of the number, right, 25 is a big number, 38 is a bigger number—but beyond that, there’s also the validation. Which is, you assume a company like Lightspeed Ventures, Microsoft Ventures is another investor in the company, Shlomo Kramer, of course, from Check Point is another investor. You would assume these people are pretty bright people and they don’t throw their good money away. That, if they’re investing this money, they invest it for two things. No. 1, they see the opportunity in the market, so the market itself is ripe and worthy of that sort of investment. And then No. 2, that the team and the product at Aqua are also worthy of the investment.

    Let’s first talk about the market, if we can, though. I mean, so this is a validation, I think, of the container security market, if you will, right?

    Osnat: Yeah.

    Shimel: The containers are real, that securing them are real, and there’s gonna be a big market for that coming up. What do you think?

    Osnat: Yeah, I mean, definitely, when we started Aqua just under two years ago, we didn’t quite know how fast the market would move and what kind of market it would shape up to be. And fairly early on, even when we were in beta in early 2016, it became fairly obvious that this is an enterprise market. That is that, you know, some of the biggest adopters of container technology are not just Silicon Valley startups or web-scale companies that kind of invented this, but Fortune 500 enterprises across verticals like financials, insurance, retail, aerospace, manufacturing, and so forth, as well as government.

    And so, paradoxically, you would expect this kind of new technology to be first adopted by startups, but the real boom in adoption comes from the fact that enterprises are adopting it very quickly, and I think there are several reasons for that. One is that the benefits of using containers are overwhelmingly positive, and the benefits are pretty immediate. So, you can see benefits in agility in terms of how fast you can deliver applications to the market, you can see benefits in moving to the cloud, and reducing your infrastructure costs and operational costs. And you can see much better control over what kind of applications you’re deploying, where, and when, et cetera, largely through automation.

    So, these benefits are overwhelming, and additionally, containers are easy, right? So, it’s—you know, one of the things that made containers, which in itself is not a new technology, it’s been around for a while, but what companies like Docker did is make containers very accessible and very usable. And so, the fact that they’re easy to use makes adoption easy as well, and you know, you kind of see a pincer movement of, on the one hand, grassroots adoption through developer and DevOps teams of containers, so they develop applications using containers. And on the other hand, you’re seeing a top down approach of, you know, CIOs, CTOs seeing the benefits of containers, adopting them, or mandating their use for, for example, cloud native applications to drive cloud usage, hybrid cloud, cloud migration—all of those good things.

    Shimel: Absolutely. So, Rani, you know—granted, I’m not as involved in container and container security as you are, obviously, but to me, a couple of different thoughts. Number one, in my limited—granted, limited expertise, containers aren’t that easy, right? I mean, I think when we talk about containers and then, like, Kubernetes and stuff to manage the whole environment—for someone who hasn’t developed on them before and is used to developing, let’s say, in VMware or in Bare Metal, there is, there’s a learning curve. Maybe once you pick it up, it’s easier.

    But the other thing is, I always try to compare sort of container adoption, let’s say, to hypervisor adoption or VMware adoption, and I would say that containers, their adoption curve is higher or steeper, faster than sort of hypervisor’s.

    Osnat: Yeah, that’s correct.

    Shimel: And really what I see, the last couple—you know, we’re always doing surveys here at devopsy.kinsta.cloud, announcing the new Security Boulevard site—what we’re seeing is, you know, we always saw people messing with the containers, right? R&D testing, what have you—but the numbers of containers, the percentage of companies that have containers in product deployment, you know, in real, frontline stuff? Where it was—jeez, just two or three years ago was low teens, below 20 percent—is now climbing, getting near that critical mass number, right, of 50 percents and stuff like that. And that’s huge, right? You’re talking about early adopters moving to mainstream in a matter of months, and that’s —

    Osnat: That’s correct.

    Shimel: — that’s the numbers you see as well?

    Osnat: Yeah, we see that. And again, I don’t think this would’ve been the case if containers weren’t easy, as I said. I mean, there’s always a learning curve with new technology, but I think if you’re a new developer, a new programmer working today, working on a greenfield application, there’s really no reason for you to use something that is not containers unless it’s something obscure for technology reasons.

    But mostly, it’s a much easier way to develop applications quickly than previous approaches. And I think, you know, if we take a broader picture, a container is—this adoption of containers does not occur in a vacuum, and we actually see a convergence of three trends that are kinda coming together and containers are probably the biggest enabler of those three trends. And these trends are happening independently of each other. So, you know, one trend is one that you are very familiar with, it’s the name of your site, is DevOps, you know, the proliferation of DevOps as a method, process, organizational philosophy, et cetera, which is all about delivering applications faster and in a more continuous way, and containers make that easier than before.

    Another trend is the move to microservices architectures, you know, for web applications or cloud native applications, which is basically adopting what web scale companies like Facebook and Twitter and Netflix started, and now bringing into the mainstream. That, too, is facilitated to a great degree by containers, because implementing microservices as containers is, today, probably the easiest method of doing that. I mean, there are the so—called serverless architectures, et cetera, but they’re much less mature than containers are at the moment. Not that containers are super mature, but they’re more mature.

    And the other aspect is the whole kind of hybrid cloud, cloud native movement where, you know, it took 10 years, but enterprises are adopting cloud services in a very big way, including industries that you would never dreamed of having on the cloud, on the public cloud, like pharmaceutical, health care, insurance, banking, federal, and so forth.

    So, all of these—and again, containers, because they’re portable, because they can run anywhere in the same way, they’re a really good way to realize this notion of hybrid cloud. For example, we at Aqua have a customer—who, unfortunately, I can’t name, but it’s a well-known company. And they are, for them, this is already a reality. They are deploying applications on both Amazon and Azure using the same framework, using the same security policies, and in a way that makes these clouds interchangeable for them, so they can run application A on Azure today and they can run the same application on Amazon tomorrow. So, that’s a real change that we see, and containers are a big part of that.

    So, all of these things together, I think, are what’s driving this adoption. And yet, you know, I think we have to distinguish between adoption and penetration or, you know, maybe some other word. But the fact is that today, if you look at the total workloads that enterprises are running, containers are a very, very small percentage of that. But, if you look at any new applications delivered over the next five years as well as looking at migrating legacy applications to the cloud, you’ll see this number growing exponentially over the next few years.

    So, yes, in terms of the number of enterprises now either looking at, working with, or already deploying containers, that number is growing very fast, and I think there’s probably hardly a large enterprise on the planet today that is not at least thinking about containers if not already using them. But at the same time, you know, it’s early days for them. Most of them—not all of them, but most of them, the vast majority are looking at the first few applications being deployed, and this is just the first drop of rain before the deluge. So, it’s gonna be a lot bigger.

    Shimel: Yep. Rani, a couple of comments, questions, follow ups. So, first of all—yes, DevOps, containers, they seem to go like peanut butter and chocolate. Again, so does microservices and containers, but to me, I—you know, when I was a kid, a teacher once taught us that all spaghetti is macaroni, but not all macaroni is spaghetti.

    Osnat: Sure.

    Shimel: So, it almost seems that, you know, wherever you see containers, you see microservices, but you don’t always see containers when you see microservices, right? In some regards, microservices can be even bigger, right? It has even more of a use footprint. Actually, on devopsy.kinsta.cloud, we’re launching a microservices channel which we hope will eventually grow into a standalone site, because I do think microservices is that big and has the potential to be that big.

    Osnat: Yeah, but microservices is not a technology; containers are. Microservices is an architecture, you know what I’m saying? There’s a difference.

    Shimel: Yeah, fair enough. Yep. But, you know, bringing it back—and, of course, loud usage and all of that. But, let’s turn for a moment and talk about the mission of securing containers and all of these architectures and platforms and so forth. That’s really what Aqua is, and when we talk about why they invested money, these investors, not just because the containers are so big, but because the mission of securing them is also big.

    Osnat: That’s true. So, yeah, I mean, when you look at—so, first of all, we’re not really, I mean, the fact that we’re a container security company might allude to the fact that we secure containers. That’s not exactly true. So, you know, we don’t look to secure a container or containers, we look to secure the workloads or applications that are running on containers.

    Shimel: Mm-hmm.

    Osnat: And that’s a difference. Because—and again, here, there are several trends that come into play. One is really that these things that I mentioned before, the DevOps trends, the microservices, architecture, and moving to the cloud, they also affect how you can and want to do security. You know, if we look at the cloud first—when you go to the cloud, a lot of the basics, you know, the bread and butter security that you use to invest in as an enterprise disappears. Why? Because these are services that are already taken care of or provided by the cloud service provider.

    So, you know, when you run your applications on Amazon, you don’t really have to worry about things like data center security, infrastructure security, network security to a degree, identity management to a degree. And so, a lot of these things are—you know, they’re basically taken care of for you. So, the focus shifts to the application layer, and that is one aspect.

    Another aspect, which is a really positive one for using containers, is the fact that, since you are now breaking up the application into a bunch of microservices, you can have much more visibility and control into what the application is doing. You know, if you look at an individual container or microservice, it’s much easier to understand the intended function of that container or service, and consequently, be able to understand what is considered good behavior, and be able to identify pretty easily what is bad behavior. And you do that at a very granular level.

    Additionally, if you look at the networking aspect, in a way, the networking has been brought into the application in a more visible way than before. Before, you could have things done within the application that maybe a malicious actor did within the application, and it’s all done internally, and it’s a black box. You don’t have visibility into it until it actually gets out.

    With microservices, you are able to see the interaction between the different components of the application, sometimes at a very fine grained level and, again, be able to identify any kind of weird or abnormal communication between, you know, parts of the application that are not supposed to be communicating that way. So, that’s another benefit. So, these things really allow us to have much more control and visibility over what the application is doing.

    The last piece is around DevOps, which is often nowadays coined as DevSecOps, which is the whole notion of shifting left security—i.e., bringing some of the control points not to the run time environment but to the developer environment. And why is that? Because containers are also immutable, right? So, if you’re using containers directly, you’re not supposed to see any change in one time environments. You’re not supposed to see patching, nobody is supposed to SSH into your host that’s running containers, it’s supposed to be entirely orchestrated and automated.

    And so, anything that happens there is by definition not good. But, at the same time, you have the opportunity to make the application more secure from the start, which is in the development phase, and that includes things like, you know, removing vulnerabilities, ensuring the configuration is correct, enforcing best practices. And if you can automate that—which we do—then you’re winning, right? Because you’re delivering a much more secure application to start with, you’re gonna have a smaller attack surface, and then in run time, all you need to do is make sure that there is no drift or no unexpected changes to the run time environment, which would be considered malicious or at least suspect.

    Shimel: Got it. So, Rani, we’re running a little low on time. We’re actually out of time, but let’s keep talking. [Laughter]

    Osnat: [Laughter]

    Shimel: So, now, we’ve got—what are you gonna do, how are you using the 25 million? What’s the plan?

    Osnat: Right, so you know, in our short life of two years, we managed to launch the product very early and get quite a few enterprise customers, many of them Fortune 100 customers across many industries, and that sort of traction is what really made Lightspeed and the other investors invest in us, and we need to continue this. The market is growing, right? As we mentioned, adoption is growing at an accelerated pace, so we need to be there.

    That means, you know, investing in the quality as well as the breadth and depth of our solution. There is—containers are not a singular technology. I mean, there’s a lot of variation in the stack, not just in orchestrators and OS’s, but in the container engines themselves, and this will become even more so over the next few months as things like, you know, additional run time engines are introduced, and not just the Docker open source run time engine.

    And so, this requires investing in the right technologies and making sure that we can fulfill the requirements of our customers and, additionally, of course, we need to expand our presence, both in the U.S. as well as in other parts of the world.

    Shimel: Let me talk to you about that a sec, Rani. So, I’m gonna assume U.S. probably represents at least 50 percent of your market, is that about right?

    Osnat: That’s correct, yeah. [Cross talk]

    Shimel: And probably Ameya the lion’s share of the rest.

    Osnat: At the moment, yes, but Asia is also growing, you know, there’s—

    Shimel: I also wanted to ask—what about AP?

    Osnat: Yeah, so Asia Pacific is growing as well. There are a lot of container users in places like Australia, China, Japan, and so forth. And so, this is—these are obviously big economies, there are big companies there. They’re as advanced in terms of cloud adoption as the U.S. in some cases. So, we need to be there.

    Shimel: Absolutely. Well, how do you—so, I was actually out at RSA APJ this summer, and it was in Singapore. And, you know, I had a chance—I actually put on a DevSecOps event there with RSA. So, I had a chance to talk to many of the RSA attendees, some of the vendors and stuff.

    I mean, what’s interesting there is, your infrastructure, you know, even in Europe, AWS is an 8,000-pound gorilla, right?

    Osnat: Correct.

    Shimel: Where do you—we don’t see that in Asia Pac. AWS is still a player, obviously, but there are other solutions there.

    Osnat: Mm-hmm. Yeah, that—

    Shimel: Go ahead. How does that affect you guys, or how do you deal with it?

    Osnat: Well, it affects us partially because mostly, you know, we’re pretty much cloud agnostic, right? We run on all the major clouds. It’s not a major effort for us to run on additional clouds. You know, even today, we support Amazon and Azure, Google, IBM Bluemix. And so, it’s relatively easy for us to support additional clouds, and it’s not a—that’s not the stumbling block, in most cases, right? It’s more like things like esoteric OS’s or weird configurations that we encounter sometimes. But, in most cases, that would not be an issue. Not to mention that a lot of companies are also running private clouds, using VMware or other types of virtualization infrastructure. So, that is really not a stumbling block for us, at all.

    Shimel: Good, good. Interesting. Rani, as I said before, we’re way over time, I need to wrap this up. But, first of all, congratulations on the raise. I’m sure Aqua has 25,000,000 good uses for this money as well, and we’ll see more coming down the pipeline in terms of both functionality products as well as reach around the world. You know, always good to have you on, you have a great handle on where the container market is and of course where the container security market is.

    So, thanks for being our guest today. We’ll have you on again soon.

    Osnat: Thank you.

    Shimel: Thank you. This is Alan Shimel for DevOps and Security Boulevard Chats. Thanks for listening. We’ll hope to see you soon on another chat. Bye bye, everyone.

    — Alan Shimel

  • Welcome to the CA Live Chat Series home page!

    As the popularity of DevOps grew, so did a common refrain: Continuous integration and continuous delivery create an environment to continuously introduce vulnerabilities into software. The idea was that the rapid speed of DevOps was in opposition to the concept of secure development. However, as time went on, it became apparent that a system that fosters collaboration and promotes automation creates continuous opportunities for integrating security testing into the development process.

    During this session Maria Laughlin, Vice President of engineering at CA Veracode, and Chris Eng, Vice President of research at CA Veracode, will provide their perspectives on how to build security into a company’s culture so that it permeates the development process. They will discuss the need for empathy and understanding between teams and offer practical advice on making security a part of the DevOps environment such that it becomes another element of quality.

     

    Welcome to the CA Live Chat Series home page!

    CA and devopsy.kinsta.cloud are proud to announce CA Live Chat, a series of webcast-like events which allow experts to host discussions on meaningful DevOps topics, from insights and outlooks to product demos. These events are typically highly interactive and thrive on attendee give-and-take. This fresh, new concept introduces a vibrant, well-trafficked YouTube Live channel of video-chats promoting DevOps and Continuous Delivery topics for the purpose of thought leadership. Please click on any of the on-demand sessions below and “stay tuned” for our exciting upcoming CA Live Chats!

    Assess and Guide Your DevOps Journey Leveraging Industry-leading DevOps Research – Live Chat
    The DevOps Research & Assessment Group (DORA) is a group of the most preeminent researchers, scientists and experts in the DevOps community—and is the force behind the annual State of DevOps Report. CA has partnered with DORA to offer a unique assessment service for organizations in any stage of DevOps maturity. By leveraging DORA’s expertise and existing research—such as over 23,000 data points from over 2,000 companies worldwide—the assessment service can help you understand both the strengths and areas for improvement in your own DevOps programs.


    Better Training for Better Security
    In this session, Pete Chestna of Veracode talks to David Wayland from a Fortune 500 Financial Company about the recent CA Veracode/devopsy.kinsta.cloud DevSecOps survey. This lively discussion reveals what they have seen in the industry and their real world experience in implementing successful training programs. Your takeaways from this webinar will be solid recommendations for creating your own secure training initiative that leads to better engagement and better results.

    Implement a Best Practices Approach to Container Monitoring
    Listen to this conversation with Kieran Taylor, CA Technologies and Alan Shimel, devopsy.kinsta.cloud to understand:
    • What new monitoring, analytics and alerting techniques are best suited for containers.
    • Why an open data model is key to understanding and tracking microservice dependencies.
    • How to establish monitoring across the entire application lifecycle to increase development speed and quality.

     

    Is GDPR Getting Ready to Give Your Company the Left Hook?

    The General Data Protection Regulation (GDPR) introduces new and broad changes for data privacy for any company that retains EU customer data. Because so many companies are global, this regulation will most likely involve your company. GDPR gives EU citizens far more control on how you use their personal data. They can limit your use, be forgotten, and even seek damages if their data is misused or breached.

    Join CA Technologies’ Jeff Hughes, Sr. Principal Product Marketing Manager and devopsy.kinsta.cloud founder, Alan Shimel, as they discuss this upcoming legislation and how companies should go the distance by preparing for the May 2018 deadline! Jeff will provide concrete steps that you can take now to start or continue preparing for this new regulation. You’ll walk away with a knock-out action plan to start moving your testing and compliance teams in the right direction.

     

    Modern Application Monitoring is the DevOps X-Factor – Discover why!

    Just as we’re starting to get our heads around application containerization and orchestration and along comes serverless computing, service mesh architecture and hype-free Internet of Things use cases. Now DevOps practitioners need to elevate monitoring capabilities beyond relatively simple guarantees of technology uptime and application performance into the more challenging area of ensuring an optimum customer experience. An experience that fuses physical and digital, isn’t time bounded, and incorporates a complex series of customer journey touchpoints.

    Join a conversation with Pete Waterhouse, Senior Director of Product and Solutions Marketing at CA Technologies and devopsy.kinsta.cloud founder, Alan Shimel to understand:

    • What’s on the near-term biz-tech horizon (including containers, serverless, IoT) and how this will radically impact your monitoring strategies
    • How traditional practices in continuous delivery, testing and monitoring must evolve to support new digital models and the Internet of Things
    • The increased importance and benefit of application performance analytics across the DevOps toolchain

    Application Monitoring DevOps X-Factor

     

     

    Data Privacy – Why your company needs to pay attention to what’s behind the compliance curtain!

    Data Privacy has once again vaulted to the forefront with Facebook and other large companies grappling with data breaches and the use (or misuse) of personal data. We are charting new frontiers in data privacy and legislation alone will not mitigate the data challenges that lay ahead. Brand loyalty and brand trust can take years to build, but can be lost in an instant with one data privacy misstep. There are a number of regulatory challenges facing enterprises globally that need to be considered as well including The US Patriot Act and the upcoming GDPR requirements. Join us  learn from Alan Shimel of devopsy.kinsta.cloud and DevOps testing and compliance expert Jeff Hughes of CA Technologies on:

    • What is Data Privacy?
    • What are the penalties for non-compliance?
    • How can companies comply with data privacy laws?
    • What else can a company do beyond adhering to compliance regulations to protect data privacy?

    Enjoy a lively discussion with useful ideas that will help your teams exceed compliance regulations and preserve trust with your customers.

     

    Links to additional resources:

    Container Monitoring and Management eBook
    This eBook from The New Stack lays out the monitoring challenges confronting todays DevOps engineers, describing approaches to metric collection and visualization, the classes of modern monitoring, and importantly – how what methods work to tame complexity and ensure these dynamic application environments deliver on their promise.
    https://www.ca.com/content/dam/ca/us/files/ebook/container-monitoring-ebook.pdf

    The New Stack Makers Podcast: Future-Proof Docker with Modern Monitoring
    In this podcast, we discuss the complexities of monitoring microservice architectures and applications deployed to Docker. Sushil Kumar, SVP of Product Management at CA Technologies, describes the importance of employing an open, flexible monitoring data model for Docker.
    http://transform.ca.com/rs/117-QWV-692/images/CAtechnologies_Podcast.pdf

    The Total Economic Impact™ of Application Performance Management from CA Technologies
    A commissioned study conducted by Forrester Consulting on behalf of CA Technologies that examines the potential return on investment (ROI) that enterprises may realize by deploying CA Application Performance Management (APM).
    https://www.ca.com/content/dam/ca/us/files/industry-analyst-report/the-total-economic-impact-of-apm.pdf

    Try CA APM Today!
    Monitor application performance in minutes. No credit card required. It’s easy and free for 30 days.
    https://www.ca.com/us/trials/ca-application-performance-management.html?mrm=620656

    Contact – Becky Arenson
    Rebecca.arenson@ca.com

  • DevOps Chat: Harness.io, Continuous Delivery as a Service w/ Steve Burton

    DevOps Chat: Harness.io, Continuous Delivery as a Service w/ Steve Burton

    Today marks the launch of Harness.io, continuous delivery as a service. The company is founded by some of the original team at AppDynamics. They want to take the same kind of approach AppDynamics brought to APM, to the CI/CD market. We caught up with Steve Burton, VP of product marketing at Harness to get the scoop on this exciting new company.

    As usual immediately below is the audio of our discussion and the transcript of our discussion follows after that.

     Audio

    Transcript

    [su_pullquote align=”right”][su_custom_gallery source=”media: 118147″ width=”300″ height=”300″]Steve Burton

    Steve is VP of Product Marketing at Harness, a new Continuous Delivery-As A-Service Platform. Prior to Harness, Steve ran Product Marketing at AppDynamics, Moogsoft and Glassdoor. Steve also started his career as a Java developer back in 2004 at Sapient. When he’s not playing around with technology he’s normally watching Formula 1 or researching cars on the internet. Twitter: @BurtonSays[/su_pullquote]

    Alan Shimel: Hey, everyone, it’s Alan Shimel, with another DevOps Chat here on devopsy.kinsta.cloud. Today’s chat is with Steve Burton, VP of products at an exciting new company – just launched today – called “Harness.” Steve, welcome.

    Steve Burton: Thanks, Alan. Great to be here.

    Shimel: Great to have you here and, hey, it’s a great day to launch a company, Steve.

    Burton: Yes. Sure.

    Shimel: Of course, you know, I’m reminded of what someone told me when I sold the first company I started, and they said, “Companies are like boats – the best two days are the day you buy ’em and the day you sell them.” So this is one of the good – this is one of the big days for ya. And speaking of that, though, Steve, before we get into what Harness is about and markets and stuff, you actually – you’re one of the founding team there, and the team’s really kind of a get-together of a lot of the guys who helped found AppDynamics, I guess, back in the day. Correct?

    Burton: Yeah, for sure, with a few AppDynamics guys here. We’ve also got people from Apple and Google as well, so there’s a nice mix. But, yeah, for sure. There’s Jyoti, there’s myself, Greg Howard, and Randy Boysen, so there’s – yeah, that’s what we love to do, is build start-ups.

    Shimel: Yeah. Getting the band back together. So, Steve, though, with Harness, though, you guys have taken an interesting kind of approach here in that, you know, you see a tremendous opportunity within the CICD market. Correct?

    Burton: Yeah. Yes. I mean, Harness has primarily kind of been built for continuous delivery as a service, so the kind of CD part of CICD, and we think there’s a big opportunity there. There’s a lot of excitement, there’s a lot of projects and customers, and there’s a lot of complexity to solve, and so, yeah, that’s what we’ve kind of founded the company to kinda help customers with.

    Shimel: Sure. And, Steve, I mean, here at devopsy.kinsta.cloud, we obviously follow the CD space pretty closely, and there’s certainly no lack of companies that have thrown their hat in the ring here, right, without naming names, from well-funded start-ups to big, traditional 800-pound gorillas, but the fact that no one has really captured a dominant market share probably indicates it’s because no one’s really come up with the killer CD app, if you will, or the – and if it’s not the app, maybe it’s the delivery means and everything else. Is that part of what drove you guys to, you know, go this route with Harness?

    Burton: Yeah, I mean, for sure. I mean, if you speak to Jyoti, one of the key reasons he founded the company was when he was, obviously, CEO and founder of AppDynamics, he met with a lot of enterprises and big customers, and they were telling him, “Hey, look, we have hundreds of DevOps engineers, but a lot of them are spending their time scripting and building automation platforms. And so there’s no real solution to help us kind of be like a Facebook or a Google, where we can do continuous delivery.” And so that was really the impetus for Jyoti to kind of take a step back and think, “This problem really isn’t solved yet.”

    Shimel: Yeah. No, I think you hit on something there ’cause there’s been some – not backlash, but there’s some talk on some of the DevOps channels that, “You know what? DevOps hasn’t been as successful as we thought it was going to be and a lot of organizations aren’t experiencing the success they hoped with DevOps.” And, frankly, a lot of it is subjective, not objective, and maybe some of it also has to do with hyped-up expectations that can never be met anyway.

    But, that being said, you hit on something there that I think is important, that, a lot of times, when we talk about DevOps teams and implementing DevOps, it’s time spent figuring out automation, right, and scripting automation. And, whether it be automated testing or automated integration or configuration or what have you, it’s kind of like groundwork, rather than actually doing the work. And you probably need to do the groundwork first, right, Steve? I mean, no one’s – I’m not saying that, but, with a SAS solution like Harness here, you can kinda not cheat, but you can get a head start on that.

    Burton: Yeah, for sure, and I think one thing we’ve tried to do is simplify continuous delivery, and so a lot of the scripting and automation that you might require teams of DevOps engineers to do, we feel we can give those teams a solid platform that they can build on top of, to do continuous delivery. And so I actually sympathize because, if you look at the complexity and the work involved to make continuous delivery work, it’s more than just provisioning infrastructure and configuration management. I mean, today, you’ve got release strategies, you’ve got canary deployments, you got blue-green. You think it’ll verify how those deployments succeed in production. You’ve got rollback, so, if things fail, how quickly can you roll back?

    And I think as well, from speaking to customers, one thing we hear a lot of is just the compliance and security around DevOps, and so DevOps is ultimately about empowering teams and giving freedom to developers and operations, but, at the same time, these companies need to control and audit and make sure that things are done correctly. So a lot of the boring stuff, like the credentials and secrets management and kind of visibility and reporting, that’s as much of the puzzle as a lot of the cool stuff.

    Shimel: Yeah, you’re talking my language, Steve. Certainly, security and compliance, and I tend to think compliance is the caboose on the security engine. You know, just doing compliance doesn’t make you secure. But, that being said, security and compliance are major – they could be your best friend or your worst enemy, in the continuous delivery game, right? If you do it right, you could automate a lot of that and shift it left. It could really help speed up a successful CD process, a successful experience. But if you don’t pay attention to it and you run afoul of your security and compliance, I mean, it’s like going back to “Go,” don’t collect the 200 bucks, and you’re starting from scratch. So it’s an important piece of the puzzle, for sure.

    Steve, give us a little background, though, in – you know, so Harness sees this and the team sees this opportunity with CD. It’s a SAS-based solution. What more can you tell our audience about what you guys are doing?

    Burton: Yeah. So, primarily, it’s SAS and we also offer on-premise software as well, so you can deploy either model. I think what we’re trying to do is three things, is one is automate the actual deployment aspects, and so, instead of needing kinda scripting or coding a lot of the configuration and wiring together all of your ecosystem, and so you kind of use CI in your repository tools, like Jenkins, Bamboo, JFrog, CircleCI; and then making those kind of work in sync with kind of Amazon cloud platforms, Kubernetes, Docker; and then, finally, all the configurations. So what we’ve tried to do is build reusable connectors and templates for all of the technologies in the ecosystem so that you can literally plug them together and build CD pipelines in minutes.

    And that sounds totally alien and kinda sounds too good to be true. We’ve managed to do it, but it’s been a lot of work and it’s been working with a lot of customers. And so, in the last 3, 4 months, we’ve literally had, I think, 270 kind of validation meetings with customers. We’ve got over 20 betas in deployment. And so it’s been understanding that complexity of how we can simplify the CD pipelines today. So, yeah, it’s still a lot of work to do, but I think we’re on the right path.

    Shimel: Excellent. And, of course, Steve, you mentioned something that I wanted to bring that out too, is that, you know, it’s sort of this “It takes a village” with CD, right? There’s no one tool that, soup to nuts, is going to do everything for you, and, as you mentioned, Harness already has integrations with a number of the most common tools that are out there, like a Jenkins and so forth.

    Burton: Yeah. Yeah, and I think, if you look at a lot of the solutions on the market, they require you to rewrite your CD process and change the way you do deployments and potentially replace tools you already have, and kinda that never really works because most customers have invested a lot of time and money and resources in those tools, in those processes. And so what we do is we can fit around your existing ecosystem and tools, as well as your processes, and so a lot of the customers who’ve gone through our beta program, within the first three or four hours, they’ve actually been able to get a CD pipeline up and running and reuse the Jenkins or reuse all of the cloud technologies and platforms, in addition to the monitoring tools as well.

    Shimel: Excellent. So, you know, though today’s launch day, Steve, you guys have actually put in months and months, if not year, already of work on this, but where do you think it goes next?

    Burton: Yeah. Yeah, so, in the intro, I talked about us building the building blocks for the automation and getting the CD pipelines built. We’ve also architected Harness to do continuous verification. What that means is, once you’ve deployed code or artifacts in production, we’re actually gonna use machine learning to understand whether that deployment had a positive or negative impact on the business. And so we’re gonna verify things like performance, response times, errors, throughputs, in addition to looking at application logs to see if there’s any regressions or anything that’s been introduced as part of the deployment.

    And so that’s what we’re excited about. That really, we believe, is unique to Harness – the ability to validate a production deployment. And if we see failures or we see anomalies, we can automatically roll back to the last working version for the customer, with no interruption to their business. So, for me, the future, it needs to be more than just deployment. If you don’t know the business impact of your deployment in production, then you could say there’s been a lot of work and time and effort for, potentially, nothing.

    And I remember, when I was a developer, I worked on projects for months and our success was hitting the go-live day. And once we hit the go-live day, it was operations’ responsibility to deploy it and we never really knew whether we had a positive impact or a negative impact. And I think, in today’s world, with kind of continuous delivery and DevOps and going from an idea to money as quickly as you can, you’ve really gotta know the business impact of every change. And if you don’t, you’re letting your company down.

    Shimel: Agreed. Agreed. So, Steve, it sounds – you know, this is something that I’ve always wondered about with DevOps, which is we put so much effort into automation and doing it better, you know, getting better code, automating all of the stuff, automating CI, automating CD to the point where we actually deploy, and then, all of a sudden, post-deployment, we’re on to the next thing or we’re looking for our feedback loops so we can develop the next iteration of it. And I guess sort of APM sort of filled that next post-deployment monitoring and so forth, but it sounds here like Harness wants to fill that gap in there. Is that fair to say?

    Burton: Yeah. Definitely. I think we wanna automate how customers verify their deployments, and so the APM market is like $5-billion market right now because, literally, every company’s building software and their software keeps breaking. One stat for you is 30 percent of application deployments fail. This was an article that Wired.com wrote – I think it was a few months ago – but that’s a staggering amount of failure. And the reactive way to deal with failure is to go and buy an APM product to help you kind of troubleshoot in production, when you do a deployment.

    The smarter way to do it today is to start and do canary deployments in production, where you roll out the application to a small amount of nodes or users and then verify it before you then roll it out across the rest of your application environment. And so you can kind of think of Harness as a safety net – we’re gonna let you deploy in production, we’ll help you with canary deployments, and, if we see something that’s wrong or we spot an anomaly, you can roll it backward with virtually no business impact

    Shimel: Got it. Got it. Well, Steve, believe it or not, we’re probably a little over time here already, but, hey, we tried. Steve, before we wrap up, though, for people who wanna find out more about Harness, where can they go?

    Burton: Yeah, they can go to harness.io, is our Web address, and we’ve got lots of videos and will be a free trial coming shortly. And you can actually subscribe for the early adopter program as well, if you wanna show your interest there.

    Shimel: Oh, that’s excellent. Excellent. And I’m sure there’s plenty of stuff on the website where people can find out more information of all of these things. Steve, congratulations to you, Jyoti, and the rest of the team. It sounds like you guys are on to your next great adventure and I’m sure it’s gonna be an amazing journey. We hope to have you back here soon and we’ll be watching Harness closely and seeing what it brings to the market. Congratulations.

    Burton: Yeah. Appreciate it. Thanks, Alan.

    Shimel: All right. Steve Burton, VP product at Harness, at harness.io, if you’re interested. Exciting new entry in the DevOps space. Best of luck, Steve, and we’ll see you soon. This is Alan Shimel for devopsy.kinsta.cloud, DevOps Chat. We’ll see you soon on the next DevOps Chat.

    — Alan Shimel

  • DevOps Chat: Shift Ops Left, Shift Dev Right with Kristian Stewart, IBM

    DevOps Chat: Shift Ops Left, Shift Dev Right with Kristian Stewart, IBM

    In this DevOps Chat we speak with Dr. Kristian Stewart of IBM’s Cloud Event Management team. Kristian has some keen insights into the role of dev and ops and how they interact. Kristian calls it Shift Ops left, shift dev right. An interesting way of looking at things.

    As usual the streaming audio is immediately below and the transcript of our conversation is below that.

    Audio

    Transcript

    Alan Shimel:               Hello, everyone, this is Alan Shimel, devopsy.kinsta.cloud, here for another DevOps Chat. Got a really hot chat today and my guest joining me is Kristian Stewart of IBM’s cloud event management team. Kristian, welcome to DevOps Chat.

    Kristian Stewart:         Thanks very much for having me, Alan.

    Alan Shimel:               And I just wanna make sure I got it right – it is Kristian Stewart, right?

    Kristian Stewart:         Last I checked.

    Alan Shimel:               Okay. Well, sometimes my funny French accent throws people off, but, anyway – and, Kristian, I mentioned you’re working with IBM in cloud event management, but give our audience a little bit behind that. What exactly does that mean?

    Kristian Stewart:         Well, let me give you a bit of a background ’cause there’s some history here, in particular with respect to the topic of the podcast, which talks about operation tools. So I’ve been with IBM for ten years. I’m one of the lead architects for a suite of IT service management tools, and my own specialization is in event management, as part of IBM’s Netcool brand, okay? So I’ve been in this industry for 18 years.

    I worked for a small start-up back in the late ’90s – the company was called “Micromoves,” and we were focused on introducing tools for disciplined fault management into telcos, into communication service providers, during the dot-com boom of the late ’90s. And then, in the early ’00s, we pivoted and successfully sold into enterprise, finance, retail, as their IT infrastructures became larger and more complex and as they matured, from an IT operations perspective. So we were acquired in 2006 and we started to work towards applying interesting mathematical and analytical techniques, for increasing the value provided by our tools, and applying machine learning and problems to this space. So that was my background up until about three years ago.

    So what do I mean by “cloud event management”? Well, more recently, as part of the movement of the industry that we serve, as they move into cloud and their investment in cloud has increased, I’ve been working with teams to build software-as-a-service offerings  capabilities we had, till that point, only available on-premise, so it includes alert notification tools, runbook automation tools, and cloud event management. And so our particular interest in DevOps is twofold, right? So our transition itself, from going to shrink-wrapped client service software, that was originally deployed on-prem, has gone through to the development of cloud-native, cloud-scale, public cloud resident offerings, and with our guys embracing paradigms like micro-service architectures, 12-factor apps, and, of course, DevOps.

    And one of the biggest challenges or one of the biggest changes our teams have had to endure is that they now have to operate these deployments themselves, right? So they’ve gone from shrink-wrapped, “Give it to the client. Now it’s their problem,” – well, kind of – through to “Now we’re hosting this for you.” So we’ve had folks from development and QA transitioning to operations roles, and they’re required to apply software engineering skills and work with app developers to ensure that we’ve got robust capabilities in private cloud. And we’re providing 24/7 ops to support these offerings.

    But we find ourselves in a fairly unique position because we’re running operations but as a service product that themselves are designed to help people responsible for running operations for their own apps and services. So it’s just sort of a renewed empathy for our end users, especially as we start to angle our offerings towards teams that are embracing DevOps. Does that answer your question?

    Alan Shimel:               Yeah, and then some. [Laughs] But good stuff, Kristian. Thank you very much. So wanted to talk a little bit today bringing this back home to DevOps, right? And one of the trends that we’ve been seeing over at devopsy.kinsta.cloud recently is, really, around kind of putting the “ops” in DevOps, right? ‘Cause we’ve spoken now to a few folks in the DevOps space from both the vendor and practitioner side, who are really emphasizing the role that ops plays in DevOps. And one of my things about it, though, Kristian, is I get the opportunity to speak to a lot of people like yourself, a lot of people who are heavily invested in DevOps, but I also get a lot – the chance to talk to a lot of people who aren’t and are new DevOps or just dipping a toe in the water and planning their DevOps migration and transformation.

    And one of the things that I try to remind people is that “Don’t leave your common sense at the door,” at the DevOps door, right? Bring it with you, number one. And, while you’re bringing something with you, it doesn’t mean everything you’ve done in ops to this point and all the lessons learned and best practices accumulated get thrown out – you know, the baby with the bathwater. With DevOps, we evolve. We’re not necessarily revolutionizing; we’re evolving. And so building on best practices and time-tested ops practices and methodologies and processes is probably a better solution than, you know, scrapping the whole thing and starting from scratch. And who has the opportunity to do that anyway? What do you say to that, Kristian?

    Kristian Stewart:         Oh, I agree completely. And, you know, amongst our clients, we’ve got a whole array of clients, from telcos to finance to retail, right? You name it. Across those clients, we have people on varying points on the curve of maturity, with respect to adopting DevOps. But what I say to those clients, as they look at those paradigms, is that the business objectives of operations teams haven’t changed substantially, certainly since I’ve been in the business. There’s a lot that has changed, but the business objectives continue to be measured by KPIs like mean time to repair, mean time between failures, and pressure on costs, right?

    So disciplines – so, within a, let’s say, a mature enterprise that have invested heavily in ITIL, they adopt disciplines like event and incident management, and that leans on a key set of capabilities, which are equally applicable when you transition to DevOps. You know, the proximity of your developers to your operations and the smoothing of the processes and the interactions between those organizations, if anything, are enhanced by some of the traditional methodologies and tools. So, you know, to list some of them, you need tools which can consume events from highly heterogenous environments; you need to minimize the amount of management noise that’s presented to the people who have to respond to events; you need to be able to integrate with other systems; you need to be able to help pinpoint probable causes, drive efficiency in operations, do things like apply machine learning and advanced analytics to gain insight from your event and incident data.

    So, if you look at the evolution of these practices, historically, right, ITIL emerged in parallel with the practice of event incident management. It didn’t dictate it. You know, these standards emerged as – sorry, event and incident management emerged as de facto practices, driven by the availability of practical tools to solve real problems, and they still solve those problems. You know, if you look at the way that operations has evolved, look at telecom from the late ’80s and early ’90s, right?

    So, in a telco network operations center back then, you might have had dozens of very highly-skilled people, _____ network engineers, with training on dozens of different management tools, and these guys and girls were expensive, right? I mean, their time cost a lot of money, and then the introduction of event and incident management tools helped transition from that to having perhaps the same number or even more operations staff of a substantially lower skill. And so the first thing event management and incident management did was to try and lower the cost of entry for running an efficient operations team.

    But now, if you look at the changes in IT management now – so I was chatting to a large enterprise just the other day, and, ten years ago, they – so these guys have always been driven by those same metrics – MTTR, MTBF, cost, right? A few years ago, their job was to provision physical hardware, static middleware, and the management of that stuff from an operations perspective. Now these guys are having to provide platform-as-a-service tools, distributed runtime container and orchestration systems, and they still have responsibility for the existing infrastructure – the network, the physical servers,  the legacy monolithic applications. And all that still requires management. So, mainly, they’ve got the challenge of managing hybrid cloud, coupled with extremely tight expectations on time-to-market that working practices like DevOps and Agile software development allow for.

    So I’d say that event and incident management tools are as relative now, if not more so, than they ever were because, when you shift to practices like DevOps and site reliability engineering, you know, back to the olden days where you have ops employing of hundreds of you know, low-skill operators, best practice is transitioning to one where you have a substantially lower number of substantially higher-skilled operations staff. Right? So these guys and girls are highly skilled, but they’re time-poor. You know, your typical operations professional, in an organization that’s embraced DevOps, wanna be concentrating on automation, on liaising with dev early on in the release life cycle, on prepping for new rollouts. They don’t wanna be grappling through log files and sifting through performance metrics to find out what happened just so they can restore a broken service.

    And I think the eventual goal of the kind of tools that we provide – event and incident management – isn’t just to help the operator find the needle in the haystack; I think it’s to hand them the needle. And that’s useful, whether you’re a low-skilled junior operator in old-school operations or a time-poor, hyper-technical site reliability engineer with 20 years’ experience. Alan, you still there?

    Alan Shimel:               Yes. Kristian, I’m sorry. I had a little internet issue. So all really good, but, you know, one of the things I wanted to touch on, Kristian, is the subject of empathy, as part of kind of your DevOps mantra, right? Sometimes, people give cultural things short shrift, but empathy is important in DevOps. And what I wanted to ask about was the idea of “How do we get developers?” Right?

    For a long time, the ops people have sort of had to empathize with developers are doing and what they’re doing in terms of development to help in deployment and stuff, as part of DevOps, but now, putting that on the other foot, right, getting developers to empathize with what the ops folks are dealing with, as part of this DevOps equation. I mean, I’ve heard of some organizations where they’ll actually embed developers on the ops teams for short periods of time so that they get empathy the hard way, if you will, right?

    Kristian Stewart:         Yeah.

    Alan Shimel:               They gotta live it. What do you think about something like that?

    Kristian Stewart:         I think, in terms of getting the development professionals involved, I think – so you mention role rotation, right? And I think that’s one tactic. You know, I like to think of this as much as “shift-right dev,” as it is “shift-left ops,” right?

    Alan Shimel:               Yeah.

    Alan Shimel:               Good. Good.

    Kristian Stewart:         And, actually, you know, it reminds me of the situation in shrink-wrapped software development, maybe 20 or 25 years ago, between development, who were the coders, and QA. Right? So there was sort of a brick wall, prior to Agile software development, where it was throwing a pile of code over the wall. And I think Agile, with that shift-left testing, really helped to remedy that, to the extent that I see, today, developers and quality assurance engineers working really, really closely together and not depending on, you know, reams and reams of documentation to communicate with each other.

    So, I think, with DevOps, I think a similar thing is happening, and I think you need shift-right dev, as well as shift-left ops. So operating an application or a service, I think, has to be a shared responsibility, and I do think that dev need to get skin in the game. So that’s gonna include dev as an escalation point, when an incident occurs. It can also include role rotation. But, for mutual benefit, I think they both need to use the same operations tools, right, so that they see what each other are doing, when an incident or a critical event occurs in the environment. And then they’re motivated to improve the use of those tools for themselves, as well as for the sake of operations.

    I think, you know, a 3:00 AM incident notification, you know, like a page at 3:00 AM, is a great motivator. So, for ops people, it’s an incentive to get runbooks and automatic incident responses written so that service restoration is as quick as possible and, preferably, automatic. With dev, it’s an incentive to make their services or applications operable, right, so they need to get the right amount of instrumentation into the app so that ops can manage an incident or, better still, that some script can, so that neither of them get the 3:00 AM escalation. So I think there’s a – with all due respect to my dev and ops colleagues, there’s a carrot-and-stick approach there.

    Alan Shimel:               Mm-hmm. Yeah, that makes perfect sense. Kristian, I want you to repeat what you said before. We need to shift ops left and shift devs right or did I have that backwards?

    Kristian Stewart:         Yeah, no, yeah, that’s what I said. Shift-right dev and shift-left ops, right?

    Alan Shimel:               Got it.

    Kristian Stewart:         So get them to have a degree of overlap with respect to their responsibilities. And, of course, what they do in response to those responsibilities can be quite different. You know, let me see if I can think of an example. So, for devs, under those circumstances, when they’re being involved in responding to an incident, it can be stuff as simple as being really clear in their application logs.

    So, if you think about events that are generated by, say, a container-based app deployed on Kubernetes, okay, so the event and incident management system is gonna be presented with logs from the app code; they’re gonna be presented with logs from middleware; logs from Kubernetes itself; logs from, potentially, external technologies and monoliths – NFS mounts, databases, legacy apps, network; and they may also be getting alerts from monitoring systems that are watching performance metrics for the running container, so transaction request rate between micro-services, CPU utilization, response time.

    So, if devs can make sure that something as simple as the concept of criticality or severity is included in the logs, that really helps with filtering _____ ops figure out whether or not something’s really a problem. You know, for example, is this event classed as a trace message? A debug message? A warning? An error? Is it known that this is service-affecting? Right?

    Alan Shimel:               Yeah.

    Kristian Stewart:         And, you know, how better to motivate devs to implement those sorts of practices than to make them consume log messages that are completely uninterpretable, right?

    Alan Shimel:               Agreed. Agreed. Hey, Kristian, we’re over time already, but, as I told you before we started, the time here goes quickly. But, you know, boiling all of this down, the last – whatever – 18 minutes down, really, I wanna make the point of – and I loved what you said about ops shift left, dev shift right – that our ops and devs can work together. And, as you said, they each will have different reactions with different roles and responses to given incidents or given stimuli, but they can work together and we can build on what ops has built over the last 50 years of the IT industry. And –

    Kristian Stewart:         Mm-hmm.

    Alan Shimel:               Right? We don’t leave that at the door of DevOps; it’s part of it. And one thing I want – you know, if we can ask people to take that with them. And you agree that’s key takeaways here for them?

    Kristian Stewart:         Absolutely. Couldn’t agree more.

    Alan Shimel:               Fantastic. Hey, Kristian Stewart, IBM, thank you for joining us on this episode of DevOps Chat. Love to hear more in the future about what you and the IBM team are doing around cloud and hybrid and some of the other stuff kicking around. And we’ll have you back again soon, but, for now, thanks for being our guest this episode.

    Kristian Stewart:         Alan, it was an absolute pleasure.

    Alan Shimel:               Great. Hey, this is Alan Shimel, everyone, for devopsy.kinsta.cloud, DevOps Chat. Thanks for joining us and we’ll see you soon on the next chat.

    — Alan Shimel

  • DevOps Chat: How DeployHub and OpenMake Software Help CI/CD w/ Tracy Ragan

    DevOps Chat: How DeployHub and OpenMake Software Help CI/CD w/ Tracy Ragan

    This DevOps Chat features OpenMake CEO Tracy Ragan. OpenMake are the producers of DeployHub and DeployHub Pro, which help DevOps with CI/CD. OpenMake is a company that was helping developers long before people spoke about DevOps. Using an open-source business model with DeployHub has transformed the company. We speak with CEO Tracy Ragan to find out how.

    As usual, the streaming audio is below with the transcript immediately following.

    Audio

    Transcript

    Alan Shimel: Hey, everyone, it’s Alan Shimel, devopsy.kinsta.cloud, DevOps Chat, and, in this episode of the chat, we’re joined by Tracy Ragan, CEO OpenMake Software. Tracy, welcome to DevOps Chat.

    Tracy Ragan: Thank you, Alan. Thank you so much for inviting me. It’s my pleasure.

    Shimel: Okay. So, Tracy, the last time I think you and I caught up, we were over at Jenkins World, out in San Francisco, a couple of weeks ago. And, you know, that was quite an event and it really kind of spoke to how big the CD space is becoming—you know, the CD market and the kinds of companies that are utilizing different CD solutions. Of course, OpenMake, you know, plays in that market arena, Tracy, but, for those of our listeners who maybe aren’t familiar with OpenMake, can you give us a little background?

    Ragan: Absolutely. So OpenMake Software has been playing in this continuous delivery space long before it was in fashion. Our initial product, our Meister product, was created about 20 years ago, and it was created to accelerate the process of compiling code once it was checked into a version control tool. So you check in and you build—we were doing that close to 20 years ago. Now we were just doing builds off of a trigger; we weren’t doing continuous integration. What we were doing was the hardcore build automation, and what I mean by that—the automatic generation of the compile and link scripts, the acceleration of the compile process, and the dependency management so you could do just-in-time builds, to speed up the process of moving code forward.

    So we were really early on in understanding the need for faster delivery of code and the ability to create what we call a “release candidate” at any point in what used to be call the “life cycle” and now what we’re calling the “pipeline.” So this is an area that we have heavy expertise in and a lot of heart for.

    Shimel: Absolutely. You know, Tracy, one of the things that I always find interesting, and somewhat sad, is that, when we talk CD, right, some people—I think most people still refer to continuous delivery, but others, you know, talk about continuous deployment, and then, unfortunately, I think there’s a Venn diagram kind of subset that think that somehow continuous delivery and continuous deployment are one and the same. And—

    Ragan: Yes.

    Shimel: – why don’t we start with destroying that myth? Right? What’s the difference? Is there a difference—I think there is—between continuous delivery, continuous deployment?

    Ragan: So if we look at the continuous integration process—and I think that’s where we have to start, to understand continuous delivery and then, of course, continuous build and continuous tests and continuous deployment underneath that— so continuous integration itself, I don’t wanna say it doesn’t do anything, but, in itself, it’s an orchestrator. It itself doesn’t compile code, it doesn’t test code, and it doesn’t deploy code. What does those steps are the unsung heroes who oftentimes write scripts or automation tools that sits underneath that process.

    Continuous delivery is the ability to have a release candidate ready to go at any point in time across what’s now called the “pipeline,” so, if you have a continuous integration at the dev state, you might also have one at a test state—now this sounds very waterfall, which is why I don’t like to describe it this way—but, in essence, what you’re doing is saying, “The development teams are ready and they have a release candidate they wanna push off to testing. Testing should automatically test it and then push it off to production.” Now what’s core, what’s the lowest common denominator between those simple three states, is the ability to get it out to end targets so you can actually run a test at dev or run a test at test or execute it at prod. That’s what continuous deployment does.

    I prefer to call it “continuous deployment.” We often hear it referred to as “application release automation,” or ARA, and that sometimes feels a little heavy and waterfall-like, but, in essence, that’s what those tools are doing, is they’re doing continuous deployment, in the same way as test automation tool does continuous test and in the same way, you could say, Maven or a product like our Meister is doing continuous build, all driven through a CI/CD workflow.

    Shimel: Excellent. Excellent. And, you know, you hit on something else there—ARA. And that, again, we’ve seen several vendors kind of stake—they’ll put a flag in the ground around ARA. So now that we’ve got some basic definitions done, Trace, let’s talk about how do we move right—in this case, it’s almost shifting right, right?—how do we move rightwards down this deployment—delivery, deployment, management—you know, timeline? And—

    Ragan: So I think what we have to do is we have to talk about DevOps, right? So we have this other big term that’s being thrown around, and that’s “DevOps.” Everybody I speak to has a different opinion of what DevOps is, so I’ll just share with you mine. DevOps is the process in which the development teams start taking over more of the operational tasks. And the operation teams’ role is mainly one of auditing or having some level of control. My guess is that, as DevOps is pushed left, so developers are doing more of the work, we’re going to see operation teams become smaller.

    And that’s really—if you really look at what companies are doing, they have large operation teams because they’re doing their own thing when it comes to doing software releases. They’re not part of the CI/CD process, in many of these cases. But, in reality, what happens is is that the developers are the ones who have to do the hand-holding at production, so the more we can get developers to declare their definitions around their software deployments, what their environment looks like, the easier it becomes for the data centers to become leaner. And that’s really what Agile, DevOps is all about, is getting, in the hands of the developers, tools that can automate these operational tasks to a point that the testing teams and the production teams can consume them and have some transparency and some level of control around them.

    Shimel: Got it.

    Ragan: So that’s what continuous deployment, application release automation, is achieving, is that ability for developers to dig in at the dev level, not with scripts because scripts can’t adapt across the pipeline, but to do that in a model-driven way that could adapt across a pipeline, so that the work – all the declarations are done down at dev, and the users of that are testing and production. Even down to a configuration change, if need be, at a Cisco router level. We’re talking low-level definitions.

    Shimel: Sure. Sure. So, you know, Tracy, it’s funny; you guys over at OpenMake have been involved in this space, as you said, 20 years, right? Way before we start using—throwing around the terms “DevOps” and “continuous” this and “continuous” that. And kind of living in the DevOps bubble, if you will, we think, “Oh, everyone’s doing this kind of stuff and this is the way it should be done and this is the way it is being done.” But, really, when you look at it—and you probably have a better view than most, Tracy—how far along are we on market adoption, around this, you know, more—I think it’s a more holistic approach to DevOps and continuous delivery and deployment, as well as ARA?

    Ragan: That’s a hard one to answer. I can give you my best guess and give you some numbers I hear from the analyst. If we talk about ARA, the analyst tell us about 20 percent of the companies have adopted ARA. Now, but, that being said, when I hear—yeah?

    Shimel: Yeah, I was gonna say that sounds high to me.

    Ragan: It does, but let me tell you why it sounds high. It sounds high because, when companies adopt an ARA solution, they’re often talking about a legacy product that somebody on the operations side of the house decided to purchase. And, oftentimes, they’re legacy because they have the requirement of in-target agents, so the decision to put an agent on every single production end point, including cloud or even define them to a micro-service, has to be done way right. That doesn’t mean that we’re doing continuous deployment; it means that their operations side of the house decided on a tool that may or may not be adopted by the left side of the house.

    You know, when I talk to companies, most of them will talk about that they’re adopting DevOps, but if you ask them if they’re able to solve a build in less than 10 minutes or if production can repeat the deployment that developers defined, the answer will be “No.” So the reason why it’s hard to answer that question, to get clear metrics on it, is because everybody’s definition of what that is is so drastically different.

    Shimel: Yeah.

    Ragan: In my mind, I’ve only met a few companies that have really established a real Agile, DevOps practice, and it’s not for the entire organization—it’s for a few teams—so we’re babies, when it comes to this. I believe we have a long way to go to really understand how to make that shift left so that this Agile concept of where testing and production are not separate states in life cycle and separate silos, but part of a circular process that is included everybody on the same team. So waterfall still persist, and to be quite honest, in continuous integration, if you look at what drives the CI process, we are still using the same waterfall scripts that we did, you know, five or 10 years ago, depending on how far you are in CI. Much of the stuff, they consider automated because a script executes it, but that’s not automation. It’s automation in the waterfall world, but it’s not automation in an Agile, DevOp practice.

    Yeah, these steps really have to truly be automated so that it’s transparent, there’s reporting, there’s clean metrics, there’s a continuous feedback loop, because that’s the best way to go fast. The best way to go fast in anything you do is control, right? The more control, the more speed. The problem with control, on the ARA side, is it costs too darn much, which is why we decided to do an open-source solution. It’s like developers, in order to achieve this, in order to really achieve this and really move as fast as possible and increase these release cycles, you have to have developers being able to get their hands on a product that will do the deployments without having to babysit a script for every state in the life cycle.

    Shimel: Agreed.

    Ragan: So we have a ways to go.

    Shimel: I agree 100 percent. But I’m gonna tell you—you know, and this, again, it’s not my original thought, but it’s something I’ve heard and I fully agree with, and that is I don’t think it’s the mission of DevOps or CD or ARA, you know, that we’re gonna totally eliminate waterfall. I think there are, you know, situations where waterfall—excuse me—where waterfall may, in fact, be the right solution to the problem. I think—[coughs] excuse me, Tracy—I think there are other areas where some sort of combination of waterfall and Agile, waterfall and DevOps, you know, some continuous kind of stuff, you know—it’s like that—you know, General MacArthur said, “Old soldiers never die; they just fade away,” or something like that. I think old development styles and techniques never die either, so I don’t necessarily believe in throwing out the baby with the bathwater. What do you think about that?

    Ragan: I think technology will change that. When you’re talking about monolithic applications that have been around for a while and they have some sort of a cadence that they’ve been going through, where they have developers developing and testers testing and production teams managing the production releases, you’re probably right. It’s gonna be some sort of a water-Scrum-fall or what we’re calling “bimodal” approach.

    But, with the introduction of microservices, that is going to change the way—if it’s not a monolithic application, as we move into microservices, even though it solves—microservice solves some problems, it’s going to complicate the landscape of the production environment, which means that the need to have these production operation types and these testing types working hand in hand, on the development team, will be critical. So, as we see microservices become more prevalent, we’re going to see waterfall slowly die.

    Shimel: Okay.

    Ragan: It’s just the nature of micro-services. Now it becomes—microservices becomes code as well. Everything has become code, which means that the productions teams are not gonna be able to track and manage that level of detail in the production environment. It’s gonna go beyond just a collection of DLLs or a WAR JAR file that’s going out to an end location. It’s gonna complicate matters and it’s gonna change the way we—it’s gonna change waterfall drastically.

    Shimel: Yep. And so it’s gonna change it drastically and maybe it does evolve, if you will, but I think that’s what technologies—I think—you know, one of the things I’ve learned over too long, being in this business, is we don’t see revolutions as much as we see evolutions. And then, when we look back over a five-, seven-, 10-year span, it seems like, “Oh, my God, that was revolutionary.” But, from the day to day, it’s rather evolutionary.

    Ragan: I would agree. When it comes to—you know, we have—if you look at the macro and the micro, when it comes to the micro level, we have changed—you know, software developer bank doesn’t evolve anymore; it’s revolutionized. And that is due to Agile. What happens is is that, in some of these enterprises, the fear of doing frequent releases has slowed down the adoption of lean practices at the production state. That, we have to fix because there should be a revolution. If an end user needs a fix, it needs to get out there. It shouldn’t evolve over a six-month period.

    Now, if we look at the macro, we take a—you know, we soar up a few miles up—when it comes to major shifts in the way the community at large does business, it is a evolution. Everybody—I mean, I was at the Gartner data center show last December, and very few of those enterprises had even completed the adoption of cloud. They still had big, you know, warehouses of servers, so you’re right. From the perspective, I would say, the data center adoption, it does evolve, but, at a micro level, I think that we see a revolution in the way developers want to develop software and are driving software, and that’s where the microservices is gonna change a lot of how we do things. And I think, as more developers push for that, I think it’s gonna happen faster than we think.

    Shimel: I don’t—I—so that, along with the evolution versus revolution thing, I think the speed of—you know, this internet time speed warps, you know, the world we live in. It just—I mean, it’s crazy. So it’s no doubt about that one. Tracy, we’re actually—I didn’t realize—we’re kind of over our time, but it’s okay. These things often run over. Let me just quickly bring it back to OpenMake. What are you guys—anything new? Announcements feature-wise? Or what’s happening over there that you may wanna share with our audience?

    Ragan: Well, we have our DeployHub product that you can become part of the open-source community and participate in freely-available continuous deployment at deployhub.org. We are also—we’ll be getting a press release out very shortly here, regarding a SAS version of our continuous deployment solution, which it will be both a supported version of the open-source, as well as the pro version which includes release management capabilities, all from a SAS model, so you don’t have to download anything. We can do that because we’re agentless. Makes it real easy to support people in a cloud model.

    Shimel: Excellent. Tracy Ragan, CEO of OpenMake Software. Thanks for being our guest on DevOps Chat. Hope to have you on soon—have you on again soon and you can keep us posted.

    Ragan: Okay. Thank you, Alan.

    Shimel: Thank you. This is Alan Shimel for devopsy.kinsta.cloud and DevOps Chat. Have a great day, everyone, and we’ll see you soon on the next chat.

    — Alan Shimel