devopsy.kinsta.cloud

Category: DevOps Chats

  • DevOps Chats: Kubernetes is the New Compute, with Rancher Labs’ Sheng Liang

    DevOps Chats: Kubernetes is the New Compute, with Rancher Labs’ Sheng Liang

    Rancher Labs is a great example of a startup that was smart enough to sense the way the market was moving and adopted a strategy to capitalize on it. And, if you know Rancher co-founder, CEO Sheng Liang, you should not be surprised. Sheng has been seeing the market and staying one step ahead for a long time.

    In this DevOps Chat, we sat down with Sheng and spoke about the recent momentum at Rancher. We also spoke a lot about Kubernetes becoming the industry standard for compute.

    It is always a good learning experience speaking with Sheng, and this conversation is no different. Have a listen.

    Transcript

    Alan Shimel: Hi everyone, this is Alan Shimel for devopsy.kinsta.cloud and you’re listening to another DevOps Chat. I’m really happy to be joined by a good friend of mine who I’ve known probably more years than we want to admit to, my friend Sheng Liang who is the CEO of Rancher Labs. Sheng, welcome.

    Sheng Liang: Thank you very much. I’m glad to be here.

    Shimel: I’m glad to have you here, my friend. So, Sheng, the news with Rancher, I guess we call it a momentum release, right, you recently put out.

    Liang: Yeah.

    Shimel: Why don’t you just share some of the highlights with the audience?

    Liang: Yeah. Every January, early February, we tend to look back the year before and see how we’ve done, how the whole industry has done, and it’s just been remarkable. I guess the top-level thing is our business, our recurring revenue grew 169%, so that’s like high three-digit growth. It’s way exceeded our expectation. That itself is good news for us, but it’s not really–I don’t think an average person in the industry needs to care about it as much.

    What I want to, Alan, spend some more time talking to you is some of the underlying technology changes and industry movement that’s kind of enabling us to thrive in this environment.

    Shimel: Absolutely. Let’s be fair, congratulations. A hundred and sixty nine percent year-over-year growth is nothing to sneeze at. You’re right, there is a little bit I think of a rising tide lifts all boats when we’re talking about the Kubernetes market. Nevertheless, 169% is still 169%. You and I both come from that startup world and congratulations.

    Liang: Thank you.

    Shimel: It’s something to be proud of.

    Liang: Yeah.

    Shimel: Also, for our audience out there who are not familiar with Rancher Labs and Rancher OS, it’s a great startup story. Sheng, your background, cloud.com, all the things you’ve been involved in, you’re the perfect person to have led this. But Rancher Labs came out there originally with the Container Revolution, right?

    Liang: Yes.

    Shimel: With an OS for running containers. You were agile, nimble, adaptable enough to see Kubernetes and say, “Hey, we’re not gonna–we’re shoveling sand against the tide, right, if we think we’re gonna beat Kubernetes. If you can’t beat them, join them.” You incorporated Kubernetes into the Rancher Lab system.

    Liang: Yes, and in a fairly unique way as well. I’d love to share the story a little bit if you don’t mind.

    Shimel: No, you’ll tell it better than I will.

    Liang: We’re just over five years old, over 200 people now. We operate in 14 countries and we’re open source business. It’s 100% open source. Every single product we ship is available free in source code form, Apache open source, and it really worked well for us. I know in the industry there’s actually been a lot of talks about “Is open source even a great business anymore in the context of cloud?” But we continue to believe so, at least especially in the industry we’re in, and we can talk a little bit more about that.

    But let’s talk about the business and market and Kubernetes. The great opportunity we see with Kubernetes is Kubernetes for once has become an industry standard for compute. Now, think about it. We all know compute these days is everywhere. Computing is on cars, in ATM machines, on your phone, certainly on the server, in the cloud, but really, before Kubernetes where the world was getting into was quite challenging for a lot of organizations.

    Because these clouds as good as they were, as good as they still are, they’re just different. If I’m a CIO, say if I wanna be efficient, I wanna maybe go all in on one cloud whether it’s Azure or Amazon or Google, but until the day comes, now I merge with another company and now bringing a whole different set of infrastructures, and I don’t really wanna worry about that because my whole point is develop the apps, transform the business, help the–all the stuff I’m doing to worry about infrastructure is just plumbing, and that’s the true power of Kubernetes.

    I don’t think enough people in our industry are promoting Kubernetes this way yet, so I’d like to make a strong point with you, that in my mind, Kubernetes is the TCPIP of compute. Now, think about it.

    Shimel: It’s the stack. It’s the [Crosstalk] as you say. It’s the protocol.

    Liang: It’s a stack and a protocol because before TCPIP if you and I could remember back in–

    Shimel: I do remember.

    Liang: There were a handful, almost a dozen different network protocols, and after TCPIP we had 3Com, we have Cisco, we have Yahoo!, we have Google. All that stuff happened, internet. Revolution happened because finally–

    Shimel: I remember when you had to add the TCPIP stack separately. It didn’t come with Windows, so you had to add that in. That was an add-on. Early open source stuff.

    Liang: That’s exactly what’s happening because–this is a great point you brought up. Someday people will say, “Oh, I remember there was a time I had to talk to vendors like Red Hat and Rancher just to implement Kubernetes, whereas today if I get infrastructure like from Amazon, Google, from VMWare, from Dell, wherever, it comes with Kubernetes.” That’s exactly the same thing that’s happening. You see that, right?

    Shimel: Yeah. History repeats itself. When you get old enough, Sheng, you realize that. Let’s talk something else. I wanted to talk a little bit about CNCF because that’s their mission to make sure that Kubernetes is part of the plumbing. Truthfully, it’s not just Kubernetes. When you look at CNCF’s mission, they truly are trying to develop a stack or at least help foster the development of a stack built on several open source projects.

    Beyond Kubernetes, for Rancher now, what are some of the other CNCF projects that you see making their way into Rancher either now or in the future?

    Liang: I would say pretty much the whole stack. We’re very deeply involved in CNCF. We’re very early member of CNCF, and my co-founder Shannon Williams is elected to the CNCF governing board. Just this week I was elected to the CNCF TLC.

    Shimel: Really? Congratulations.

    Liang: So, I have some power kind of deciding what projects are gonna go into CNCF. CNCF fundamentally is a governing body for projects, and really our goal is anything and everything that–we wanna provide basically the complete software stack that organizations need to complete their cloud native transformation. Kubernetes is really just the start. Beyond Kubernetes you see everything else that’s going on from monitoring to logging. I’m talking about projects like Prometheus and ____.

    Those are some very mature projects now. Some of the more recent projects, things around security and policy management like OPA. I think those are very, very–

    Shimel: Love OPA.

    Liang: Yeah, and service match Linkerd. That’s also part of our CNCF. We last year contributed to a project called Longhorn. It’s cloud native storage, Kubernetes storage into CNCF. It’s a CNCF sandbox project now. So, it’s pretty much really the whole stack. If you think I was saying Kubernetes is TCPIP, then the rest of the CNFC, it’s the rest of the software stack, from the platforms to some of the key applications to some of the tools and equipment that more people have used.

    They’re gonna build this cloud native ecosystem. So, I’m really excited about it. I’ll be serving this CNCF technical oversight committee for the next two years and play a role in really growing the scope, further grow the scope of CNCF.

    Shimel: I just wanna say one other thing about CNCF, and that is we did a virtual event here called Predict last month where we took a look at a year in the future, and one of the points I made is that to me, CNCF is sort of the poster child for this third generation of open source that we’re in. I call it the first stage, sort of the cathedral in the bazaar back in the day. I know you know about it, Sheng.

    And then we moved from the cathedral and the bazaar to what I call the big brother era where every open source project was really owned by one company, and everything that that project did was for the benefit of one company, and that discouraged others from joining in and helping do it. Now we’re in sort of the foundational era of open source where you have foundations managing these projects for the good of the whole community with multiple vendors involved, people like you on the board who are doing it.

    Yes, it’s good for Rancher, but it’s good for all of the community, and that’s a very different–as I said, it’s the third generation of open source and it’s a great thing. Let’s focus in on Rancher a little bit though, Sheng.

    Liang: Yeah, absolutely.

    Shimel: So, we’ve got all these great projects. You’re literally building the stack or at least distributing the stack, your distribution of the stack. Fantastic revenue growth. What are people doing with Rancher that this demand is in such high demand?

    Liang: Yeah, I know. It’s one thing that Rancher of course supports and distributes a lot of CNCF technologies including Kubernetes and Prometheus and Linkerd and Fluentd, and we’re gonna soon be doing OPA. That’s all valuable, but fundamentally, a company like us really only have a reason to exist if we innovate. We basically create things that people can uniquely get from us and not from anywhere else.

    There are two pieces of content or two technologies that we created that actually help put Rancher on the map. One is K3S. It’s one of the most–if not the most–popular open source project in the Kubernetes ecosystem today outside of Kubernetes itself, and it’s just got tons of momentum. What K3S achieved was it took Kubernetes that was already proven in data center and cloud, and we brought it out to Raspberry Pi’s and surveillance cameras and set top boxes.

    Shimel: This is K8 lite. I remember we spoke about this, the one in the KubeCons. I remember.

    Liang: We launched GA at KubeCon, so it’s very big. We’re very excited about it. You remember, again I keep using TCPIP as an example because in the 90s when TCPIP just came, it was obviously a computer networking standard, but nobody probably could’ve imagined like a phone would also be talking TCPIP, right?

    Shimel: They all do.

    Liang: They all do now. I can see in ten years every phone, every camera will be running Kubernetes as well. Every car will be running maybe multiple Kubernetes. So, we’re very excited about that. Then the other big piece of innovation that really Rancher helped to make it happen is this thing called multi-cluster management. We believed Kubernetes again is a protocol. It runs everywhere, so you kinda need something to tie them all together.

    It’s not really good enough to be able to operate individual clusters. You need to treat all the clusters as a resource pool. In fact, you actually kinda have to treat every single cluster as a cattle or as a pet. That’s where the Rancher analogy came in.

    Shimel: I remember talking to you about that three, four years ago.

    Liang: I know.

    Shimel: It’s cattle, not pets.

    Liang: Our namesake product Rancher, which is named after our company, is a Kubernetes management platform. It takes all of your Kubernetes clusters in the cloud. In the cloud, that could be GKE, EKS, AKS. Could be Kubernetes clusters on-premise. It could be RKE. It could be open shift. It could be Project Pacific, PKS. It could be K3S clusters on the edge in cameras.

    We present it to your DevOps teams and ITOps teams as a unified computing platform, so you can manage access, you can deploy apps, you can run CICB, and that’s what Rancher is. It really started to resonate last year, even though we’ve been building it for three, four years, and that’s ultimately what propelled our tremendous business growth last year.

    Shimel: Great. Sheng, we’re coming up on the end of our time. I’m gonna throw one more kind of topic question out at you we can discuss, and that is really what does the future hold for Rancher not necessarily from a product deployment point of view but you’ve started and had exits with multiple startups now in your career. Where do you see Rancher in two to three to five years? Do you go public with this? Is this something maybe some larger entity wants to acquire to kinda own the stack if you will? What do you think?

    Liang: I think these days I probably think a little less about ultimately how Rancher will exit, just making sure we build a great business. I think we’ve kind of grown past the point that sort of an early stage easy equal hire or a technology would be viable.

    Shimel: It’s not a tuck in.

    Liang: We’re big enough to be sustainable by ourselves, and honestly, I absolutely believe it’s a big enough market. We solve a big enough problem that there’s a potential for tremendous business. Think about people actually all talk about hybrid cloud and multi-cloud, edge computing, but I think a lot of these terminologies are very attractive from a market demand or business case perspective but always lack some technical meat to actually bring it to reality. That’s actually been the problem.

    Of course, we all know hybrid cloud is a real business need, but the problem is because there was no fundamental consistent experience that could actually implement that. So, as a result, no big businesses have been built. We kind of approach it a little bit differently. We’re very much centered around Kubernetes as a common protocol for compute.

    I think by building on that, I really believe there’s a tremendous business opportunity here because fundamentally, we’re gonna solve these multi-cloud, hybrid cloud and edge computing problems. Those are business problems with Kubernetes as the common compute protocol. So, I think that will keep us busy for the next five, ten years, and I think this business has lots and lots of room to grow.

    Shimel: Let the chips fall where they may. Sheng, we’re about out of time. I wanna thank you for taking time out of your busy day and joining and talking to our devopsy.kinsta.cloud and Container Journal audiences. Always a pleasure to speak with you and get an update on what’s happening at Rancher Labs. I will see you in about a month and a half in Amsterdam for KubeCon.

    Liang: Thank you. Thank you, Alan. It was great talking to you.

    Shimel: Always a pleasure. Sheng Ling, CEO, co-founder Rancher Labs. This is Alan Shimel for devopsy.kinsta.cloud and Container Journal, and you’ve just listened to another DevOps Chats.

    — Alan Shimel

  • DevOps Chats: XebiaLabs/CollabNet Version One Merger Executive Discussion

    DevOps Chats: XebiaLabs/CollabNet Version One Merger Executive Discussion

    In this special DevOps Chats Video Edition we dive into the XebiaLabs/CollabNet Version One Merger. Joining us for this video discussion is the CEO of the new company, Ashok Reddy and President and former XebiaLabs CEO, Derek Langone.

    Ashok, Derek and devopsy.kinsta.cloud editor-in-chief, take a deep dive into the reasons for this merger and what it means in the end-to-end DevOps market.

    Have a listen for some great background on this merger and what we can expect to see in the market soon. Below is the transcript of the conversation if you would like to read along.

    Transcript

    Alan Shimel: Hey, everyone, this is Alan Shimel for devopsy.kinsta.cloud, and you’re listening to another one of our—actually, our video DevOps Chat edition. Thanks for joining us.

    We have a great DevOps Chat video lined up for today. As many in our audience know and in the DevOps community, last week was a pretty exciting week with continued consolidation in the DevOps space. The big story was a merger, acquisition, private equity—I’ll let them define it, but the joining of XebiaLabs with the CollabNet VersionOne team, financed by a private equity company. And then kinda the surprise cherry on the cake was the addition of a CEO, a pretty well-known name to us in the DevOps community for both his days at IBM, CA/Broadcom, Ashok Reddy, as CEO. And we’re lucky enough to have Ashok Reddy, CEO of the new company. I’m gonna let Ashok handle the name.

    And joining Ashok is a friend of ours who needs no introduction in the DevOps space, our friend Derek Langone. Derek is the CEO of XebiaLabs, you know, one of the main companies in this merger, and Derek will be the president of the new entity going forward.

    So, Ashok, Derek—welcome to DevOps Chat. Thanks for joining us.

    Ashok Reddy: Thank you, Alan. Great to be here. Thanks for inviting us.

    Derek Langone: Hi, Alan.

    Shimel: Yeah—no, it’s great. Okay. So, guys, let’s jump into this. Ashok, as I said, you need no introduction in the DevOps space. You’ve accomplished, between mainframe and DevOps and IBM and CA, you’re an accomplished executive in this space. What motivated you to come on board here as the CEO of this new entity, and also would you mind—I don’t even know, I’m not sure what to call the new entity. Should I call it Xebia, CollabNet, some Newco? I’m not sure. So, go ahead.

    Reddy: Yeah, so start with that. I think we are working on, obviously, we’re doing a branding initiative. We’ll be coming up with a new name shortly.

    But as well, Alan, I think in terms of, you know, why did I join this is, it’s really, in the past, if you look at what I personally have done is focused more on transforming existing businesses, very large businesses, whether it’s in the Mainframe or in the large existing portfolios, and I think that was very good in terms of what I did there.

    But what I found here with TPG, as a private equity firm, which is very well known for transforming, the investments they make are really transforming industries. As examples, TPG was an investor in Spotify or Uber, an early investor in Airbnb and some other companies like Chobani and others where, they’ve kind of transformed the industry. So, when they had this thesis around DevOps is one of the most important transformational things going on in the industry, and there’s really, it’s not just a consolidation, but really, coming up with a true platform all the way from end to end, from how do you define value to how do you measure value and how do you plan and track and deploy and secure?

    To me, it was more of a sort of just transforming existing things to build out a new platform with, starting with some of the best of breed companies, right, technologies. So, to me, that was really what the attractive part is. Plus, you know, we can define—there’s no boundaries in terms of which industries and customers we want to, but really, truly focus outside in from the market perspective, and every company is becoming digital, everybody talks about that, but I don’t think we have a platform which really provides that.

    So, I think that’s really why I changed. And also, you know, in terms of the naming, as I said, we are still working on something which is, represents the broader platform we are building and not just the things, what we have today.

    Shimel: Fair enough. Derek, I’m gonna ask you a slight—I’m gonna ask you the same kind of subject, but slightly different. Hey, man, you’re CEO, you know, you’re the boss, and now you’re president of the new company which obviously still covers a lot of territory, a lot of responsibility. But, you know, as an organization, why did you feel it was best for Xebia to do this, and then you personally, right? Is this—where do you see, you know, what gets you excited about your role in this new adventure?

    Langone: Yeah, so, I’ll try to answer that a couple ways, Alan. First, from a business perspective, why this made sense—I mean, as you followed our progression at XebiaLabs over the years, you know, we’ve established ourselves as a premier vendor to large enterprises and helping them, you know, sort of realize real release orchestration and deployment automation and now reporting on important aspects of the software life cycle.

    And we’ve been able to gather meaningful clients, you know, the largest banks, insurance companies, airlines, health insurers use our platform because it’s so scalable and so robust. However, the feedback we got from customers quite often was,  “Hey, you know, you really only own a small slice of my life cycle, and what is sort of adjacent to you is a tapestry of individual tools that we’ve woven together, and it would be great if you had a wider footprint.”

    So, the opportunity with TPG, looking at the market, and the opportunity at the very, very top of the market in the Global 5000 to create an enterprise scale platform gave us the chance to consolidate with CollabNet VersionOne to put two best of breed technologies together to have much more of a horizontal influence on the software life cycle and the power of those two pieces together really does give you that formula of one plus one equals three or greater. So, that was what was exciting from a business perspective.

    For me personally, it’s terrific to be able to have Ashok as a leader in our organization and someone that I can learn from. And you’re absolutely right, I’ve been the CEO, so I’ve sort of been the boss. However, I’ll tell you, it’s an easy transition, for two reasons. Number one, Ashok’s a great leader with an incredible track record and a skill set that I would like to learn from. And then, number two, I haven’t been the boss at home for many years, so it was sort of an easy transition to not being the boss at work.

    Shimel: [Laughter] Amen to that, brother. I’m with ya. We won’t tell anyone—no one’s listening, though, don’t worry.

    So, we’ve got this new entity now, which represents the assets of CollabNet VersionOne, XebiaLabs kind of put together. Ashok, you mentioned sort of best of breed technologies within their places.

    Let’s lay it out for our audience, right? Where are the strengths of this new entity? Where is the best of breed functionality? You know, when we look, I call it the end to end DevOps kinda market, right, the players who are trying to do end to end DevOps. No one has the whole enchilada yet.

    But as you sit here today, gentleman, where do you see the strengths? Where are the real assets of this new entity today?

    Reddy: Yeah, no, I think it’s—so, let me start with, you know, when you start with the overall, it’s really, the focus is value stream management, for example. Companies who look at how do I measure value; how do I look at agile? From an agile planning perspective, I think there is, VersionOne is best of breed in the sense it was built ground up to be almost like a reference architecture for the same framework. If somebody is adopting the new, say, 5.0, for example, updated where it’s got value streams and how do I drive value streams—it’s actually built for it. All the way from how do I blueprint things to how do I create epics and stories to be able to plan and do the resource planning?

     So, that’s one aspect of it, but then the second part is, even with CollabNet, you know, people kind of forget CollabNet started many of this open source movement around Subversion and it’s, 35% of the market today still uses Subversion.

    But the Git part, which is where I find not many people know that, but you have Git which is secure Git. Because many times, there is many vectors, the three security vectors in enterprises—if you used distributed Git, you don’t get. And something like CollabNet actually has implemented really strong security protocols and a centralized way of managing, so you can actually manage Git in a centralized way with secure. And the people like Intel, you know, tens of thousands of people use it.

    So, you have a system of record in agile planning and resources to the source control and version management in CollabNet and in the TeamForge SCM, which is a part of GitAgile. Then, of course XebiaLabs. I mean, I think, you know, we competed with XebiaLabs in the past, but it’s really in terms of how XebiaLabs was, you know, both from a release automation to release orchestration, because it can do it without actually having scripts and people—scripts break in a transformation. Things change. The technology is platform agnostic. It can run on any cloud, supports on any cloud. And just being end to end, and now they have complete visibility with AI and machine learning.

    So, I think those are the three elements which I feel like are the best of breed. And then there are some things which we, you know, obviously, we’ll organically develop and then we’ll acquire other best of breed technologies. But I would say those are the strengths is connecting all the way—it’s not just orchestrating releases, you can now orchestrate business outcomes in a very agile fashion.

    Derek, did you want to add anything else?

    Langone: Yeah, I think, Ashok, you covered most of it, and you also said something important, which is our goal, to be that software life cycle system of record.

    But really, Alan, if you look at these two pieces together, what VersionOne has done for the engineering side of software life cycle, giving you that velocity and that sort of iterative agile approach. We’re trying to apply to the entire software life cycle. So, take those principles and apply them to release orchestration, deployment automation, and collect data along the way to be used for value stream, for compliance reporting, security reporting—things like that.

    So, essentially, create this platform that does a couple of things—gives you the ability to increase your velocity, scalability and so forth in your software factory, but the important element to understand about our approach is, even though we’re building an end to end solution, it’s going to be reliant on its integrations with all the best of breed tools that have already been invested in by these big organizations, right? So, I won’t name tools, but they’re very common CI tools, planning tools, provisioning tools, ITSM tools. We have all those integrations built out of the box. So, we leverage all of those investments as part of the process, so we’re not displacing any technology. Quite the contrary—those technologies come along with our process, and it allows them to be a part of the solution of that system of record validation for software life cycle.

    Shimel: You know, and I think it’s worth mentioning—many of those tools, Derek, are actually open source projects as well, right? Because open source plays such a big part in this.

    You brought up, what both you and Ashok said here brings up kinda two thoughts in my mind. Number one is, no one just wants a collection of disparate tools. We all want things that work together, right? And Xebia actually—truth be told, I’m much more familiar with XebiaLabs than I am with the CollabNet VersionOne suite. Xebia did a great job of integrating with open source tools, right, as well as third party tools.

    I wasn’t aware of a XebiaLabs CollabNet VersionOne integration, though. Are there any? What are the plans for them? Let’s hit that first and we’ll go from there. I don’t know if that’s an Ashok or a Derek thing, or maybe it’s too far in the weeds.

    Reddy: Yeah, we’ll—I think I’ll just say that, you know, as far as from a VersionOne and CollabNet perspective, I think there has been, I think, as you said, XebiaLabs has done a fantastic job with this periodic table type of things, tools in the DevOps tool chain. So, as part of that, the whole integration on SCM, the Git with CollabNet’s support for the GitAgile, there is already integration.

    There are some things in terms of more of API level integrations between VersionOne and a couple of the other things at XebiaLabs, but we are now strengthening that. But Derek, you may have, you know, some things based on what we have done in the past with Xebia and the CollabNet VersionOne that you may want to add other things.

    Langone: Yeah. So, Alan, to your question about the integration that exists today between CollabNet VersionOne and XebiaLabs—that’s been a long standing integration. We have several customers in common that have benefited from those technologies together, and frankly, that was one of the validation points when we decided to become part of this merger, that we had already seen customers benefit from these two technologies together and felt like if we did that in a more purposeful way, we could built out a higher degree of value for these customers and benefit ourselves by expanding our functionality on sort of both sides of the balance sheet.

    So, I think you’ll see much deeper integration and a lot of our opportunity to build out that value stream thesis by building these bridges to other products, including, you know, these two companies’ products, collecting that data, and then providing that data for these purposeful value stream management reports.

    You know, I look at it from a couple of perspectives. The data that we are collecting from all the tools in the process is used to give high degree performance measurement metrics, KPIs, things like that. Some of the unanswered questions that CIOs have been struggling with for years are, “Hey, I’m spending $500 million a year on Devs,” and now, for the first time, they will have that. And then, for really practical reasons, a lot of these customers that we have—big banks, insurance companies, airlines are heavily regulated, so we’re able to give them chain of custody reporting, compliance reporting, security reporting, testing coverage. Things like that, that have largely been unanswered questions for years.

    So, I think the power of these two technologies together and that deeper integration will only serve to provide more and more value from the information we collect.

    Shimel: Excellent. So, guys, excellent answers from both of you, and I understand what you’re saying. I think one of the—it’s sort of a, you know, this balancing act that we’re seeing, maybe it’s because it’s early in the game for the end to end DevOps provider. Because on one hand, you have to be open—you know, you have to be able to integrate with this whole multitude of tools that have grown up, some open source, some not, within that DevOps end to end software development and delivery life cycle, right?

    You know, so that’s out there, and if my hands go off the screen, I apologize. But at the same time, part of the allure, part of the benefit that we would get with an end to end DevOps solution is that, maybe I could use less. I don’t have to have all of these different tools that I have—I have one suite, one platform, right, that gives me a lot of it, right?

    And I understand, Derek, you framed it well, and Ashok, I know you’re familiar with it is, there’s this balancing act of, you gotta play nice with everyone. Because all of these companies are using all of these different tools, but at the same time, you know that your end game has to be, “I’m gonna try to give you as much as you can from one provider.” Is that—you know, and it’s a tough balancing act, frankly.

    Reddy: Yeah. No, I think, to me, if you look at the DevOps landscape—yes, there are, some of the vendors would say they have built out a single application or whatever. But the reality of it is that, you know, we can actually do the same thing with the type of capabilities from planning to tracking to SCM and, you know, some of the release, and as we buy other companies. We build it out, it’s completely integrated, and we make that work, right?

    But the only thing is, it’s, in enterprise Global 5000, many people already are using different things, and it’s, what’s an application in an enterprise? It’s typically a multi-tiered application. Some of it will be, you know, maybe an Android aspects, there will be a mobile app in iOS. So, you have to use Xcode, you have to use Git for something, you have to use Jenkins and you may have to use now, in a multi-cloud, Spinnaker.

    So, I don’t think there will be a choice to just use something from one vendor on everything. But even if it did that, we are saying that we can still provide that entire end to end from one vendor, or you actually have an option to replace parts of it because you’re already using something, and things get updated, even open source. You said open source is everywhere, and it’s going to continue to change because of innovation. So, accommodating that, and then, is it compliant, is it secure? That’s a big aspect of it.

    And one other thing which I think is important, Alan, is it’s not—you know, from a DevOps and agile perspective, it’s not just about the tooling and the end to end. It’s really about the transformation and the cultural parts of it and coming in with more of agile cultures and others, which is what we do a lot, and customers start with that. You know, people can use all sorts of tools, they deploy a thousand times a day, but that doesn’t get them the business outcomes and how do I transform them to work? To me, that’s another aspect of it which we are to consider. Even if you have one tool, how do you change the culture, how do you get people to change in a constantly changing world?

    Shimel: Absolutely. Guys, I’d like to move in a different area, if it’s okay. We’ve spoken about sort of what the market looks like and what the strengths are. You know, we wouldn’t be fair and balanced if we didn’t talk about maybe what the weaknesses are or where. And I don’t mean weaknesses, but where—where do you see places where this new entity can improve from the current lineup? Whether it be building organically or buying, right? You know, it was—at the announcement of the merger, it was made pretty clear that this isn’t gonna be the last acquisition or merger, that there were funds set aside for more acquisition activity. I assume you’ll also be building holistic additions to the platform in the suite.

    In what areas can we expect to see additional functionality added here?

    Reddy: Yeah, I think it’s a great point. I mean, we—I mean, you start with the life cycle. I mean, obviously, in the front end, you know, we have the whole notion of planning and then how do you actually track things and how do you deploy. But then, you know, the quality is a big aspect of it, right? The whole continuous quality, the continuous testing is something we obviously can integrate today with many of those things, other products and open source technologies. But if you think about from a platform perspective, I do expect, you know, some sort of continuous testing and quality as a key aspect.

    The other one is, you know, security. Because I think, especially with applications and the more and more data driven applications, security is a key aspect. And some form of more feedback—we have some feedback loops built into every aspect of our platform, but I do say from a customer experience monitoring perspective, being able to bring in some of the feedback loops is important.

    So, Derek, do you have any other thoughts in terms of the various areas we could improve based on some of the customer interactions you’ve been having?

    Langone: Yeah, Ashok, I think you hit on the fundamental part to DevOps transformation. So, I think that, you know, we’ve got an eye towards that functionality, and whether we have, you know, sort of an inorganic, acquisitive approach to that or primarily a partnering approach. I think we’ll continue to kinda round out that functionality.

    And again, it has to dovetail back with our thesis. We are building a platform for the enterprise that has all of the elements required of enterprises, right? These businesses are heavily regulated, so security is front of mind, quality of what they release out to their customers and consume internally is top of mind, right?

    So, again, when we look at how we build out this platform, there’s gonna be a few elements to it, but it will all be towards the approach of giving best of breed functionality and then always staying open, because we are gonna have to work around investments that have been made in the enterprise.

    So, we recognize that this is not the end of the story and we’re moving aggressively to round out the story from a product and messaging standpoint, and that’ll be an endeavor that happens in earnest throughout the rest of this calendar year.

    Shimel: Fantastic. I’ll just add one thing to that—Ashok, I long for the day that when you say, quality, we mean security. Right? Security has to become synonymous with quality, and then, hopefully, we’ll really start achieving what we need to achieve in security—not just in DevOps, but in IT in general, and in business in general.

    Guys, I wanna thank you. We’ve only got a few more minutes, here. I wanted to just kinda bring us towards our end goal line with this thought and get both of your opinions on it. You know, you’re talking to the devopsy.kinsta.cloud community. It’s probably the largest singular focused community on DevOps in the world today, worldwide, right? It’s a worldwide community. I want you to look in the camera and talk to the audience. As DevOps practitioners, why should they care? Why is this new entity important to them? Why do they need to take a good look at this?

    Reddy: Yeah. I mean, I think, you know, part of the—I mean, I’ve worked with the DevOps community for many years now. And I think, you know, I find they’re really the, what I call, the digital athletes of the digital transformation, right? Because I think, when you look at the DevOps engineers and the community, they are the ones who are helping change the companies they work for. I mean, everybody is trying to become a digital business. They are the heroes—I mean, I use the Nike analogy of just get it done, just get IT done. And who does that? It’s the DevOps community.

    And so, what happens is, it’s the ops, to me, is the constraints enterprises have, which is, I want to go fast, everybody talks about going fast and speed, but you need a direction. But you have the constraints of, you need to still be reliable. SRE is a big aspect. So, I want to shift left. So, you want to be compliant, you want to have quality. So, you can’t just go fast.

    So, the challenges with what DevOps engineers are trying to do is, how do I go fast, but still have the reliability, the quality, the security, and still transform the companies they’re trying to do and remove the constraints for the business to go faster in an ever changing world? To me, that’s what we provide with this platform is really to help them make much more productive and make sure that we remove the constraints. They can—even if they are working for a large enterprise, they can still have the benefits of being a born in the cloud company and not have the constraints.

    That, to me, is one of the fundamental things. And they are the digital athletes of the world. And so, to me, that’s really what we are here for, to support the chief digital officers to everybody who is transforming, but the DevOps engineers of the community is who are making this happen.

    Shimel: Fair.

    Langone: That’s a great point, Ashok. And I think, Alan, to just add a little emphasis—you know, what we’re trying to build is a platform that provides functionality that’s complementary to the activity that’s already going on today, right? So, you had mentioned it earlier that, you know, as you’re making these changes inside an organization, a lot of it is sort of people changing their day to day routines, right? So, you know, being more agile, following new processes and things like that.

    What our goal is, is to provide functionality that a lot of the change has abstracted from the user, right? So, in other words, DevOps engineers continue to use the same tools that they were using months ago and weeks ago in the same way, but they plug into a process that’s largely abstracted from them, but gets all the benefits that—delivers all the benefits that an enterprise needs, right? So, to try to make the change and the transition that’s required to really embrace DevOps as, you know, as lightweight and as little impact on people’s days as possible.

    So, that’s really the goal, is to provide functionality that has not existed before, and do it in a way that doesn’t have to require changes of behavior.

    Shimel: Fair enough. Ashok Reddy, CEO of CollabNet VersionOne Xebia, until we find a different name; Derek Langone, president of the entity as well—thank you for joining us, both, today on DevOps Chat.

    We’re gonna—you know what, we’re gonna be watching. This is a great story, it’ll be interesting to see how it progresses, right, both in terms of mergers and acquisitions activity as well as in terms of just—hey, man, this end to end, as I wrote about last week, this end to end DevOps market is open for the taking. There is no one, I think, who really has that whole story yet. It’s a developing thing, and it’ll be interesting to see, you know, how the market in our community helps pick the winners here as well as what you guys do with it, right? It’s yours for the taking.

    Best of luck to both of you and to the combined company.

    Reddy: Thank you, Alan.

    Langone: Thank you. Good to talk to you.

    Shimel: Great talking to both of you. This is Alan Shimel for devopsy.kinsta.cloud. Thanks for joining us on this DevOps Special Edition Video Chat. Stay tuned, and we’ll see you soon.

    — Alan Shimel

  • DevOps Chat: Increasing Delivery Velocity, with Launchable

    DevOps Chat: Increasing Delivery Velocity, with Launchable

    Jenkins founder Kohsuke Kawaguchi (KK) and respected DevOps veteran Harpreet Singh have launched a new company called Launchable.

    Launchable aims to help you improve your delivery velocity by prioritizing and applying some ML and intelligence to testing. There is still much work to be done but with these two industry luminaries behind it, the expectations are high for something impactful.

    The VC community and a blue chip list of investors are already on board.

    Yes, this means KK is no longer at CloudBees (several CloudBees execs are investors in Launchable) and Jenkins, while he is still an advisory. But, with Jenkins now in the capable hands of the CDF and Linux Foundation, and CloudBees well established, he felt this was a good time to pursue an issue he thinks needs to be solved.

    Hear from KK and Harpreet themselves in this great podcast.

    Transcript

    Alan Shimel: Hey, everyone, this is Alan Shimel for devopsy.kinsta.cloud, and I am here, you’re listening to another DevOps Chat. We have some really great breaking news on this DevOps Chat. I’m really happy to have, snagged a dynamic duo of industry veterans in the DevOps space who have just started what promises to be perhaps a game changer in the DevOps industry. But they’re gonna tell us more about that, I don’t know.

    Let me introduce you to Harpreet Singh.

    Harpreet Singh: Hello.

    Shimel: Hi, Harpreet, how are you?

    Singh: Very well, thank you.

    Shimel: Thank you. And Harpreet is co-CEO of the new company, and his co-CEO is none other than my friend, Kohsuke, K.K., founder of Jenkins, CloudBees and all of that good stuff. K.K., welcome.

    Kohsuke Kawaguchi: Hello. Thanks.

    Shimel: Okay, so, let’s talk about the big news. We have a new company in the DevOps space called Launchable.

    Kawaguchi: Yeah.

    Shimel: Is that—and the both of you are co-CEOs?

    Singh: That’s right.

    Shimel: And, of course, with your track record, I’m sure, you know, it wasn’t as hard as it is for a first time entrepreneur, but Launchable’s already raised some venture capital to help it get off the ground, including some high profile angels.

    Kawaguchi: Yeah. We’ve been very lucky.

    Shimel: Talk a little bit about it—who’s involved, who’s put some money in?

    Kawaguchi: So, there are two big institutional investors. One is Barclays Ventures, and that comes from Harpreet’s long relationship with their partner, Gomesh. I think, in fact, your wives know each other. Like, it almost feels like an insider. [Laughter]

    And then another VC is the unusual ventures, and that’s started by Unusual Ventures, and that’s started by John Vrionis. And John actually has been also an investor at CloudBees, so his people have seen us in action and know us well enough so, you know, you get their validation and then you endeavor it. It’s been—I’ve been truly grateful.

    Shimel: Sure. Well, I’m familiar with Battery back in my day, it was a guy named Gerry Conlon out of Battery Ventures. I don’t know if either of you know Gerry, but he was instrumental early on in Battery. But yeah, they’re a great company. It sounds like two, you know, kind of blue blood venture capital firms in there.

    There’s also some high profile angels including Sacha—Sacha Labourey, CEO of CloudBees, right?

    Singh: Yeah, yeah. So, I think what was exciting for us as we were talking with people was just, you know, the phenomenal backing that we got from investors as well as—you know, these marquee investors as well as individual angels, right? So, we both have been at CloudBees.

    So, I don’t like to think of ourselves as first time entrepreneurs, because we built CloudBees from less than 10 people to whatever it is today.

    Shimel: Yep.

    Singh: I had a sojourn to Atlassian, where I spent about a year and a half really learning from that phenomenal company. So, we were very lucky when people from CloudBees like Sacha, Michel—Michel Goossens, who was the VP of sales there at some point came in and invested. Bob Bickel, who’s been an industry veteran invested in us. And on the Atlassian side where the CTO is Sri Viswanath and Noah, who’s the head of tech teams, which is Jira, Bitbucket, Opshimi and all these comfortable investing in our capabilities and our mission.

    Shimel: Absolutely. And make no mistake, you’re not first time entrepreneurs, because first time entrepreneurs don’t raise money that easily, guys.

    Singh: [Laughter]

    Kawaguchi: [Laughter]

    Shimel: I mean, I think there’ll be a lot of people who listen to this podcast who are first time entrepreneurs and say, “Man, if we only had their background and their resume and their experience, it wouldn’t be as hard as it is.” I think a lot of first time entrepreneurs struggle—struggle with doing that.

    But that’s great. It’s always good to have money in the bank when you start a venture, because it’s needed, but as we were talking about off camera a little bit, I’ve never met a successful startup entrepreneur who did not think what they were doing was in some way changing the world, making it better.

    And so, I wanted to ask each of you individually—what really got your juices flowing? What really got you excited about what you guys are doing at Launchable?

    Kawaguchi: Sure. Alright, so I’ll start. So, you know, one of the things I started—I created this sort of thinking and I felt like I put the state of the automation and the software development power to the next level in the world. And this automation, they produce a lot of data. And what I realized after a while was that, you know, in the rest of the world, in the rest of the business world, data is increasingly being used effectively to drive decisions and make people more efficient and so on.

    But whereas in the software development, I feel like that actually hasn’t really happened. And I felt like this was a great shame, one as somebody who, you know, helps people produce a lot of data with these automation systems, but also, you know, we are the number people, we are the data people, right? We are the pride of ____, at least I think of it that way.

    Shimel: Mm-hmm.

    Kawaguchi: Yet, everywhere I look, I feel like we are making decisions, engineering decisions based on gut and instincts, and this can’t continue. And so, that’s what I kind of started thinking about after a while, and then this idea kind of grew and grew on me. So, yeah—so, that’s probably where my, how my juices got flowing.

    Shimel: Excellent.

    Singh: Yeah, very similar. I think Kohsuke and me used to sit next to each other in the same office room at CloudBees and we used to talk about this all the time. We got the opportunity to see sales and marketing really be completely numbers driven, and we would come back and say, “Oh, if we could only do this on our engineering product side of the house, it would be so much better.” And the answer always was, “Well, it’s a craft.” And we think bringing data in there is important.

    Shimel: Right.

    Singh: And so, that’s sort of what got us both going. My personal anecdote was, after I left and moved to Atlassian, I was talking to somebody out in my customer base there who had this challenge where their delivery velocity was slow and, you know, their aim was to put a few engineers to go in and look at it and how to improve it. And having the DevOps background and you understand CI/CD, I felt like—there has to be a better way to do this than just, “let’s allocate a few engineers and see what comes out of it.”

    Shimel: Right.

    Singh: So, that’s sort of the problem space, and that’s what excited us.

    Shimel: Excellent. And it is, it’s almost ironic or oxymoronic, even, when  you think about it, right?

    Singh: Right.

    Shimel: Because—so, my background’s in information security, right? And one of the things we have seen in security is information overload, like data overload.

    Singh: Right.

    Shimel: The noise to signal ratio. But there have been—there have been a lot of security startups over the last, let’s say five years, seven years, who have really tried to fine tune that dial to get more signal to noise, to be able to automate going through the mounds and mounds of log data, for instance, right?

    Singh: Mm-hmm.

    Shimel: So, you get a company like a Splunk that can just suck a lot of data in from everywhere. And then try to figure out, in an automated way—because there’s just too much to go through by hand—

    Kawaguchi: Exactly.

    Shimel: What’s important from a security point of view or maybe even from a network operations point of view. But as you guys said it, I’m thinking—we don’t really do that with our software development, do we?

    Kawaguchi: Yeah. And I mean, when you describe it like, it might sound a little abstract program, but idealized, it’s actually connected in so many ways to frustrations that I have every day as a developer, right?

    Let me give you an example. You know, I was working on this Jenkins project, and it’s a sizable project with lots of test cases, which is great, but what that means is that every time when anybody wants to make, like, a ____ change, the first thing that happens is, the CI system comes in and they run the whole one hour test cycle before the code review happens, and then somebody makes some observations or a suggestion that, “Oh, you should change this part to something else.” And then you follow that and then another whole hour of testing. And then that’s basically, like—two hours is a long time to wait if you get some change in, actually. And that happens all day in that area.

    Shimel: No, no—I mean, so, let me be a half glass full kind of guy. It’s better than—the one hour is better than the one day it used to be before we automated some of this stuff, right?

    Kawaguchi: That’s true.

    Shimel: I remember back in my StillSecure days and we would—we didn’t even have a big enough infrastructure to do the testing, we would have to outsource, like, unit testing and all this stuff.

    Kawaguchi: Yeah, so, let’s talk about that one big thing, right? Because that’s actually probably an even bigger program. And you said that that’s before automation, but I’ve seen teams every single day that actually have tests that run for more than a day, you know, [Cross talk].

    Shimel: With automation?

    Kawaguchi: Yeah, like an auto maker company putting a car together, a lot of big software. If you test them all, it’s in that cycle for weeks. So, there are a lot of pieces of software getting assembled together into something big, and that test gets even bigger.

    So, what I started doing is, like—okay, so, every time you do this, it’s not like everything is changing all at the same time, right? There’s only small parts of the system it’s changing, and then when we re-run the whole test cycle. So, what if we could ____ the right subset for that test to run? They’re most likely detached programs, so they’ll not only finish quickly, but they also run cheap, and then you still get, like, a pretty sizable, meaningful feedback. You know, you’re not gonna get 100% coverage, but you might get 95%, and that’s plenty good, and that cuts the execution down in, let’s say, half.

    And so, these are the kind of things I thought the data would actually enable, and I’ve seen some prior work, I’ve talked with teams who seem to be on the cusp of doing these things, and I felt like—yeah, this clearly makes sense. More and more people need it, and I need it. And so, that sort of became the founding idea behind Launchable.

    Shimel: So, that was the itch you were looking to scratch here, as I say, right?

    Kawaguchi: Yeah, right.

    Singh: And you asked, like [Cross talk]—

    Shimel: I’m sorry, go ahead.

    Singh: Yeah, and as we were sort of thinking about this, right, we identified this problem and we’re thinking about, like—so, what’s the company’s mission, right? And we realized that every sort of software teams in the future or even today will drive every significant human achievement there. It’s really hard to think about anything that’s not being software delivered.

    And our mission now is to take this artisan teams and actually make them data driven scientists to drive their velocity, you know, and deliver software confidently. So, that’s sort of our big itch and our big mission.

    Shimel: Absolutely, got it. So, let me play devil’s advocate a little bit. You know, one of the things we always used to hear—I heard it in security, I heard it in QA—was, “Look, we don’t have enough time to do a full test, full testing. Let’s just test 80%, 70%, 90%”—take your pick, I don’t care what number you pick.

    You know, Murphy’s Law says whatever you don’t test, that’ll be the thing, somehow, that breaks, right, or that causes a problem. And when we say, “Oh, my God, because we didn’t test that”—why didn’t we test that?

    Kawaguchi: Right. So—

    Shimel: “Why shouldn’t it get done?” Right.

    Kawaguchi: Yeah. So, if you think of testing in isolation, then yes, that’s naturally a concern, because, you know, we need every help and every confidence we can get out of the test. So, like, why would you skip that? That doesn’t make sense.

    But I think where it does make sense is actually, if you think about it, the software development life cycle is composed of a lot of different tests at a lot of different points. So, collectively, they should be seen as a whole system, right? I mean, this is things like—so, the code review, right? If that’s the only point in which you are getting tests run, then you want to run them all. But what happened is, after the cold gets reviewed and merged, then another set of the same tests will come in. So, it is really important that you run all the tests before the code review and spend one hour, or are you okay with 80% of the confidence that happens in, like, five minutes, right? Then knowing that the rest of the 20% get code immediately after.

    I think there is an argument to be made that, overall, you’d be better off, even if some program slips down into the ____, so long as it gets eventually caught before the customer sees everything, so.

    Shimel: So, I guess then that that begs the question is—can that be accomplished? And, to accomplish that, what’s it going to take?

    Kawaguchi: Mm-hmm, yeah, and that’s what we are working on. I think, you know, the—so, I have some preliminary numbers, and they look good. Because if you think about it, the baseline that we are competing against is basically people running all the tests in a random order, and so, literally, you cannot do any worse than that. So, and instinctively as a developer, I know that certain tests are designed to catch certain programs in some specific things.

    So, really, the only question is how well a machine learning model can learn these things over what amount of data. And that, over time, I think I’m sure is gonna get better and whatnot. And I don’t know exactly what kind of impact we can create or what product, but overall, I’m feeling pretty confident. You know, it just needs to be good enough to start making an impact, and it’s only gonna go up from there.

    Shimel: Harpreet, what does success look like for you?

    Singh: Well, this has been a question that’s been asked to us while we were raising funds. I like to think in terms of—so, we’ve done this before. We’ve helped companies get successful. What’s in it for me is actually solving a problem for customers.

    So, if you are able to get and solve a software team’s problems everywhere, to get their products in the hands of customers first—that’s success for me. I know it’s not sort of putting the typical Silicon Valley 100 million to 200 million ARRs. That’s not where we are. That’s not where my head is at. It’s really helping people who are struggling with delivering software.

    Shimel: Got it. K.K.?

    Kawaguchi: So, you know—

    Shimel: What about you?

    Kawaguchi: One time I was working in San Francisco Airport, and then somebody was, you know, working behind me and she said, “Oh, you got a Jenkins thing. That’s nice. Where can I get it?” Right? And that, to me, is like a success. So, it’s gonna be a difficult one to beat once more, but you know, I feel like it’s worth giving it a shot.

    I mean, that’s the kind of—I guess I sort of think that the sign that you made a big enough impact in the world, and some people, and touched some people in pretty deep ways. And that, to me, is success. That’s what I’m itching for, yeah.

    Shimel: Okay. Let’s—let me hold both of your feet to the fire a little bit. So, you’re now launched, this is—you know, it’s out there, the website’s out at Launchable Inc, that’s L-A-U-N-C-H-A-B-L-E-I-N-C dot com, right? When can people hope, maybe, to see a beta? When do you think you’ll be out with something people can actually start using?

    Singh: You’re really holding our feet to the fire, right?

    Kawaguchi: [Laughter]

    Singh: You have a product manager and an engineer, here. I would like my engineering head to give me the dates on deliverables. [Laughter]

    Shimel: You know what? Classic. You’ll make a [Cross talk].

    Singh: Boy, you and me on the same side, here.

    Shimel: That was a great CEO move right there.

    Singh: [Laughter]

    Shimel: Alright, Mr. co-CEO, he threw you under the bus. What do you think?

    Singh: Jokes apart, I think one of the things that we are actually doing at the moment are, we are actively talking to prospects and customers. A lesson that we’ve learned over the years is not build a product in a vacuum and then hope that we’ve solved the problem.

    So, we are actually—there is actually a beta invite list button on our website. What we are looking for is actually partners who can come with us and tell us if we are solving the right problem and work with us to solve the right problem, right? So, that’s sort of where we are today. I’ll leave it at that and I’ll put the dates to you.

    Kawaguchi: Yeah, so, you know, we have preliminary numbers, like I mentioned. We made some models that made some impact, and now what I’m really hoping to find is event partners where we can apply the same technique on their system, their projects, and their source code to see what kind of impact does it produce. And then, so that’s—you know, that may be fundamental in these teams who are willing to work closely with us, and—

    Singh: Is that some number that you can share on preliminary numbers?

    Kawaguchi: Yeah. So, I mean, there is no guarantee that it’s gonna apply to your system. But, for example, while I was able to do this, select 10% of the subset for the best case use and it’s gonna catch like 40% of ____ right off the bat. So, I guess, you know, that’s—yeah, that, to me, is a pretty promising number, and you can only go up from here, so.

    Shimel: Excellent. So, look, both of you have garnered a ton of good will, not only among your peers as evidenced by the angel, you know, the investors’ list here, but also in the community at large, and K.K.—I mean, it goes without saying within the Jenkins community how well respected and, I don’t wanna use the term worshiped, that’ll go to your head, [Laughter] but you know, how honored you are in the community.

    What, people maybe from the community listening, what can they do to help, here?

    Kawaguchi: Yeah, I think giving us their feedback probably is the most. You know, we are working on something that we think is pretty exciting, and then we wanna hear that, or if that’s not the case, then that’s also great to hear. So, so far, the conversations we’ve had with people who are interested in this space who are thinking about this space have been incredibly valuable.

    So, if—you know, if you feel like some of what we discussed here is you and your challenges, we’d love to hear from you. And also, we are hiring—wink, wink. So, you know. [Laughter]

    Shimel: Everyone’s hiring.

    Kawaguchi: Yeah, so if you—

    Shimel: Alright. Is it on the website?

    Kawaguchi: It is. You know, one thing I got a lot more passionate at, you know, CloudBees and Jenkins is sort of passing back onto the next people, right? I felt like I got a lot more joy out of talking to the junior engineers and folks, we steer and mentor them. So, I feel like what we can offer—and I know it’s true with Harpreet, too, so I feel like what we have right here is an incredible opportunity to work, you know, sit side by side with these people, you know, people like us who, I think ____ creativity. We’re also passionate about mentoring other people. So, you know, I think if it sounds like your cup of tea, then…

    Singh: I’ll jump on that. So, I think, CloudBees was a fantastic company. You know Sacha, Alan—he’s just a great leader. He built a fantastic company. So, we learned a lot of good lessons on what it means to be a good company.

    Then I ended up at Atlassian where, you know, I learned the same things from Scott Farquhar and the other folks, and we’ve tried to bring in all the great points from these two companies into this company, right? We are sort of values driven. We’ve really thought hard about the kind of company we want to build, the kind of impact we want to have in the world, and we are looking for people who share that sense of mission, right, to come on our team, right? And if you find some of those guys on our team, that’s fantastic.

    On the former side of the house, again, we are looking to solve problems for customers. We are not looking to do this in a vacuum. So, there’s a general profile of people who are like, “Yeah, I hear this. I wish I could partner with someone and help you define what—go solve this problem for me.” If you’re that person, we would love to talk to you and we’d love to partner with you through those things.

    Shimel: Fantastic. Guys, we’re way over time. I promised you 15 minutes. We might be coming up on a half hour; I apologize.

    But let me close off by saying—you know what? Congratulations. Someone once told me new companies are like boats—the best days are the first day you get ‘em and the day you sell ‘em. [Laughter] But that being said, it’s also a lot of fun sailing ‘em. So, enjoy the sail, guys—change the world. If there’s anything, of course, we can do here at MediaOps, you know you can count on it. I couldn’t think of two nicer people for this new adventure and to take on this challenge.

    Harpreet Singh, K.K.—co-founders, co-CEO of Launchable, you can find out more at Launchable Inc. This is Alan Shimel for devopsy.kinsta.cloud. You’ve just listened to another DevOps Chat.

    — Alan Shimel

  • DevOps Chats: Simplifying Digital Adoption, With WalkMe

    DevOps Chats: Simplifying Digital Adoption, With WalkMe

    WalkMe recently announced a funding round of $90 million, bringing its total raised to over $300 million. But according to President and Co-founder of WalkMe Raphael Sweary, the real measure of WalkMe’s success is its annual recurring revenue being more than $100 million (which it is) and whether the company is helping its customers.

    WalkMe’s customers are comprised of businesses undergoing digital transformations and looking to accelerate their digital platform adoption.

    In this DevOps Chat, Raphael and I discuss WalkMe’s plans for its latest funding, including plans for its SaaS-based digital platform. As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, it’s Alan Shimel for devopsy.kinsta.cloud and you’re listening to another DevOps Chat. You know what, we’re recording this—oh, December 23rd, here, getting near the end of the year. This is probably the last DevOps Chat of the year. So, what a great way to end our year of fantastic DevOps Chat with Rafael Sweary, president and co-founder of WalkMe. Rafael, welcome to DevOps Chats.

    Rafael Sweary: Thank you, Alan. I’m super excited being here. It’s one of the last days of selling in Q4, and you know how it is when you’re a B2B business. It’s a big quarter for us, so I’m actually feeling privileged that I can spend some time with you.

    Shimel: I appreciate you taking it, I know how crazy it is. You know, Rafael, just on the subject, that’s something I’ve noticed the last two, three years. Years and years ago, kinda the consensus was that especially here in the U.S., after Thanksgiving, things kind of shut down, people got in that holiday mode and business slowed up. But I don’t know whether it’s because we live in a more global economy now or just, things are that different. The last couple years, the last couple weeks of the year have been absolutely frenetic with people doing things and deals coming in and sales being done, right up to the last second, it seems.

    Sweary: Yeah. So, I’ve never felt the Thanksgiving effect. For us, it’s always, the last quarter of the year was strong. I think it’s because of budgets that companies have and they want to use it. And sometimes people have something on their desk that they want to come to closure with by the end of the year. So, next year, they can start building and start implementing the new software that they bought. So, I’ve never felt it, but I’ve definitely heard people talking about this phenomenon.

    Shimel: Yeah.

    Sweary: So, it might be, as you say, the globalization and everybody’s connected all the time. And so, we’re kind of hooked. We’re kind of—

    Shimel: Yeah. I agree, I agree. But Rafael, the big news is—well, the big news for WalkMe is, you guys have recently announced, what was it, a $90 million round?

    Sweary: Yeah, it’s a $90 million round. It brings the total that we raised to $307 million.

    Shimel: Wow.

    Sweary: We went into this funding, this funding we didn’t really thought about raising money, and we started entertaining with the idea—because we were very, very well-funded before and we’re selling a lot, so we are, we didn’t really, we’re not a startup that is in a dire need of cash. We weren’t in such a situation. We were very lucky there. And we were planning—we’re planning, we had, our hope is to go cash flow positive and start thinking about going public, et cetera.

    Now, what is happening is that, on one hand, we saw a huge opportunity to grow much faster than what we planned, because things were going really, really well with our larger accounts. And, on the other hand, there is a lot of uncertainty crippling into the market. So, you see some instability in the financial markets. Overall, it’s not—it’s not sustainable, but you keep seeing people are, you start seeing those articles about being, writing about the evaluations being too high, et cetera.

    And the second thing, you’re going into an election year. Now, uncertainty is not good for business, and those two things brought us into the option of possibly raising capital. We didn’t plan to raise so much money, but when you are dealing in such evaluations and when you’re raising money from later stage funds, they have a minimum that, below that minimum, they’re not interested in the deal, even if the deal is great. Just because they can’t, from a portfolio management, they gotta deploy a lot of money to every company. The minimum for Vitruvian, which is the fund we selected to work with, was 70 million, and then, you know, we have preempt rights, which was exercised by our existing investors, which made the round into 90 million.

    Shimel: Right. That’s crazy. So, you know, Rafael, as we were talking off mic, you know, we live in a bit of a crazy time—I don’t know if it’s crazy or if it’s just the new normal—but we live in a time where large numbers like this are not as unusual as it was when I was doing startups, you know, even 10, 20 years ago. And so, first of all, congratulations, alright, because I don’t care what anyone says. It’s hard raising money and it’s hard raising that kind of money, so obviously, you’re doing good stuff there.

    But I think the question for our audience is—okay, so now, we have another well financed startup here, raised 307 million, unicorn valuations. But what does it mean to them? How is this gonna change and affect their life, right? When we look at WalkMe, right, the digital adoption technology to help bridge the digital skills divide—

    Sweary: Yeah.

    Shimel: How are you helping people with that?

    Sweary: So, I want to first start with the unicorn. It’s a nickname I don’t like, I gotta say.

    Shimel: Fair enough.

    Sweary: And it’s overly inflated. So, when I get up in the morning, I’m not worried whether we’re a unicorn or not, right? [Laughter] What I’m concerned about is our stakeholders. Our stakeholders is our, first of all, our employees and our customers are in the number one place. And I wanna make sure that we’re providing a business for them and we’re working in such an environment where we make everything better for everyone, okay? So, this is my concern. My concern is not whether we’re a unicorn or not.

    By the way, speaking about that, I think that TechCrunch, I think it was TechCrunch, suggested to stop looking at whether you’re a unicorn or not, but over you over 100 million in ARR or not?

    Shimel: Right.

    Sweary: I’m glad to say that we are—

    Shimel: Good for you.

    Sweary: Well over 100 million in ARR. So, it’s not—now, let’s talk about, a little bit about digital adoption. So, for us as users, there is a lot of technology available, and we can use technology everywhere. The problem is that the technology is changing very, very fast, and for us as humans, we don’t change that fast. The common saying is that when practice makes perfect—but actually, practice makes a habit, and changing a habit is something that is very hard to do. You’re accustomed to do something in one way and to change how you adapt to these changes is something that takes very long.

    So, I don’t know if you ever moved from a Windows to a Mac machine or from—

    Shimel: Oh, I have. [Laughter]

    Sweary: You’ve been?

    Shimel: Oh, yeah.

    Sweary: So, you know that the first few days were painful, right?

    Shimel: Mm-hmm.

    Sweary: They were painful. They weren’t fun.

    Shimel: I know.

    Sweary: Right? It takes some time until you—so, Mac, I don’t know what the move you’ve done, but Mac might be much better than what you had before or Windows much better than what you had before. But when you make the change, those first few days, weeks are very tiring, you gotta focus very hard.

    Now, but this is a change you wanted to do. Nobody forced you to do this change, you wanted to do it. Now, when you go into an enterprise environment, what is happening is that you’re forced to make—you are forced by the company to make many, many changes to the way you work. Because the company implements a new software, because the company is, has a new process on an existing software, et cetera, et cetera.

    And what happens is that you find yourself overwhelmed with all those changes. You go to do your performance appraisal and the process changed or the application changed, and you find yourself learning it again and again and again.

    Now, this is where WalkMe comes in. WalkMe reverses the relationship between the user and technology. If today when a user uses technology, he has to focus and he has to think and understand what he needs to do in the software—we do the opposite. The software tries to understand the user and help him achieve what he came to do.

    Now, usually what he came to do is to do something specific. So, this is how we help on the user level. But the way we—so, when you are in a company and you’re an employee of a company, today you have many software packages and it’s very hard to track them and we help you to do that. But also important is what we do for the company itself.

    So, the general idea when you implement new software is to improve your business. Usually, companies don’t buy products for spite. Like, I might buy a jacket because I feel like it. But no company buys anything because they feel like it. They buy in order to do something.

    Now, usually the things that they’re looking to do is to sell more or to spend less or to reduce risk. Like, any expense that a business will do will fall into one, at least one of those buckets. Now, what companies buy most is technology. It’s the biggest expense after human resources. So, companies invest massive amounts of money in technology, but a lot of the times, they’re not gaining the benefits.

    So, let me dive a bit deeper, there. When a company buys or implements a CRM, what is their goal? Is their goal to replace the Excel file or the notebook that the salesperson was using? No. That’s not their goal.

    Shimel: [Laughter]

    Sweary: Their goal is to cell better, right?

    Shimel: Yeah.

    Sweary: Their idea is that if the salesperson will use the CRM correctly, they will be able to use AI to recommend which pipeline to focus on, on one hand. On the other hand, they would have better predictability so they can report to the market, and also the HR team will be able to look at it and understand if they need more employees or less, and the purchasing department will understand what supplies they need. So, the hope is that, with putting digital, the company will operate better.

    Now, what happens? What happens is that the salesperson, when he comes to the CRM, it’s not helping him. He’s struggling. And what happens is that he’s forced to use the CRM. So, what he does is documenting in the CRM. He actually continues to work on his Excel file, and he works on his notebook, okay? So, you’re not gaining all those benefits because it’s not predictable, it’s done retroactively, okay? It happened—once the action happened, okay, it’s a lagging indicator.

    And a lot of companies, a lot of time I talked to them and they say, “I have a great usage for my CRM.” And I said, “Look, if you wanna know if you have adoption for your CRM, go to a salesperson and ask him what he’s gonna close this quarter. If he opens the CRM, you have adoption. If he’s going into his notebook or his Excel file, he’s documenting, and you’re not gaining all the benefits.” And then it’s very easy to see that a lot of, for example, the pipe moves from stage two to stage eight, right? Because you’re not really keeping current.

    This is, of course, one example. The same thing is true about any type of professional applications that companies use, and even more on everything that is a digital workforce application. If it’s interesting, I can give you some more examples of what we see at our customers.

    Shimel: Agreed, agreed. You know, I’m sitting here listening to you talk, Rafael, and you know, you can’t help but put your own, how it fits in your—we went through this right here at MediaOps, right? We have a CRM tool, and I’m trying to break the people of using whiteboards and spreadsheets, you know? [Laughter].

    Sweary: Mm-hmm.

    Shimel: It’s just as you describe, right? All the reasons, all the motivations. But it’s also a process.

    Sweary: Yeah. How do you handle this process?

    Shimel: So, I’ve gotten older. I think I’ve become a better parent and a better supervisor. I used to try to force people and say, “You can’t use the spreadsheet, wipe the whiteboard, I want it all in the”—in our case, it’s HubSpot—“I want it all in HubSpot.”

    Sweary: Yeah.

    Shimel: And what I found is, you know, they would pay lip service to me and they would just move the wipe board out of the way or they’d have a spreadsheet that they didn’t show me, and then it would be like double work, Rafael. So, they would do it in a spreadsheet and then, to make me happy, they would kinda transfer that stuff into the CRM [Cross talk].

    Sweary: Right.

    Shimel: But a funny thing happened—as they transferred more and more into the CRM, the CRM became more and more valuable, right? So, you wanted, having things in there that you couldn’t put in there, just as you’re talking about.

    Sweary: Yeah.

    Shimel: You couldn’t put these things in a spreadsheet any more, right? Because there was more intelligence, if you will, built in there. And so, eventually, we started using it more. But—

    Sweary: Yeah. Now, think about this, about, with an enterprise that is global and has multi-branches and has a diverse workforce with different digital IQ. And no parents is there to tell them, “Put it into the Excel sheets and then—uh, put it into the CRM,” and then you start understanding the problem. Now, I assume that your CRM was stabilized.

    What is happening a lot of the times, it’s not stabilized, it keeps on changing because companies have new processes and then the process changes. So, for example, you keep learning from your business and you’re like, “Well, if it’s 30% discount, you need this approval process versus 15.” And now you add another complexity, and the salesperson is not able to work, right?

    So—and I think there is a solution. There is a solution. I think that what is stopping us from using technology to its fullest is the gap that happens between what the technology can do, what we can do as users, and how much, how long does it take until it gets better, right? So, anyone who’s ever used the GPS would never wanna go back to a map, right?

    Shimel: Right.

    Sweary: As soon as you experience it, it’s much better than anything you did in the past, do you agree?

    Shimel: I agree.

    Sweary: So, you don’t need to work—yeah.

    Shimel: I agree, I agree.

    Sweary: You don’t need to work—exactly. You don’t need to work on adoption. But when you’re talking about an enterprise application, there is a very big gap between the perceived value to the organization, and until you see the value and until it trickles down to the employees.

    So, let me take you through another example. So, a lot of companies are implementing HRMs, and they idea behind the HRMs is that the employee can self-task, right? You want vacation, you can take vacation. Now, in the past, let’s assume, let’s go back 20 years ago. An employee wanted vacation, he would fill out a form, it would go to his manager. His manager will scribble something, put it in his outbox, it will go into the HR department and then into the finance department and then back to the manager and then back to the employee.

    Now, all those people worked on something that didn’t add any value to the company. It’s just an employee taking vacation. So, the company has this great idea—we’ll put a ____ and when an employee wants to take vacation, he would see whether he has a balance or not. Once he asks for the vacation, if there is an issue and other people in his positions are already on vacation, he’s gonna get an alert, and if he doesn’t have the balance, he will get an alert, et cetera, et cetera. And you say, “Let’s automate this process.”

    Now, it comes down to the employee. The employee wants vacation. He doesn’t care about the process and how much time is spent, he wants vacation. He comes into this software, he doesn’t remember the log-in, because the last time he took vacation was a few months ago. And he goes in and he’s not sure how to ask for vacation and he gets stuck a few times and what does he do? He sends an e-mail to his boss. What does his boss do? Sends an e-mail to HR, that sends an e-mail to finance, that sends—and then you’re not gaining the benefits, right? Because you don’t see the value.

    Now, for the company, it’s super important that people use this application, because it makes sure that all the vacation days are deducted on time, and it really closes the loop, and less people are working on it. But you’re not gaining the benefits. Make sense?

    Shimel: Yeah, makes perfect sense to me.

    Sweary: Mm-hmm.

    Shimel: Perfect sense. Rafael, I wanna take a moment, because we’re running low on time—you know, we spoke about how much money and all of that and it’s all great. I wanted to give people a sense, though, of how many organizations you’re helping with WalkMe, right? And the amount, and how many applications you guys have done integrations.

    Sweary: Okay. So, customers, we have close to 2,000 customer logos, which means that when I say a logo, if—let’s say a company of ours, let’s say just as an example, Coca-Cola. So, we have, let’s say, Coca-Cola Greece and Coca-Cola U.S., it would be one logo, although there are two separate companies and that might be completely unrelated. So, we look at the logo level, and we have close to 2,000, including 30 percent of the Fortune 500.

    Regarding applications, WalkMe is platform agnostic. We really don’t care what application you use WalkMe on. We have some enhancements that are specific for the big and popular applications such as Salesforce, Dynamics, Workday, SuccessFactors, and those enhancements—again, they’re not about integration, it’s about, we already have a lot of data about what’s working in those systems and what is not, because we did hundreds of deployments over them. So, there is a lot of things you learn between customers.

    When we come to a new Workday implementation today, because we did a few hundred of them, we already know what the employees are gonna struggle with. We already know what the managers are gonna struggle with, and we built it into our system so it’s easier for our customers to create content, specific content. Make sense?

    Shimel: Mm-hmm. Got it—got it, got it, got it. Excellent. Rafael, I feel like you and I could talk about this stuff all day and lessons for people out there to learn to hopefully make their businesses better, but we’re out of time. I try to keep these, as I mentioned to you, to 15, 20 minutes. We might have gone over.

    Sweary: Sure.

    Shimel: But first of all—hey, congratulations on all of the success with WalkMe. And more than just the raising money—2,000 organizations using your product and really making a difference in trying to bridge the digital skills gap. It’s a great, great thing you’re doing, and keep it up. Secondly—love to have you back on again soon, maybe in the new year, and we’ll keep—

    Sweary: Sure. I would love to join and talk to you a little bit about what we do specifically for companies, how we help, and try to give some tips to some organizations that we see and what we learn from there, because we’re always humbled by what we learn from our customers. And it’s some of the best businesses in the world, and it’s really fascinating to see, when technology works, the competitive advantages that you’re able to gain.

    Shimel: Yeah. Agreed. You know what? You’ve got an open invitation, my friend, so when you’re ready, you’ll reach out to me and we’ll continue this.

    Sweary: Awesome. I hope Marissa hears that as well. So, Alan, thank you so much.

    Shimel: Thank you.

    Sweary: And I look forward to talking to you again.

    Shimel: Okay. Rafael Sweary, president and co-founder of WalkMe. This is Alan Shimel for devopsy.kinsta.cloud, and you’ve just listened to another DevOps Chat. Have a happy new year, a happy season’s greetings, merry Christmas, holidays, Hanukkah—whatever you’re celebrating, enjoy, folks.

    — Alan Shimel

  • DevOps Chat: Security Graph SDK Powers Better Policies, with vArmour

    DevOps Chat: Security Graph SDK Powers Better Policies, with vArmour

    The increasing breadth and complexity of the environments we manage are nothing short of breathtaking. Understanding the security risks and applying policies across the cloud, multi-cloud, private clouds and a plethora of software technologies is a challenge faced by every enterprise.

    vArmour recently announced version 5 of the vArmour Application Controller, which opens more significant access to vast amounts of information the Security Graph represents and organizes through its Security Graph and SDK. By streaming cloud, network and agent information into the Security Graph, enterprises now have a centralized understanding of application relationships across their multi-cloud infrastructure, enabling centralized risk and policy management that is simple, accurate and secure.

    In this DevOps Chat, Marc Woolward, vArmour CTO and CISO, and I discuss how establishing and assessing policies becomes more manageable through expanded access to information about software infrastructure, systems, environments across the organization.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.


    Transcript

    Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today, I’m joined by Marc Woolward, who is CTO and CSO at vArmour. We’re talking about policy and risk of cloud applications, how to manage it. Marc, welcome to DevOps Chat.

    Marc Woolward: Thanks, Mitch. It’s great to be here.

    Ashley: Well, thanks for flying over from across the pond, too. [Laughter] You’re in the U.S., with your U.K. accent, so thanks for being on.

    Woolward: There’s quite a few of us out here in Silicon Valley. To be honest, I think it’s half British.

    Ashley: There are, there are quite a few of you here. [Laughter] Would you do us a favor and just introduce yourself, tell us a little bit about you, what you do at vArmour, and just a brief on what vArmour does.

    Woolward: Yeah, of course. Alright, well, hi, everybody. As Mitch said, I’m Marc Woolward, I’m chief technology officer and also, in my spare time, the CSO at vArmour. My background is, I’ve been at vArmour for five years or so and I look at kind of the strategic direction, I speak to the senior technologists and our customers to understand their challenges with cloud security, and I lead some functions such as, you know, the data science team, and handle things like CCPA and GDPR in my role as CSO.

    So, my background is, before I joined vArmour, I was a technology fellow, and I was the CTO for telecommunications and networking at Goldman Sachs. I think I was there for 18 years. Did lots of things there. One of the interesting parts of the role of a tech fellow at Goldman Sachs is, you kinda get involved in all sorts of things. So, I was a member of the private cloud architecture team, you know, almost a decade ago, we built one of the first private cloud environments. And also, I had an interest in application resilience and application design patterns, despite my background keynote infrastructure. So, that kinda set me up really well for a job in a product company in Silicon Valley.

    Ashley: Yeah, very much so, especially the proliferation of cloud and all of that. So, let’s jump into it. I think we’re gonna talk about an announcement that was made back in October—what was it, version 5 of the application controller from vArmour?

    Woolward: Yes.

    Ashley: Can you tell us a little bit about what that announcement was, and then we can get into some of the effects that you’ve seen in the market from it?

    Woolward: Yeah, of course. Well, I mean, first of all, I think I probably should tell everyone a little bit about what vArmour does.

    Ashley: Oh, yes. Please.

    Woolward: So—yeah, no problem. I didn’t answer that in your last question. So, really, we focus on helping our customers to understand how their applications behave within cloud and multi-cloud environments. So, we have a risk and a policy engine that kind of builds a picture of your application behavior, its dependencies, and suggests the ways in which the application should be secured from a policy standpoint, and we help to make it—we make it simple for our customers to implement safe, secure, simple policies within cloud environments.

    So, what version five was all about, underpinning the application controller function itself is a security graph, and affecting that is a graph that describes all of the nodes, you know, the compute instances, the Kubernetes pods, the servers within a multi-cloud environment and the relationships between them. And what we’ve been doing is enriching that graph with all sorts of information about the application, compliance requirements, information about what the application does, is it there, what ___, what realm does it run in, what region does it run in?

    And really, what version 5, release five was about was opening that up and exposing more of the information within the graph so that customers can benefit from it more. So, what that means is the ability to plug more and more information into the graph to gain more value out of it. Things like information that reside in CMDBs, information from systems that handle vulnerability management, information from control planes, things like Kubernetes so that that information within the graph becomes richer and richer and more valuable. And it can be used for all sorts of use cases, and it also allows companies to build the policies based upon their view of risk and their view of their applications, whether that’s compliance driven or risk driven or business unit driven.

    Ashley: Now, can you kinda dig down and open up the hood a little bit more for us? Talk about how all this information gets into vArmour addition the application controller. Are you taking it from logs, configurations, active applications? What are some of the sources for this?

    Woolward: Yeah, and—great question. And what I do, I’ll start out by answering that by saying, in the cloud, we do not have a deficiency of data. So, if you’re running a reasonable sized cloud environment, you have tremendous volumes of logs—you know, cloud trail logs, flow logs, all sorts of event based information. And that’s—well, event based data. And that’s data.

    The problem with that is data—it’s unstructured, and it’s in huge volumes and it’s very difficult to kind of glean too much value from it. And what the application controller does is, it takes the data and it transforms it into knowledge. So, it provides you with insight into your application, which makes it a lot easier to make good security decisions.

    Ashley: Mm-hmm.

    Woolward: So, I’ll kinda come back to that in a minute. But the way that we do it is, we have a distributed telemetry pipeline where we use edge compute functions distributed out to the cloud environments that consume the logs and also consume events off of event buses and APIs. So, when a compute instance is deployed, we pull all of the metadata associated with it.

    The edge compute function we call message bus summarizes that data, enriches it, and then distributes it into the centralized graph where we effectively are transforming that data from being the individual flow log information into adding information into an edge within a graph, which represents the relationship. So, all the time, we’re enriching and summarizing, and that enrichment provides context, which makes it a lot easier to understand what your application is all about.

    Ashley: Well, and given the dynamic nature of our applications—cloud native, all of that, whether it’s Kubernetes clusters instances, microservices, just knowing what you have is half the battle, more or less what’s this information telling you? So, that’s got to be a value, too.

    Woolward: For sure. And, you know, Kubernetes is a fantastic example. Because if you’re capturing that information at the wrong level, if you’re looking at the IP address level or even the pod level, there’s a tremendous amount of noise in there and it’s very temporal and it doesn’t make any sense. If you can consume that at the abstraction, that makes sense to the application, so that would be at the deployment or at the service level and you ingest that into the graph you can build a picture of your application over time.

    And then when the, kind of the instantiation that’s just been deployed now varies from that model, you can see it very quickly. So, what we’re trying to do is summarize the information and represent it as—I say represent it as knowledge while continuing to process kind of the very dynamic and very high volume data and look for the anomalies and look for places where it varies.

    Ashley: That sounds, too, like a perfect application for a graphing technology to be able to visualize, explore, dig in deeper, back out, look at it at different levels.

    Woolward: Yeah. For sure, and you know, like I said, my background of working at Goldman and working in financial services, graph technology is fantastic for describing relationships, and that relates to risk. So, if you look at financial risk modeling, that has been almost the archetype application for graph technology for over a decade. I really would take that same approach and that same principle to cyber risk and cloud security risk.

    Ashley: Very good. Now, I know also that you’ve introduced an SDK with this. What are some of the applications where customers might want to use the SDK?

    Woolward: The whole point of the SDK—so, we integrate with all of the common cloud environments and we integrate with SDNs and the other product is designed to integrate into these different environments, just as part of the product out of the box.

    Ashley: Mm-hmm.

    Woolward: But what we’ve been finding with our customers is, they have different repositories for this context that would enrich an understanding of risk. So, if you have a vulnerability scanner, you have a tremendous wealth of information about the posture of the individual systems running in your environment. By plugging that into the graph, and by using the SDK, it’s very easy to do that for whatever system you’re using, you can begin to ask questions like, “Well, for my application, do I have any dependencies on systems with, let’s say, a CB severity of seven or above.” So, the SDK is a way of pulling in the vulnerability information, pulling the specific fields that are of value to you from your CMDB from, say, your service now, effectively plugging in that context that kind of illuminates your whole kind of understanding of your application and your risks.

    So, really, it’s about allowing our customers to integrate the data sources that matter to them into this graph that plugs into their cloud.

    Ashley: Mm-hmm. So, both in kind of an in and out application—bring in more data, also be able to pull it out for other uses reporting.

    Woolward: Yeah. That’s a superb point. I missed that. Another application that customers use the graph for is, embedded within the app controller, we have classification engines. So, what we do is, we will classify systems based upon our understanding of their behavior, based upon the relationships they have and the systems they have relationships with, we’ll classify them.

    So, one great use of the SDK is to compare the results of our systems classification with the information within a CMDB. So, you know, does your intent, which is in the CMDB match what’s actually happening in the cloud environment?

    Ashley: Mm-hmm.

    Woolward: And by doing that, what you can do is, you can validate that your inventory information is correct, is accurate, continues to be so—which, again, improves your security posture because, you know, one of the fundamental pillars of security is understanding what you have, and having an accurate understanding of the systems that you want to secure, so yes, for sure.

    Ashley: Absolutely.

    Woolward: But, you know, it’s a two-way thing, the SDK.

    Ashley: Well, and you’ve got a little bit of time under your belt with GDPR and we have the California Consumer Privacy Act coming up here. How does this kinda technology help satisfy, meet, report compliance—what are the various ways that you could leverage vArmour and the application controller to help you with those challenges to meeting regulatory requirements?

    Woolward: For sure. Yes, so, I mean, there were different regulatory frameworks that are more or less specific about the security controls required. So, if you take something like PCI, it’s very specific and it’s very directed in terms of the way in which you structure your controls. So, the app controller can allow you to scope your in scope infrastructure around your card holder data environment and build the layers of controls around the tiers of systems, according to the PCI DSS. So, it’s very good at doing that.

    If you look at something more abstract, which talks more about the outcome rather than the means of getting to the outcome—so, if you say it’s something like GDPR where you’re looking at protecting personal information relating to data subjects in Europe or CCPA where it’s more around protecting the personal information around California based consumers, that really comes down to managing risk and ensuring that the control—suitable sets of controls are placed around the systems that either store or process this information. And the first step to do that is to understand the scope of the problem and the scope of the relationships of the systems that are processing this personal information.

    So, the app controller, first and foremost, can allow you to understand, you know, the scope of these environments and begin to put controls and reporting around their behavior and access to those systems. So, it all starts from scoping and understanding your environments and ensuring that you can put the controls around the—you know, place the controls around the edge of them, and you continue to monitor that those controls are effective and that, effectively, in the cloud is what the app controller does.

    Ashley: Mm-hmm. Very good. Well, I certainly don’t profess to be an expert on either of those two regulatory frameworks. [Laughter] I’m really curious, we’re about a month, month and a half in since you’ve released version 5. As a former product creator myself, it’s always interesting to see what people do with new technology, new versions. But also, what you didn’t expect them to do with it. Have you seen kinda instances of both, you know—yep, we thought we’d help customers this way, it’s worked out that, and then we’ve also seen some interesting ways they’re using it they didn’t think they would.

    Woolward: Yeah. So, I’ll start with kind of what we expected and kind of the, if you like, the continued illustration of the value that we designed the product to offer. One of the things that we never—well, I never fail to be amazed by is, you deploy our system within a cloud environment and it begins consuming the telemetry. And within an hour or so, the graph is already beginning to hydrate and you’re finding hundreds or thousands of workloads.

    And all of a sudden, the extent of the interrelationship and often the external dependencies, the relationships outside of the VPC or outside of the VNET kind of are visualized. And almost every time, as we sit there for the first few hours with the customer, we’ll get into the same discussion about, “What’s that relationship? Oh, I have a dependency on something in GCP.”

    So, generally, that kind of uncovering the unknown and unsuspected dependencies opens up a whole kind of, you know, can of worms for understanding what’s going on here and ensuring that the correct policies are in place and ensuring that everybody  understands the extent of the attack surface and the blast radius. So, that, we always see, and that’s just accelerated with version 5.

    In terms of what was less expected, that’s the beauty of the SDK. So, the SDK allows the customer to plug in the information and the metadata that’s important and pertinent to them. An area where I didn’t really expect it was actually the application not just to your traditional cyber risk and your attack surface and your blast radius, but asking some interesting operational questions.

    So, this use case came up about two weeks ago where a customer was taking kind of an operational term for their applications and plugging it into the graph, and asking the question around, for my critical tier applications, my systemically critical applications, if one of their dependencies, one of the applications that they depend upon for, I don’t know, reference data or for other types of communication, like communications gateways failed, what would be my recovery time?

    And all of a sudden, what you can do is, you can draw within the graph the relationship between your application and the dependencies and plug in that recovery time objective, and all of a sudden, I understand that if application A, B or C failed, I might be waiting eight hours for recovery, whereas my system has a recovery time requirement of one hour.

    Ashley: Mm-hmm.

    Woolward: So, they’re beginning to ask these questions that you would not expect from, necessarily, a cyber tool around operational risk. So, I think that’s the beauty and the power of the graph—it allows the customer to think about, what does risk mean to them, and they can plug the information or plug the data in that they require and then they can ask these questions that lead them to knowledge.

    Ashley: Excellent, yeah. It sounds like both information, operational information, but also behaviors of what’s happening between applications and greater insight into it.

    Woolward: Absolutely, yeah. Yeah, for sure. And I think the other thing, I mean, the final thing I’d say is, if you look at most of these data breaches over the last year or two, it kinda comes down to, if you look at cloud and you understand the way that security works, it’s this shared responsibility model. So, the cloud service providers provide a tremendous array of tools—you know, micro-segmentation, identity access management policy enforcement. There’s a tremendous suite of security capability there.

    The problem is always that the lack of knowledge about the application means that the policies that are written and the permissions that are provided by the customer, because that’s their side of the, kinda the shared responsibility model have been deficient. And what the application controller allows you to do is to build the policy in a simple fashion that’s secure and meets your business requirements, and then that’s allowing our customers to meet their part of the shared responsibility model for cloud security.

    Ashley: Mm-hmm. Wow. Well, I feel like we’ve barely scratched the surface and we’re already out of time. Hopefully, we can get you back and we can explore this some more as we get closer to the CCPA coming into effect in January.

    Woolward: Yeah, and as the CSO of a California based company, I’m obviously spending a bit of time on that as well, so.

    Ashley: I’m sure you are.

    Woolward: That’d be fun.

    Ashley: Yeah, exactly. Well, Marc, thank you so much for being on DevOps Chat with us.

    Woolward: Yeah, it’s been a real pleasure, Mitch. Thank you—thank you for inviting me.

    Ashley: Absolutely. The pleasure has been ours. I’d like to thank my guest, Marc Woolward, CTO and CSO with vArmour and of course, thank you, our listeners, for taking your time to join us and listen to this podcast episode. This is Mitch Ashley with devopsy.kinsta.cloud. Have a great day, and be careful out there.

    — Mitchell Ashley

  • DevOps Chat: Is the Software We Create More Secure? Veracode’s 10th Report

    DevOps Chat: Is the Software We Create More Secure? Veracode’s 10th Report

    Application security is top of mind now more than ever. For more than a decade, Veracode examined increasing amounts of code as it passes through their source code vulnerability scanning service. During this period, automation is increasingly prevalent, making it easier to run scans more frequently and regularly. But has automation helped? Is the software we create more secure? We gain key insights about this in Veracode’s The State of Software Security Report X (10th edition).

    Chris Eng, chief research officer at Veracode, joined us on DevOps Chat. We talked about many insights uncovered in the latest report, such as 50% of applications are accruing security debt over time, the regularity of scanning correlates to vulnerability fix times and that scanning frequency directly impacts security debt.

    There is a wealth of information in the report, and you can get a jump on the key findings on this podcast episode with Chris.

    Transcript

    Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today, I’m joined by Chris Eng, who is chief research officer at Veracode. And we are talking about their latest report, the Veracode State of Software Security. This is volume 10. Chris, welcome to DevOps Chat.

    Chris Eng: Thanks, Mitch.

    Ashley: Great to have you on. Why don’t we start out by having you tell us a little bit about yourself, what you do at Veracode, and for those that don’t know Veracode, just a little bit about Veracode?

    Eng: So, as mentioned, I’m chief research officer here at Veracode. I’ve been here for 13 years now, and my teams are responsible, essentially, for building the knowledge that goes into our products.

    So, we have a number of different products that help our customers identify vulnerabilities in their software. And my team essentially identify—well, what are the patterns that we’re looking for? What are the common mistakes that developers are going to make? What are the important things that we should scan for in each language? And specify to our engineering teams how we do that.

    I also own product security for Veracode, so making sure that we deliver secure products to our customers. Our customers are basically anybody that builds, buys or deploys software. And so, we help them build programs around securing that software at scale, using a number of different technologies, including static analysis, dynamic analysis, software composition and so on.

    So, we work with anybody that deals with software, which is pretty much everybody.

    Ashley: [Laughter] Just about everybody these days with digital transformation, no doubt. Thank you very much for that background on yourself as well as Veracode.

    Well, let’s start out with, you’ve been doing this report for 10 years, and your longevity with the company, you’ve been able to oversee and be part of that over that decade process. What kind of things have you learned since you started out? You know, kinda the size of how many applications and things that you’re scanning to what’s happening today and any trends that you’ve noticed from that?

    Eng: Yeah, it’s been a great process to do this every year, primarily because we’re in this unique position where we actually can see what’s going out there in the industry and when we started doing this in volume one, we only had about 1,500 applications in the data set, and then fast forward to this year, we had 85,000 unique applications.

    Ashley: Wow.

    Eng: So, 50 times more than we had before. And I’m not aware of any other study that has quantitative information on software security that’s anywhere close to as big as this one.

    So, in addition to the size of the data set getting bigger, we looked at fix times, and we found that those have roughly stayed the same, which is sort of a little bit depressing—but that being said, there’s a lot more software now than there was before. So, there’s a lot more to scan, there’s a lot more to secure, and so, you only have so much bandwidth to do this stuff.

    So, we saw fix times stay roughly the same. We did see that, over that 10 year period, there are fewer apps that have no flaws. So, that’s more of a factor of our capabilities than anything else. We can detect more than we could before. But we did also see that there are fewer apps these days with no high severity flaws. So, customers are getting better at identifying and remediating high severity flaws.

    We looked at a number of different angles, including compliance trends and things like that, but for the most part, I think that the fix times is kinda the interesting part, and we dug a lot more into that and explored some of the factors that lead into fix times.

    Ashley: Well, if you just think about what’s changed over a decade, I mean, 10 years ago, we were thinking about SQL injection, cross site scripting—you know, those are the kind of app security flaws, at least a lot of what the focus was, things have changed drastically since then. [Laughter]

    Eng: Yeah, we are still seeing the vast majority of the same flaw categories that we saw 10 years ago. And I don’t think anybody who does app sec on a daily basis would be too surprised by that.

    Ashley: Mm-hmm.

    Eng: These things are really easy to fix technically, but when there’s so much of it and it spans across such a huge application inventory, it does take a lot of effort to close these. So, we still see SQL injection at roughly the same rate that we saw it 10 years ago. Other categories, like cross site scripting, have actually gotten more prevalent.

    And we’ve seen some go down. We’ve seen, like, buffer overflows and numeric errors reduce in prevalence. And part of that, I think, is due to the change in programming languages as well, right? There’s a lot of that that’s stuff that’s prevalent in native code like C++ and we’re seeing less, fewer applications being written in those languages today. So, I think that’s contributing more to the prevalence differences that we’re seeing.

    Ashley: I wonder, too, if service oriented architectures, microservices, things that promote reuse of code maybe help some of those things. Maybe it won’t—we’ll see. [Laughter]

    Eng: Yeah, it certainly code. I mean, any time there’s reuse, you know, you inherit the functionality, but you also inherit the risk, and that’s sort of the trends as we’ve seen people start using a lot more open source than before. Again, you get the functionality, but you also get the risk, and so you introduce new vulnerabilities to your applications that way.

    So, certainly, with code reuse, like you mentioned, you could get that effect as well.

    Ashley: Mm-hmm, potentially even infrastructure as code. There were some really great things that came out of the report. I’d love to have you talk about some of the applications incurring debt, how many folks are driving that down—you know, what kind of effects that had.

    Eng: Yeah, we took a look at security debt for the first time in this report. And, you know, most people have a concept of technical debt—just, you know, things that get old and crusty in your software over time, maybe architecturally, or things that you meant to go back and fix but you never did, and security flaws kinda have the same tendency to build up over time.

    If you think about it like financial debt, right, if you charge something on your credit card and then you only pay the minimum amount every month, you’re gonna be paying a lot, for a long time. And when you add that up, it’s gonna be a lot more than it would’ve just been to pay off your balance right when you accrued it, right?

    And so, security debt, when we look at security debt from that angle, we look at, are applications kind of accumulating new security debt over time, or are they driving it down? And when we look at the number of flaws that are kinda left unfixed in an application, we kinda view that as the security debt, right?

    And, you know, there’s only a certain amount of capacity, right? It’s very difficult for an application that, let’s say, has been accumulating security issues over many years to just say, like, “hey, we’re gonna pay that all down today,” right? You’d have to dedicated a lot of engineers to doing that, you’d have to really focus all your efforts to do that. And so, it’s not typically very practical to do that all at once, but it is important to kinda be measuring whether or not you’re fixing more than you find or finding more than you fix, because that’s kinda directional, right?

    And so, we found that nearly half of the applications are finding more than they fix. And so, essentially, they’re accruing that debt, so they’re never gonna climb out from underneath that unless they start to reverse course there, right? Put more effort into fixing issues so that they, over time, reduce that debt and get to a point where—okay, now there’s nothing kinda outstanding, and you can then make a rule that says, you know, “we’re not gonna allow this application to be promoted into production or whatever if we accrue new findings.”

    Ashley: Mm-hmm.

    Eng: But that’s rare, right? So, barely anybody is there yet. And so, we looked at how different DevOps tendencies or methodologies, practices affect security debt. So, I think those are some of the findings that I’d like to talk about a little bit.

    Ashley: Sure, let’s go to that, because I’m very curious about, as we shift left, right, we’re trying to get security in earlier—

    Eng: Exactly.

    Ashley: [Cross talk] earlier, all of those things, hopefully you fix it before it ever goes into a QA function or an automated testing.

    Eng: Right, there’s all those studies to kind of show that it costs a lot less, right, when you fix it earlier. And that makes sense—if you’re fixing a flaw as a developer is writing the code, if you can sit there and tell them what they did wrong before they even check that code into the repository, it’s gonna be a lot cheaper than if you find it in a penetration test, let’s say, after the code has been deployed and then you’ve gotta go figure out the root cause and then you’ve gotta find a developer that understands that code and get it in their backlog. So, that’s kind of an accepted understanding that it costs more, the later you do it.

    Ashley: That’s after you get done pointing fingers on whose code is it in or is it in the network or the servers or the—you know?

    Eng: Exactly, exactly—then it’s kinda tracing it down. And so, we wanted to really say, like, does DevOps make a difference in how quickly we can fix things and how much security debt we accrue?

    And so, sitting from where we are, remember customers submit their applications to us and DevOps is not just automation, it’s also culture and process and a lot of things that kind of feed into whether you’re doing DevOps or not. But from our vantage point, we can’t see culture, we can’t see process, we can’t see automation. We can’t see to what extent automation has been incorporated into an applications testing cycle, for example.

    We use scan frequency as a proxy for whether an organization is using DevOps. So, by that, I mean, how many scans do they run per year on an application? And so, you have a lot of applications that are scanning literally once per year, right—36% once a year.

    Ashley: Wow.

    Eng: And if you think about how quickly software is changing and how many features are getting added every week or every month, once a year is—that’s not very good. On the other end of the spectrum, you have about 0.3% of apps that are scanning basically every day or more. So, you know that that’s probably being done by automation. You don’t have a person sitting there submitting the application, hopefully.

    And so, if you look at—there’s some charts in the report that kind of show what that distribution is, but essentially, if you scan once per month compared to if you scan every day, your median time to remediation is significantly faster. So, 19 days median time to remediation for a flaw if you’re scanning daily, versus 68 days if you’re scanning monthly or less.

    And so, we then kind of broke it down into different, even more buckets, right? So, we did kind of what does one to three scans per year look like, four to six, seven to 12 and so on. And it’s kinda hard to describe this just via audio, [Laughter] but if you pull out the report, you’ll see all these kind of like iceberg charts. And so, you’ll see, like, pink and blue, and there’s—the pink represents the debt. It’s kind of below the line and that kind of shows the number of findings per app that are outstanding.

    And what you’ll see is that, for the apps that are scanned more frequently, less debt accumulates. Essentially, the teams were able to get after that debt faster, and while it doesn’t go away completely—again, because this is aggregating all of the apps we have—it accumulates less. And so, we see a direct correlation between that scan frequency and both the fixed time and the security debt accumulated.

    Ashley: It certainly makes sense automating that. That’s gonna show up in your numbers around the 19 days to fix, scanning once a day. Is there a way to tell if it’s happening? Do you see scanning happening more frequently than on a daily basis? Is that helpful, or are there other things that help—again, help this shift left?

    Eng: You know, I think once you get to a daily basis, you’re probably—I think there’s gonna be some diminishing returns after that. I mean, imagine you did a full scan every time somebody checked something in. You’d just be getting a lot of information—you wouldn’t be able to act on that quicker than, you know, I think the span of a day or a few hours. So, scanning every few minutes really wouldn’t buy you anything.

    Ashley: Mm-hmm.

    Eng: So, another thing that we looked at that we thought was interesting would be scan cadence. So, not so much how frequently were you scanning, but how regularly are you scanning? And so, you can imagine—and again, there’s this great diagram, it’s my favorite diagram in the report, actually, and it’s just a bunch of dots on a chart. But what it does is, it maps out that every dot represents a scan.

    And so, if an application is scanning on a very steady basis, you would see evenly spaced dots across the course of the year, whereas if you were scanning in kind of a bursty fashion—so, basically, a lot of activity followed by no activity, then you would see a clustering of dots followed by just a bunch of white space.

    And so, we actually calculated that cadence for every single app in the data set, and then we grouped them into buckets, again, so of the ones that were either scanning on a steady basis or a bursty basis or something in the middle which we called irregular—and irregular is basically just a bunch of mini-bursts, right?

    Ashley: Mm-hmm.

    Eng: So, what we wanted to answer was, how does that scan cadence affect security debt? Are you less likely to accumulate debt if you’re scanning steadily or in a bursty fashion, alright? Which one’s gonna be more effective at wiping that out?

    Because you could see it going either way, right? You could set it, you could just—people could get used to seeing results and then they get to the point where they ignore them, maybe. Whereas bursty, you’re like, “okay, well, I’m paying a lot of attention to this. We’re gonna focus all our efforts on this” and then, you know, drive it down.

    So, there was actually a question there. We didn’t know what we were going to see. But when we did break it out, we found out that when you do the bursty scanning, you have all that white space and you just put a flurry of activity over time, like, you’re just—that security debt that accumulates is just massive. You see this huge increase in the amount of pink, which represents the findings that are not addressed, whereas with the steady and even to some extent the irregular scanning, you see the security debt grow a little bit, but then start to decrease. And so, the curve is actually going in the right direction towards the end of the time frame that we’re able to chart out. And so, that was another good finding for us.

    So, what we can do is, we can actually take those conclusions and advise our customers and, really, anybody that’s building software that, if you want to reduce security debt, the data suggests that you should scan frequently and that you should also scan in a steady basis. You shouldn’t ever let up, right? It has to become—and that makes sense, right? Anything that we make a habit of tends to just become part of the way that we do things. And so, it was nice to have some data that actually reflects what we think would be true actually does turn out to be true.

    Ashley: It’s kinda like, brush your teeth daily, right, not the day before you go to the dentist. [Laughter]

    Eng: Yeah, exactly—that’s not gonna do you much good.

    Ashley: [Cross talk] [Laughter]. You know what also occurs to me is, this bursty style is, there’s really no predictability around how long a security flaw is gonna exist in your code, because you may find it and fix it now, but it may be—if it’s another three months or whatever before you scan again, who knows when it gets fixed? It could live out there for a long time.

    Eng: Right. We actually did find—it’s funny that you mention that—that the highest probability for a flaw to get fixed was in the first month or so. And so, the longer you wait, the less likely a given flaw is to get fixed.

    And so, essentially, developers are prioritizing kind of, like, last in, first out. They’re more likely to prioritize something to get fixed if it’s kind of fresh. And that’s not what we wanna see, right? We wanna see developers fix things that are more important. We wanna see them fix more severe items or items that are in applications that are more critical to the business or items that are more exploitable than others.

    But when we measured all of those and we looked for patterns that would suggest that they are prioritizing in that way that’s sensible to a security practitioner, we found that that recency, like, how recently was it found was really the highest correlation in terms of whether something was going to get fixed.

    Ashley: Mm-hmm.

    Eng: And so, we have to do a better job of prioritization, not just understanding what a security person would do, but getting a developer to adopt those same priorities.

    Ashley: You’re obviously not measuring the human psyche element of this, but I have to believe you’ve built up such a mountain of that debt, at some point, it gets too hard to grok, understand, fathom, and there’s probably even an abandonment rate that’s just like, “too big—let’s just deal with what’s on our plate right now, and here’s the scan results” and jump on it.

    Eng: It does, and that’s not—that’s not uncommon to see customers adopt that type of strategy. That, the idea that—alright, well, I’m starting this program now, on day zero. I’m responsible for appsec now. And, you know, anything that happened before I got here? Well, that’s not my problem. Let’s just focus on not introducing new flaws—which is fine. Like, not introducing new flaws is great, but that doesn’t help you with anything in the past.

    Ashley: Right.

    Eng: And ultimately, it’s all risk to the application, right? You can get attacked any number of these ways. But we have seen—we’ve seen big customers try to take that approach and, you know, it’s not advisable. You have to chip away at that debt. Even though you can’t do it all at once, maybe you work in security sprints into your life cycle, you find a way to pay it down over time, and eventually, you get to where you wanna be. But you cannot just ignore it and say, “No new flaws going forward.” It’s not gonna get you where you want as far as the debt.

    And it costs more to fix stuff later, too, right? The longer we wait—you know, imagine you’ve got a library to patch and that library is, you know, one year out of date or five years out of date. It’s gonna cost you a lot more in terms of effort to fix the one that’s five years out of date. Things will inevitably break, so the cost of fixing also increases over time.

    Ashley: There are hundreds of findings, if not more, [Laughter] in this report. Not to pick out just one, but it stood out to me that there was one stat in there of C++ carries three to five times more unresolved flaws than .NET over the same period of time. So, this is to the point of, languages make a big difference, and why is that so?

    Eng: Yeah, you know, we did break it down by language and just kinda took a look at how much security debt accrued into each one. And it’s not really to say that—okay, well, if you’re using C++, you should rewrite that in .NET or some other language. That’s not really practical for most people.

    Ashley: Sure, yeah.

    Eng: But what it does show is that certain languages are more likely to accumulate debt over time. And it’s important not to so much look at the raw number of flaws, because some languages just are inherently more secure against certain classes of flaws, right? It’s a lot harder to shoot yourself in the foot in certain languages than others.

    Ashley: Mm-hmm.

    Eng: But in looking at the shape of the curve, like, does security debt tend to increase for certain languages? That’s kind of—that’s something to look at and at least be aware of, if you have certain parts of your application inventory written in, you know, PHP, for example, you should be aware that those apps are probably gonna be more likely to accumulate debt than, say, the .NET ones.

    Ashley: Mm-hmm.

    Eng: And so, it’s something to take into account as you’re doing your planning.

    Ashley: Okay. Very good. Well, we could talk about this for maybe days, [Laughter] this is so much information, here.

    Eng: [Laughter]

    Ashley: So, we don’t leave everyone with a mountain of, “oh, gosh, there’s so much in here to learn and understand,” are there two or three takeaways if you’re a developer, lead developer, development manager, architect sitting out there listening to this, going “okay, so what do I need to know?” What are the couple of takeaways you would suggest?

    Eng: Yeah. You know, I’ll kinda reiterate some of the things that we talked about, but essentially, security automation, especially if we look at scan frequency, it is definitely lagging the adoption of DevOps in general, right? DevOps has kinda just taken off like a rocket ship and security automation, like I mentioned, only 5% of the apps were being scanned weekly or better. And so, there’s some catching up to do there.

    The good news is that, when you actually do that, and when you actually do that frequent and steady testing, you will probably get to a point where you can start chipping away at security debt and eventually, over time, drive that down so that you can take a strategy of, you know, no new flaws and keep yourself at a clean pace.

    And the last one, I think there’s conversation that needs to happen between security teams and developers in terms of prioritization. You know, I talked about how developers are not prioritizing in, really, a security appropriate manner, though recency is appearing to kind of outweigh every other factor. And so, there’s, I think, some improvement that most teams could make there where, even with the same amount of bandwidth to fix flaws, they could spend their time fixing things that are more important to be fixed as opposed to the ones that are just appearing most recently.

    So, I think those are some of the takeaways, and like you said, there’s a lot in the report. It’s a pretty interesting read, and I would encourage people to go grab a copy and read through it.

    Ashley: And very well done, if I do say so myself—very well put together, there, Chris. Where can folks get the report?

    Eng: It’s on our website, so Veracode.com, and it should be on the front page there, and—yep, there’s about a 50 page PDF behind it.

    Ashley: Okay, perfect. We’ll include a link in the description for this episode, too, so.

    Eng: Excellent.

    Ashley: Well, thank you so much. I appreciate you being on, Chris.

    Eng: Yeah, my pleasure.

    Ashley: It’s been great to have you. Again, thanking my guest today, Chris Eng, who is chief research officer at Veracode, and of course, thanking you, our listeners, for joining us today. We know your time is valuable and it’s a great topic—security is important and having this information, I think, is very valuable to all of us.

    This is Mitch Ashley, with devopsy.kinsta.cloud, and you’ve listened to another DevOps Chat podcast. Be careful out there.

    — Mitchell Ashley

  • DevOps Chats: 1-Click Vulnerability Scanning on GCP, With Qualys

    DevOps Chats: 1-Click Vulnerability Scanning on GCP, With Qualys

    Sometimes the best way to accomplish something is to choose a path requiring the least friction, or amount of change.

    Qualys customers now have that path available to them in bring vulnerability scanning into Google Cloud Platform. Qualys’ recent announcement means a one-click configuration change enables vulnerability scans in GCP with the results appearing in both Qualys Cloud Center and GCP Security Command Center.

    Join me on this DevOps Chats to explore this announcement with Sumedh Thakar, Qualys President and Chief Product Officer. We discuss Qualys’ and Google’s collaboration and how this benefits DevOps teams.

    Transcript

    Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud and you’re listening to another DevOps Chat podcast. Today I’m joined by Sumedh Thakar who’s President and Chief Product Officer with Qualys. We’re talking about an important announcement that just happened with Qualys and with Google about security natively embedded within Google. Welcome to the podcast, Sumedh.

    Sumedh Thakar: Hey, Mitch. Thank you for having me.

    Ashley: Great to have you. Well, let’s start by just having you introduce yourself and, of course, as President and Chief Product Officer we can imagine what you might do at Qualys, but go ahead and tell us a little bit about what you do.

    Thakar: Yes, I’m not sure you can imagine because I’ve been here for 16 years, so I’ve done a lot.

    Ashley: Okay. [Laughs]

    Thakar: Today, as President and Chief Product Officer, I’m responsible for product strategy and implementation of Qualys product and where we have come from where we were many years, mainly focused on just vulnerability assessment. So today I have the engineering product management support operations team as well as the sales team and that kind of gives a pretty unique perspective because of the engineering and the DevOps teams within Qualys. Also, part of the organization for me helps me to really get a very good understanding of today what are the needs for DevOps and what evolution we’re going through with the DevOps team internally within Qualys, and being a massive SaaS platform ourselves I think that that’s something that I’d really like to focus on when we develop security solutions for DevOps teams.

    Ashley: Well, it does give you a very hands-on perspective, both _____ as well as living with DevOps in your own products internally. It’s great.

    Thakar: Exactly.

    Ashley: It’s good for you. Great role to have. Well, let’s dive into this announcement. So, security that’s natively embedded within Google – Google Cloud I think we’re talking about.

    Thakar: Right.

    Ashley: Tell us a little bit about what it is.

    Thakar: Yeah, we’ve had a fantastic partnership with the Google team and, you know, I think as digital transformation, moving things into cloud container and DevOps have been really getting traction for us to _____ differently than what we have done in the past, which has always been after the fact, adding security solutions. So, today with DevOps and DevSecOps the opportunity to embed security upfront into the infrastructure in an automated way so that you can ensure that you’re not missing – you know, that’s always the biggest issue with security is, oh yeah, I did all of that, but then I missed that one thing because I didn’t know that existed.

    So today, Google Cloud has a fantastic solution that really allows DevOps team, as they’re pushing code continuously into the cloud, to have the ability with this new Qualys integration to automatically have that capability of assessment of the security of that virtual machine embedded directly and done transparently so that the end user does not really have to go about installing agents and consoles and all of that. It’s done; completely embedded in the platform.

    Ashley: Excellent. So interesting, and we’d love to dive into this some more, because you’re talking about not bolting things on after the fact. You know, this is the whole shift left idea within DevOps.

    Thakar: Yes.

    Ashley: So, I imagine with the Qualys cloud agent for Google that’s something that you’re embedding within the software platforms that developers test production environments you’re using.

    Thakar: Yes.

    Ashley: Is it that kind of a lifecycle?

    Thakar: Yes, exactly. So, developers today use that solution to first of all ensure that in their – whether it’s Jenkins or whatever it is that they don’t even push solutions out to production that already have vulnerabilities, right? The unfortunate side effect of doing this bolting on after the fact is that the moment you spin up a new machine it already has so many vulnerabilities in the last two years, you spend all your time trying to patch them. So, this DevOps pipeline does give us the opportunity to ensure that first of all in your build process you already eliminate and patch your images so that you don’t go out with images that are already vulnerable, and _____ customers absolutely use the Qualys agent and scanning to really achieve that.

    But then once it gets pushed out to product you also want to have a monitoring to ensure that somebody doesn’t go to an S3 bucket or to a Google storage unit to pull down a vulnerable version of some sort of a software in their virtual image and create new vulnerabilities, so you also need to have that monitoring. And so that native embedding ensures that once you have done your DevOps process and cleaned the image, as the image is being pushed into the production environment in the cloud, in Google cloud, it is going to have that monitoring capability already embedded transparently without having to do additional effort.

    Ashley: Great. I imagine it’s not a heavy lift for developers to install this in the development test environment, right?

    Thakar: Yeah, and that’s the beauty of it is everything is automated, right? So, once you write the scripting it’s not a process that requires you to run commands every time and do all of that. Once you push that into the CD process then it’s really transparent, because in this case the developers don’t even need to run any command to get the agent onboarded. It’s an option that they check in the Google security center and then every single image that spins up will have that sort of one-click integration and automatically embedded in there.

    Ashley: Great. I imagine then you’re also tying back into integrating into the security consoles both within the Qualys environment and also within Google?

    Thakar: Yes, absolutely. That’s a unique ability here, because you have multiple business units for organizations. Different teams have different accounts in Google and they are, of course, only looking at their own application or their own account. And so the second part of this announcement really is also the fact that those vulnerabilities and those issues that are detected are then pushed back into the security center from Google so that the individual owners of the accounts can see all of the vulnerabilities on all of the issues that they need to fix right there within their console and they can create playbooks and use the automation that is available to quickly fix those issues so that they don’t have to know go to another tool and go to another console and try to do all of that.

    For the IT folks and the DevOps folks who are pushing these applications into the cloud, they just see their own individual view directly in Google, but then of course this information also is available in the Qualys console, which is a wider visibility into the overall risk posture because it will also include the security and vulnerability findings from other types of infrastructure that the customer also has, things like laptops and handheld devices and containers and other things as well. And so the security team will leverage the Qualys platform because then they get a holistic view of the entire outside vector. The individual team in Google will get their view within the Google console so that they can really focus on fixing what they need to fix.

    Ashley: Really kind of empowers that idea of SecOps, right?

    Thakar: Right.

    Ashley: Better right upfront.

    Thakar: Exactly.

    Ashley: Now I imagine that the security engineers – you know, usually there’s some kind of a corporate security team maybe involved in operations, maybe not, but also would be super pleased about having this implemented earlier into the dev cycle and also that it’s pre-integrated already into the Qualys platform, Qualys monitoring as well as Google’s cloud monitoring.

    Thakar: Right. And that’s really a very good point because that’s a big struggle for security teams, right, is that just getting – ensuring that the entire infrastructure is covered by the security solution is a challenge when you do it after the fact, because they may not even know that there’s a new instance that was pushed by the dev team overnight into the Google account, the security may not know. So, with this integration of course it ensures that no matter what gets pushed Qualys is always monitoring every single image that goes out. Now the security team is good for them because they don’t have to spend a lot of time as they have been doing so far trying to get the security solution implemented.

    So instead of focusing on the findings of the security solution a lot of security teams just spend time trying to get that actually installed and making sure it’s up and running and it’s connecting and all of that. So, having the DevOps team actually do all of that work is fantastic because now the security team can only focus on what is important, which is your policy. So, they can be the one that says, “Well, this is a failure if you have a _____,” and they don’t have to worry about how the thing is installed or whatever it is. Qualys is then the third party that ensures that the policy that the security team has defined is actually tested and a pass or a fail is given to both sides of the house and then they each can go ahead and do what they need to do in terms of fixing and reporting those issues.

    Ashley: Now, you described in the announcement sort of one-click integration. What does one-click mean? Can you visualize that for us, how that works?

    Thakar: Yeah. So we say one-click because if you have ever deployed security solutions they are absolutely painful, you know, trying to get the multiple commands –

    Ashley: Or like one month or one quarter. [Laughs]

    Thakar: Yeah, it’s a thousand clicks and multiple consoles and whatever it is. So the one-click is really because when you go and look at that integration, once you kind of go in Google security center and define that Qualys account that you connect to, then after that it’s really just checking one box that says that include the Qualys agent in every single image and then you don’t have to do anything after that, like you don’t even have to run a separate command every time you initiate a new image because that is already taken care of in a very easy manner with the automation that is provided by Google.

    Ashley: Wow. So it truly is a checkbox kind of process.

    Thakar: Right.

    Ashley: Just included.

    Thakar: Exactly.

    Ashley: Are there any other things that maybe you’re already doing or that are part of this announcement in terms of what you’re doing to monitor scan and report vulnerabilities in the Google cloud environment?

    Thakar: Yeah. I mean we have a very strong partnership with them. We’re listed, of course, in the marketplace as well, and with this sort of integration it really provides sort of a security reference architecture where, you know, Google can now provide along with Qualys customers the ability to make it significantly easy by defining the architecture of how you want to secure your workload that you’re putting in Google and then have – not only define the right solutions that should be used but then also work with vendors like Qualys to get those directly implemented with a single click so you also remove the resistance at integration and really simply that overall process. And so that will enable existing Qualys customers as well as new customers who already are going through the digital transformation, they may have Qualys already on their on-prem infrastructure; now it makes it – it removes another hurdle for them to really move into Google cloud because they can get the same kind of reports and the same kind of analysis as they have been used to because their auditors are very comfortable and used to having the Qualys reports so now they can get the same reports and don’t have to go through an entire process to recertify another solution, so there’s a lot of these kind of benefits that go across the board, and just having that embedded I think really helps.

    Ashley: Yeah. It certainly lowers the bar to entry –

    Thakar: Right, right.

    Ashley: – into the Google cloud if it’s already integrated into the Qualys and Google platforms. Let’s circle back to talking about what are some of the benefits to the DevOps teams getting the Qualys agent into their environment earlier? What kind of things do they learn in the dev cycle and the test cycle group normally they would’ve skipped and not found out till way later when they’re ready to go to production or in production?

    Thakar: Yeah. So because the way the platform is structured with APIs, the biggest advantage, there is automation of – and that’s what DevOps is all about, right? How can I script? You know, you talk about infrastructure as a code. I talk about security as a code, right? Can you just code in to say – you know, like you do your infrastructure, can you say if vulnerability is greater than this than failed _____ or, you know, can you write that as a thing?

    And with the way the Qualys platform is and being a cloud-based platform ourselves and with AP as available, that makes it extremely easy for the DevOps folks to really create directly these scripts that they can basically say, “In your pipeline I got a new image, bring that image up, Qualys agent gets already embedded, it does its analysis and the findings show back up in the console and now you can really code that thing with a human, never having to go in and verify and look at the scans or whatever it is, right?” The way the architecture of the Qualys platform and the agent is, is that agent is an extremely lightweight agent that is not focused on doing just vulnerability assessment.

    It does inventory assessment. It does vulnerability, also configuration assessment. And so what happens is that single agent can be used to look at a larger picture of the security of the device and not just if the software is not updated, right? You can do everything from is unauthorized software being put on this build, is end-of-life software being put on this build, is there configuration that is incorrect even though you may have patch or all your vulnerabilities, and so on and so forth. With that sort of an integration, what happens is that once you have done that one task of getting that agent on the virtual machine you can now suddenly with just APIs do a lot more to get a broader view and make sure that that overall workload is heavily locked down, because now you really are ensuring that all the configurations are as per the golden images and all of the CIS benchmarks are followed and all the software is updated and unauthorized software is already eliminated from that whole thing. DevOps is all about how do I eliminate the manual work and how do I use automation and reduce the amount of work that I have to do, and that’s exactly what that architecture enables us to do.

    Ashley: I think it’s an extremely good point; with all the automation, the increase in speed, the decrease in manual intervention or tasks that are being performed. It’s easy for something to slip in.

    Thakar: Yeah.

    Ashley: New opensource code we haven’t used before or something now has got a vulnerability that didn’t two hours ago and you’ll find out right away.

    Thakar: And, you know, one of the things that we have seen is the DevOps push to go quickly into production and sort of was that initial thing of, “Oh, I want to bypass my operations team, I want to bypass my security team,” and the flipside of that is like now your code that goes directly to production and has an issue, that’s on you as a DevOps – like you cannot blame the security person because, you know, they are not really involved anymore. And so that also incentivizes the developers up front to ensure that they’re doing all the right things from a security perspective also so it’s not their microservice that goes out that has their name on it that gets compromised, right? Because then you can basically say, “I’ve coded everything, done everything on my side up front so that what I am pushing out to production using automation is not something that is already vulnerable.”

    Ashley: Excellent. So, let’s talk about how do you get access to this with the release of this capability? Is it already native within Google cloud? You just connect to it and say, “Here – you know, I’m already a Qualys customer,” or do you have to go to Qualys to sign up for something or how do you get access to this?

    Thakar: Yeah. So, we – and as Qualys we provide directly – if you go to Qualys.com there’s – you can sign up for a free trial if you’re not a Qualys customer and, you know, we’re a SaaS service so you immediately get credentials that you need to access the platform, but then it’s completely embedded in the Google cloud console, so when you got there and there’s the configuration settings you will be able to now pick Qualys as a vulnerability management capability that you would like to add and, you know, there’s a one-time configuration where you provide your Qualys agent information and from that point on, once you have that done, every single – you know, do that one-click checkbox, after that that embedding will happen automatically and Google cloud will be the one doing that for you. So within a few minutes you can be up and running with this integration and starting to see the results in a couple hours.

    Ashley: Excellent. Is there any incremental cost for this or is it already included, just an added capability in the Qualys product?

    Thakar: Yeah. So, it’s a cost that the license of vulnerability management that customers already have. They can basically just take that license or use that or they can come and purchase additional licenses to run additional infrastructure in Qualys, but it’s the same license that they would otherwise use that they can use – that they get from Qualys and they can use in Google Cloud as well.

    Ashley: Yeah. Just the licenses they would need to manage a Google could environment anyway.

    Thakar: Exactly.

    Ashley: Okay. Excellent. Well, great. Fantastic news. Anything else we should know before we wrap up?

    Thakar: No, I’m really – you know, as heading the product, I’m always curious to get feedback, and so I encourage as many of your listeners to go and sign up and, you know, send us feedback on how they feel it helps simplify, and if there’s opportunities for us to simplify the DevOps process even more with better integration we’ll be happy to hear that.

    Ashley: Great. Well, let us know how it goes and we encourage our listeners, go check out Qualys. Qualys.com. And especially if you’re an existing customer, it sounds super easy to start to use this, and I know – I’ve been a Qualys customer and also worked with Qualys before as a partner and super easy to get set up and going, so I encourage folks to check that out. Well, thanks for joining us, Sumedh. Congratulations on the announcement.

    Thakar: Thank you very much. Thanks, Mitch. Good talking to you.

    Ashley: Nice to talk with you. I’d like to thank my guest, Sumedh Thakar, President and Chief Product Office of Qualys, and of course, thank you our listeners for joining us today. This is Mitch Ashley with devopsy.kinsta.cloud. Have a great day and be careful out there.

    — Mitchell Ashley

  • DevOps Chats: Software Architecture for Cloud Native, .NET Core and Open Source

    DevOps Chats: Software Architecture for Cloud Native, .NET Core and Open Source

    In this episode of DevOps Chats we talk with Donald Lutz, principal software architect specializing in systems integration and creating large, scalable cloud applications. Occasionally DevOps Chats is fortunate to spotlight DevOps and cloud native developers doing trailblazing work in contemporary software architectures. Donald fits that bill to a T as an entrepreneur and employee at startups such as Faction, BoldTech Systems and his own company Technetronic Solutions, and established companies including Via West.

    Our discussion focuses on creating cloud native applications in startups and large enterprise IT. Donald’s currently working with one of the world’s largest financial institutions to move from legacy applications directly to cloud native apps, bypassing any interim lift-n-shift moves. His work spans many Microsoft technologies, including .NET Core, runtime framework Dapper for RDBMS mapping, Service Fabric and Azure, opensource Kubernetes, Terraform and Puppet (and Enterprise).

    During our discussion, we cover the challenges of architecting and scaling very large cloud native applications, implementing DevOps in less mature software organizations, how established software patterns benefit DevOps and cloud app developers, and the importance of giving back by hosting meetups to share knowledge and mentor others. Donald also gives back as an active leader and mentor of the FIRST Robotics Team 1410 since 2005.

    Join in on our conversation as we explore the complex and sophisticated inter-workings of a cloud native software architect.

    Transcript

    Mitch Ashley: Hi, everyone, this is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today, I’m joined by a very special person, a good friend of mine, Donald Lutz. He’s a principal software architect at many companies. We’ll kinda get into that, but he’s focused on microservices cloud, he also is very involved with Microsoft in his background.

    He’s currently involved in several things, but he’s with his own company, Technetronic Systems, and he’s worked with Faction, ViaWest, he actually was one of the original co-founders of BoldTech Systems with me back a while ago.

    So, our topic today is microservices going cloud native. Donald, welcome to DevOps Chat.

    Donald Lutz: Thank you, Mitch. Thanks for having me on. I really appreciate this.

    Ashley: Oh, it’s a great honor of mine, you’re a good friend. And to that point, why don’t you tell us a little bit about yourself, introduce yourself to our audience, tell us about what you do.

    Lutz: Yeah, I’ve been sort of a principal software architect and a director of software engineering for different companies like Faction, my own company TSI and I’ve really in the past five or six years focused on cloud native and taking—you know, getting microservices and applying that to building cloud native applications. That’s really kind of what my focus has been. And there is a lot of DevOps that goes in there. There’s also a lot of architecture and design, so it’s kind of a combination of all of the above.

    Ashley: Mm-hmm, excellent. Well, just full disclosure, I mentioned before, you and I have known each other a long time. We started working together back in the telecommunications industry and I did a startup called BoldTech Systems, which you were a part of founding and getting going and, you know, you’ve hired me, I’ve hired you. [Laughter] We kinda worked with each other—which, by the way, for folks earlier in their career, that’s a great thing to do. Keep your friends close, because you’ll get to work with them again.

    But you’ve had this consistent pattern in your career where I know you were very Microsoft and .NET focused for a while and then kinda moving into cloud and broader technologies. I mean, you’ve worked on Java and other things, too, it’s not like you just did Microsoft, but tell us a little bit about your philosophy on technology. Are you a fanboy Microsoft or how do you kind of view that with everything else that’s going on in the technology world?

    Lutz: I mean, I could be viewed as a fanboy of Microsoft. I mean, I helped—I’m running now .NET Core, an open source Meetup for Microsoft. I really like where they’ve taken .NET Core. I mean, I would claim, six or seven years ago, Microsoft realized that they needed to stop being so proprietary and get more into open source.

    Ashley: Mm-hmm.

    Lutz: So, they really kind of redid all of .NET, and .NET Core is fundamentally, it allows you to build microservices, traditional apps, too. But, you know, they moved to the open source arena because they knew their future was Azure, it wasn’t really Windows in the traditional sense. You know, 75 plus percent of all their things that were under Azure are Linux that run with another product called Service Fabric and, you know, that to me, is a great technology. I have done Java, I’ve done Go, I’ve done a lot of different things, you know? I have a large Microsoft contingent, but they’re all part of the same thing where everybody’s trying to get to cloud native—what does cloud native look like? Whether you’re Microsoft, Pivotal, VMware—everybody’s trying to figure out how to you get there, you know?

    And a lot of people have taken steps in between, like, some people are trying to run VMware on the cloud, but I’m seeing a big push where I’ve been talking to a lot of people, I’ve been working with a really large financial institution right now, and they’re actually punting on the in between and they’re going completely cloud native using both multi-cloud and in their definition, is Azure an AWS?

    Ashley: Interesting. And I’m aware of who you’re working for. I know we’re not talking about the company name specifically, but give us an idea of the scope and magnitude. Because we’re not talking like, you know, a credit union or a large regional bank.

    Lutz: No, they’re one of the largest financial institutions in the world. So, they are looking—they’re moving everything at the top level of their management, you know, CTO level, they decided that they wanted to do microservices, domain driven design. They wanna go all cloud native. It’s either gonna be running in AWS and Azure. They don’t wanna own any hardware any more, they don’t wanna have any of it. For them, it’s an amazing kinda revolution, because they’ve never gone there, and they’re kinda viewing that they wanna run their company in a technological era kind of like a startup, so they’ve really changed their whole view.

    So, I was kinda surprised that they were that interested in that and the fact that they looked at the cloud and they said, “We need to go cloud native, because we don’t really need in between steps because then again, we have another shift and lift then in five years,” and we only wanna have one shift and lift, so I was really surprised they even thought of that.

    Ashley: Well, it is, and I’m very surprised by that, too. It’s certainly—you know, I don’t talk to every big financial institution in the world, but to make that level of commitment of, we are gonna go cloud native, not like we’re gonna lift and shift what we have and just kinda live with legacy, if you will, and build new things in cloud native, they’re actually taking the major part of their software assets, the systems rely on day to day and they’re shifting that just going to cloud native in one big step—well, several one big steps, apparently. [Laughter]

    Lutz: It’s a lot of big steps, but I mean, it was kinda surprising, because you know, they brought in executives who had been more out in Silicon Valley and not traditional financial people. So, they’ve made a different, and they went and found a bunch of us who have been kind of in the cloud native microservices space and brought us on. They kind of viewed it as, they’re trying to build, like a, sort of the Super Bowl winning football team, that if you get the top players, then you can get this to happen.

    Ashley: Mm-hmm.

    Lutz: That doesn’t mean there still aren’t interesting political battles, since this is a company that’s historically, you know, they still have DB2 and green screens, so, you know—

    Ashley: Sure.

    Lutz:—it’s not like it’s completely, “Oh, yeah, this is awesome!”

    Ashley: Right. [Laughter] It’s not like they started in the Linux area, though—era.

    Lutz: Yeah.

    Ashley: So, say a little bit about—so, you have a unique perspective of, you’re not the kind of entrepreneur that’s worked in a from scratch startup in the garage every time, repeat, repeat. You’ve done that, but you’ve also worked with a lot of existing organizations. I know ViaWest had been acquired, I think, by Shaw Communications before you joined and they were going through a transition.

    Lutz: Yeah. I went there because they were—I was looking for a place that wanted to move to microservices and they had been acquired by Shaw. I knew their CTO, we chatted, so they hired me as a Director of Software Development, and the goal—they were all .NET, so I moved into .NET Core and we started building microservices, which is the whole idea of, there’s a lot of stuff that comes out of major design and bounded context.

    So, instead of having, you know, your monolithic application that has customers and data center stuff, we started building services like, “Here’s your customer service.” And it all deals with that. We also brought in some other, DevOps people know about more CQRS and event sourcing. So, the whole idea, each service has its own database that it writes its data to. It then issues commands—so, say I created a customer, I say “create customer.” That causes an event called Customer Created that gets stored in an event store, which almost looks like an audit log. You never delete from it, you only add to it.

    So, if you ever have any problems with all these distributed services failing, you can rerun this audit log and get back to where you were in the beginning.

    Ashley: Mm-hmm.

    Lutz: So, it’s kind of a powerful concept. These concepts have been around for a while, but you’re starting to see them being employed at a larger level. Because, you know, the idea of synchronous communication doesn’t work well in cloud native. In fact, you really just want to expose public APIs, and behind the scenes, you want data streaming, you want eventing. You don’t want any asynchronous communication, because it’s not the right approach. You can’t scale, and you’re gonna be dealing with eventual consistency.

    So, you know, once you start dealing with that, once it’s not in one box, you know, eventual consistency is it.

    Ashley: Mm-hmm. Where you’re going to kinda stateless applications, right? Microservices—it’s interesting that the trend is, obviously, is build much smaller things, reuse them, create services out of them. But there’s gonna be many, many more mass quantities of them. It’s not like you’re going from—you’re going from one monolith with a lot of functions and APIs to really small pieces of software that are containerized microservices.

    But what comes with that is now, all of a sudden, you have the problem of managing that, communicating across and synchronizing when it is asynchronous how you communicate, handling troubleshooting, knowing what all those services are. It sounds like what you’ve done is, you’ve reached back into some maybe techniques and technologies or knowledge of past things and bringing that into this microservices Fabric world. Is that true?

    Lutz: It’s true. I mean, I get a lot of Kubernetes and all that, and you know, even though you start doing Kubernetes and a lot of DevOps things, it’s sort of the thing that I find interesting. It’s kinda like, sorta what you on the Phoenix Project, you know, you need to be able to manage all those things and be able to deliver all the time and monitor it and log it. The problem is that that’s not just answered by creating a bunch of scripts, you know what I mean? Like, even though I’ve used Terraform, Puppet, I think there’s a larger scale of solution.

    Microsoft just released a new framework called Dapper, and Dapper is basically dealing with the microservices run time. So, it’s a run time that can sit on top of whether the microservices can go talk to Kubernetes, because you need to be able to handle that run time stuff as well as the development issues that go on. Those things are not two separate things, and I still think that’s the—one of the large DevOps challenges is how do you get everybody on board, you know?

    And at ViaWest, it was pretty sophisticated, because all our DevOps people were software engineers. So, we just wrote software for most of our problems. We either built it ourselves or used various tools. But then I went to a company called Faction where we had a less mature DevOps practice, and we were building multi-cloud storage, so we had a lot of microservices that allowed us to do Isilon, Dell storage, you know? So, we built a fairly sophisticated microservices platform. We didn’t have the DevOps practice, which resulted in some interesting issues, because the organization was not mature. Everything was manual. So, it’s really hard to roll those out, because you roll them out and then you can’t roll them out as well as you’d like to, because nobody’s catching up.

    Ashley: Mm-hmm.

    Lutz: They’re like, “Oh, well, how do we do that?” I’m like, “Eh, yeah, that’s an issue.” There’s no tests going on, you couldn’t do a Chaos Monkey. So, that was—in that case, in that case, it was pretty challenging, you know? And also the whole—I’m thinking a lot lately about security and the fact that you have all these cloud native accounts, but you have, you know, you have these very, you have credentials that give you access to RDS and Amazon, and then those aren’t really secured real well, and there’s Vault and there’s multiple ways to do it. And the whole security thing is, I don’t think, well cooked in a lot of ways. I would claim it’s not. [Laughter]

    Ashley: Well, application security is now become—becoming, maybe both—you know, one of the top topics. It seems like we’re just sort of, we do things differently, and then we have to go back into and go back and address, like, “Okay, well, how are we doing this now?” Because, you know, it’s not about protecting servers and network devices. We’re exposing so much of the tax service as an application through all these microservices, APIs, et cetera. You gotta kinda rethink it.

    And then you’re automating the development environment to do 15, 20, 100 code deploys a day, potentially. Well, that takes automation—that means all those credentials, security keys, certificates, whatever are built into a process that’s all that secure. I think that’s what you’re talking about is, you have to do it, but you have to also go back and re-look at it to say, “Are we doing this securely?”

    Lutz: Right. You have to look at things, “Do I have—is data at rest encrypted? Do I check any kinda hash on it? Is the data in flight encrypted? What are the levels of security? Do I let my JWT token float to one service, to another service, because that could be an attack, too.” Because I went to service one and now I go to service two and I’m using the same JWT token, but maybe the claims on that second service, it shouldn’t be allowed to do those things, because the claims that it used from the first service lock it down better, but in service number two, it allows you suddenly, “Hey, I could go look at different accounts that Mitch has, and well, that’s interesting.”

    So, then you start thinking about, you know, how—there’s a couple good security books about that whole idea of, you know, JWTs are interesting, but the problem is that if they float, if we haven’t figured out the right way to share those, that’s another thing that causes lots of problems.

    Ashley: Mm-hmm. And for me and for our listeners, can you define what a JWT token is?

    Lutz: Yeah, it’s a JSON web token. It’s just—

    Ashley: JSON, right?

    Lutz: Yeah, yeah. It’s just a token that basically you can use with an API. You know, it’s a little—it’s really for APIs. OAuth, you can do both. OAuth is more effective with users, but those are, you’re seeing the industry starting to adopt JWTs between services. So, now people are saying—but the problem is, the concept behind it is a claim. I wanna make a claim that your name is Mitch and you live at this address and you have this phone number. And in this case—oh, by the way, you can look at your Social Security number, but in this case over here, you shouldn’t be able to look at this other person’s Social Security number.

    So, you start making all these claims, and then you have to start getting really smart about how do we define all these claims for these very distributed systems.

    Ashley: Mm-hmm.

    Lutz: Which, historically, distributed systems were everybody—oh, if you had the root password, you’re golden. We’ll just use that and everything will connect to it.

    Ashley: Yeah, interesting. It’s really interesting, not being that close to it but following what’s happening with it. I think it’s based on, like, RFC 7519, so it’s based on a spec. It’s about how claims essentially get bedded into what we traditionally think of as a data interface specification between services and applications. Anyway, we’re not a JSON JWT podcast, but interesting sideline.

    I wanna get back to talking about microservices in cloud native. So, as you’ve worked with existing applications, and even at Faction with something that maybe didn’t start out in a DevOps world, what do you see as some of the kinda approaches that make sense and maybe things that you or others have tried that turned out not to be the best path or dead ends that you’ve corrected from and now here’s how you do things?

    Lutz: The first problem I see is, you make nanoservices. So, instead of—you know, you make a microservice that tracks time, and it’s so small that it’s not very effective. There’s a software architecture discipline called the main driven design, which is about when you build an application and a service, there’s a domain that it covers, whether it’s a customer or a product, and it needs to be the right size. And if it’s the right size, it’s what’s called bounded context. Because the whole idea is, you want that service just to answer that bounded context, which is based off the idea of a single responsibility principle. If it does more than one thing, you have a problem.

    So, kinda getting people to understand that is complicated at a business level, because you know, business people historically want to throw everything in whatever they’re building. Like, “Oh, can’t we just, why do we need to have such a small bounded context? Can’t we, like, throw all the addresses and products in here?” I’m like—no, because we need to keep those separated, because we’re trying to, each microservice has to have a very distinct separation of concerns, because if you don’t have that separation of concerns, you end up with a monolith, which is what we’re trying to avoid.

    Ashley: Yes, end up back into the spill over again of domains into, this microservice has too many things in it and now one thing’s affecting another, versus let it be a single purpose in our service.

    Lutz: You know, and that’s why we actually saw that this large, large financial institution has adopted domain driven design and they’re actually building domain models for all their applications and everything. So, there will be demand models behind them.

    So, it’s just not the microservice, they actually can say, “Oh, by the way, this is what this financial instrument looks like, and it actually maps to our true domain model and then will map to a true microservice.”

    Ashley: Mm-hmm, okay, so domain driven design, nanoservices, or kind of focused functional microservices that don’t kinda spill over.

    Lutz: There’s another thing, too, you need is what historically—Chris Richardson has a book called Microservice Patterns, and it’s the idea of what’s called a microservice chassis. So, once you start building microservices, you either need to make a framework or use a pre-existing framework, whether it’s Spring Boot or some stuff in .NET Core so that when you write the microservices, all the logging is done the same, all the separation and concerns are handled so you deal with all those non-functional requirements so you don’t spend any time developing it.

    Ashley: Mm-hmm.

    Lutz: But then once you do that, then you have to start saying, “Well, how does this play in the run time world of Kubernetes?” Which gets into something you said earlier—we need to sort of connect the framework with the run time stuff, so that’s kind of a problem you see in Pivotal, Microsoft. They’re all looking at how do we get these really interesting software engineering frameworks that then wrap into run time things that are equivalent, so we end up with, you know, “Oh, they’re the same thing, they’re isomorphic.” They’re not different, they just look different because we view them as different domains.

    Ashley: Mm-hmm. Interesting. Well, we have a couple minutes left. I wanted to just take a little bit of a left turn—not quite a squirrel moment, but a little bit of a left turn. I’ve been doing some podcasts here recently with folks, usually developers, DevOps engineers, Ops folks who are talking at the Spinnaker Summit Conference, speaking on Kubernetes and Spinnaker Open Source.

    So, we’ve had a great opportunity to have other developers, real hands on technical folks on the podcast. And one of the things that I always encourage folks is, even in the open source community, you contribute not just by writing code, it’s by sharing your knowledge, applying these tools and software and sharing the knowledge and networking and building community with others. That’s vital to having, you know, vibrant software is having a vibrant community about it. And I know you’re very active in your own developer circles and communities in creating those. Tell a little bit about what you’re involved in now and why you do it.

    Lutz: I like doing it to share. I mean, I would say about four years ago, there was a gentleman running a group, there’s a group worldwide called ISO, the International Association of Software Architects. And he was trying to get the group growing locally in the Rocky Mountain region—you know, Denver, Wyoming, that kinda area. He hadn’t gotten very far, so I got kind of involved, and we’ve actually grown it over the 4 years that we have 2,000 members and we get about 60 to 100 members coming every meeting, and it’s truly talking about architecture. It could be the Kubernetes architecture, it could be domain driven design. Last month, I talked about how to build microservices that are stateful. Surprisingly, there’s a company called Cloud State. Microsoft and ACCA, they figured out how to build stateful microservices. We won’t get into that.

    Ashley: They didn’t bring pitchforks and, you know—

    Lutz: They didn’t bring pitchforks. It works pretty well.

    Ashley:—torches? [Laughter]

    Lutz: But we get different people. We got the guy who’s the original guy at FedEx who created all their stuff that looked like Kubernetes in the ‘60s come talk. We’ve had people from Slack. We get a lot of people to come talk. I’ve talked at the Carnegie Mellon architecture conference, I can’t remember what it’s called. I talk at a lot of Microsoft groups. I run a Meetup for Microsoft. They have a few .NET groups that are dying, so I pitched to them—why don’t we turn that into a .NET Core and open source group?

    Ashley: Mm-hmm.

    Lutz: And so, I’ve been running that. I’ve grown it—that’s almost to 1,000 members now. It was actually only 200. So, in the three or four months, I’ve actually grown it. I like creating communities of technical people, because when you do that, you actually can solve interesting problems. You also get to hear other people’s experiences.

    I can give you an example—one of the guys who get involved with ISO, I was talking in Colorado Springs at an event, I can’t remember where. And I was outside the event and this guy, Tyler, came up to me and started talking to me and I discovered he was really technical. And he said, “What are you doing?” I said, “Oh, I’m talking about microservices and that.” And he says, “Why don’t I come in?”

    And we actually—he became the person who asked me the most questions during the event and then he became one of the four members to help us run the Denver Software Architects Group.

    Ashley: Mm-hmm. Interesting.

    Lutz: And we became really good friends and it was like, you know, just talking to somebody outside an event I was gonna speak at.

    Ashley: Very interesting. Well, you know, I think it’s also, all of the things you said, I totally agree with. It’s also a way for someone who wants to know about a topic, maybe, and I like how you’re kinda keeping the groups fresh on what’s happening and new in Microsoft and other worlds of software. It’s a place for folks to go and learn about it and hear what other people—even if you’re very engaged in a new technology already, it’s, what are other people doing and what have they learned from it? What can you share, what can they share with you, and vice versa. So, it’s a healthy thing to do.

    Lutz: Yeah, I really enjoy it. I find it more enjoyable—it’s up there with this robotics stuff FIRST I do in terms of raw enjoyment, because it’s so, you’re plugged into so many people where, when you know, when I’m doing consulting or helping build a startup or all of the above, it’s enjoyable. But it has—this has more of a kind of like, not selfless, but it has more of a giving back vibe that I like a lot.

    Ashley: Mm-hmm. Very good. Well, hey, we’ve run out of time. You and I could talk for 12 hours and, you know, up ‘til 3 a.m. in the morning. There’s so much fun things that you’ve done. I really appreciate you being on the podcast.

    Lutz: Well, thank you for having me on. I really appreciate it. I really enjoyed it.

    Ashley: Well, good. Me, too. You’ve listened to another DevOps Chat. I’m very pleased to have my guest, Donald Lutz, on with us today. Donald is a principal software architect, he focuses on microservices, cloud, cloud native, generally Microsoft technologies, but a lot of other things and he’s with Technetronic Systems. He’s also been with Faction, ViaWest, BoldTech Systems, a number of different companies, so he has a lot of software background. And again, our topic today was microservices in cloud native.

    Thank you, also, to your, our listeners, for joining us. This is Mitch Ashley and you’ve listened to another DevOps Chat. Thanks, and be careful out there.

    — Mitchell Ashley

  • DevOps Chats: App Attention Index and The Era of Digital Reflex, with AppDynamics

    DevOps Chats: App Attention Index and The Era of Digital Reflex, with AppDynamics

    Application performance company AppDynamics (acquired by Cisco in 2017) recently released its 2019 edition of the App Attention Index Report. The 2019 report is subtitled The Era of Digital Reflex. Joining me on this episode of DevOps Chats is Steve Long, AppDynamics regional CTO and technology strategy.

    Digital connections are a near-constant. Interacting with our world through digital apps and transactions are omnipresent: making a restaurant reservation using OpenTable, making payments using an app or our smartphone digital wallet and checking our smartphones to plan and move through our day. Thus the name of the report, The Era of Digital Reflex.

    In addition to insights from the report, Steve and I talk about measuring the customer’s experience, tying performance back to the desired business outcoming, strategies for finding problems and reducing the time it takes to fix problems. Join us on this compelling discussion about application performance, customer experience and business outcomes.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Mitch Ashley: Hi everyone, this is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today I’m joined by Steve Long. He’s regional CTO and technology strategy with AppDynamics. Today we’re going to be talking about the 2019 edition of the App Attention Index Report findings, The Era of Digital Reflex. Whoa, that sounds heavy. Steve, welcome to DevOps Chat.

     Steve Long: Thanks for having me today, Mitch. It’s a pleasure to be here today.

    Ashley: Sorry, I kind of put on my ‘60s accent. “Whoa, man, heavy.” Would you start out be introducing yourself? Tell us a little bit about what you do and of course, in case somebody doesn’t know what AppDynamics does and what AppDynamics is about.

    Long: Yeah, sure, absolutely. So I’ll go back a little bit in time. I was a vice-president of technology operations for a media company and then my last job before joining AppDynamics, I was a chief information officer for a financial tech company, a large banking services company. And I joined AppDynamics because–for two reasons. One was finding problems in the sea of–no longer a needle in the haystack–it’s a needle in a needle stack, at this point. With all of the complexities happening, it changed my career. And when Cisco and AppD joined and that merger took place, I couldn’t think of, you know, two companies that I would love to be part of, that when they joined–you know, after I left being a chief information officer to come and help drive technology strategy for AppDynamics.

    So AppDynamics is a application performance management company that back in the dawn of the day I became one of their first 30 customers, about nine years ago, and really attaching to the run time and basically looking and analyzing your application performance. If the application is the north star of your business and typically how businesses are driving revenue and transactions, we’re at the forefront and the heart of that, to be able to tap into that and give an MRI to what’s happening in your cloud, your data centers and what’s happening with the complex technology running in the business.

    So that’s a little bit about AppDynamics. We’re an application performance company and part of Cisco, and driving businesses to really see into what’s happening in their technology ecosystems.

    Ashley: Excellent. Well you share something with me, and that is both the kind of end consumer of technology like this is a CIO but also a tech person in the product, the tech vendor. So that’s fantastic. I enjoy sharing that with you. Now tell us a little bit about the App Attention Index Report and what that’s about–what it measures, what kind of information that it discloses for us.

    Long: Yeah, so “The Era of Digital Reflex” is the title, but at the core we’ve really entered into really consumers saying they use digital services almost like you would a human, like an extension of your body, like a reflex. And today we’re doing that almost automatically or unconsciously. And that shift in consumers–like if you look past five years ago, we had to proactively do that. Now, with something in the palm of your hand, you don’t realize it, but now you’re relying on digital services like every day and it’s really helping you get your jobs done and your life duties, and you’re taking decisive actions by it being a reflex.

    And it’s kind of a wake-up call for the industry and businesses, because at this point we’re looking at the performance of these applications being application loyalty and it’s taken over to be the new brand loyalty. So if businesses really want to stay competitive, they really have to have a really flawless experience for their users. Otherwise it impacts their brand loyalty and their application loyalty and other things. So that’s the heart of the report at a high level.

    Ashley: Makes total sense. I mean, you’re talking about our digital world and our connection to it. It’s not only true today, but it’s even more so–every day there’s a new function we do on our phones, on our computers, on our TVs, on whatever device is connected to some cloud somewhere. It’s almost all a digital transaction.

    Long: Yeah, absolutely.

    Ashley: So tell us a little bit about this year’s report–what some of the findings were. What would be some of the most interesting things that came out of it?

    Long: I’d like to actually use an example that I have personally, and it’s based on the report. So I actually had a lunch with colleagues, a friend and a customer last week, and it was about six of us. And I said, “Hey, just out of curiosity, how many applications have you guys used today?” And to a one, they looked at their phone and all kind of raised their heads back up and one rattled 13, one 17, one 12, one 19. I was like, “How did you do that?” And they were like, “Well we tend to close out our applications at the end of the day and it’s 2 p.m. and I counted up from the last application used to now.” And I was blown away. And so I did the same thing and went back to the last one that I hadn’t closed and counted and I was at all 11 at that point in time, at 2 p.m. And I thought to myself, “Wow, okay.”

    And then I asked–the secondary question was, “Well, did any of those give you any problems? And I’m relating back to the app index, because if it had a problem, what did you do?” and the first example that came back was, “I had a reservation at a restaurant last night and I had to go close one and go to the other because it didn’t have my reservation, so I ended up having–for every application that I have, I have a competitor.” And if it’s really sticky, maybe like banking or something else, maybe they don’t, but they’ve got a few ride shares, a few travel apps, a few restaurant apps. And it’s really that wake-up call, because the metric that comes out of the report is about 55% of the people can’t go without their mobile device for up to four hours.

    I did that today before this, just to see if I could do it. And right before I got on here I had to give up, because I needed to see if I had a notification or a text or something that I had missed. And so personally it is that reflex, that extension. It’s what we do now unconsciously.

    Ashley: I think I’d rather hold my breath than try to live without my digital–

    Long: No doubt. No doubt.

    Ashley: So when you count the number of applications that people are using, do you count like, for example, on your phone you can look to see, how much battery time did you consume with different applications is another way to measure effectively kind of usage. How do you look at it?

    Long: Well I mean, it’s habitual. So we just looked at ones that were open. But I mean, if I go back and look at my day, my alarm goes up and I pick up my phone. And the first thing I did there was, “Now that I have my phone in my hand, I’m going to look at an application.” My schedule, maybe my e-mail, maybe a text. And we’re basically saying now that the first thing we do is not talk to humans; it’s to look at our apps. And nowadays–you know, if I go back five or six years, my house is now fully connected. So I’m looking at my security camera and any events, I’m looking at my thermostat. Did I leave my air conditioning or heat on? Is my car charged? You know, and do I need to set the thermostat in there? Is it 100 degrees out and do I need to put the AC on before I get in my car? And all of a sudden I’ve got energy usage tools.

    So these applications can be anything from a browser-based service, but they’re there to help you with your life. And then as you get to work it becomes even more–more of a need, with reminders and you know, calendaring and everything going on in your work world and sharing documents and collaboration. And that’s why at 2 o’clock you can hit somebody who’s had 20 and in a day they can be hitting 30 to 50. So it’s pretty amazing what these digital services are doing and how intrinsic they are to our daily lives.

    Ashley: Well it’s amazing, because one of the things–I’m guessing I’m probably not part of your report–but occasionally I’ll look to see how many connected devices or IP addresses have been allocated in my home. And it has gone from a dozen to two dozen to over 70 in a year and a half.

    Long: Wow.

    Ashley: And that’s just as I kind of went to a connected home. And every one of those is not just talking to stuff in the home. It’s talking to the cloud, you know, for the temperature management, for the door locks, for the security system, whatever–it’s all talking to something, if not me or one of my family members through an app on the phone or browser.

    Long: And think about when they don’t work and they can’t talk to that cloud or digital service. And since it’s an extension of your human interactions, now you can have expectations that are super high that you just had a disruption, and it can impact your mood. It can cause frustration. It can do things that you don’t even know are happening. And that’s why we call it–you know, it’s a reflex now. It’s intrinsic, it’s unconscious and it can cause strife in our lives when it doesn’t work.

    Ashley: That’s a really good point, because we always think about networks as end consumers as speed, but the reliability has also gone up significantly, and it’s going to have to increase in reliability moving forward, because that’s–it’s a reflex, it’s a dependency, it’s an expectation, it’s a customer satisfaction measure, right? Of what that experience is like.

    Long: Yeah, and I think the expectation just a few years ago was much lower, and every year that goes by–I think three quarters of the people we queried–that their expectations had gone up. And it’s growing and growing, and the tolerance levels now have also become more important, and these services are just more important because they’re prompting a higher demand and better experiences, so that businesses can help you with your daily lives. And so the  net of it all is we’re hungry for very positive digital experiences, and we’re willing to pay for that and have them be more personalized and know who you are and kind of know what you want, and that’s a huge differentiator between businesses competing with each other.

    Ashley: Interesting. As you looked at this data, were there different ways that you slice and dice, segmented–you know, home versus business or mobile versus browser or any of those kind of ways of looking at the information, to gather some interesting facts?

    Long: Yeah, we did. We have a crosscut of that. Mainly though looking at the consumer base side and how it interacts with the businesses and how the expectations of the consumer are there and need to be met and exceeded. The experiences need to grow. I know in our daily interactions from AppD, we look at e-commerce companies and banking and a number of different online services, and it’s pretty clear that the performance, the loyalties are pretty clean cut when you have a performing app versus a non-performing app. And the impact it has on the consumer. And so those can transcend almost every industry. It’s almost industry-less, where the application performance wins.

    You know, every hundred milliseconds of latency–you know, that causes some level of distress, and when you start to get into situations where something might take 10 seconds, in industry A that might be acceptable because it’s a longer-term type of transaction. But in industry B, which may be more of a real-time field, like I need a ride or I need a stock trade or I need something like that, 10 seconds is unacceptable. And if you’re not able to measure that and have application performance, you can’t dissect what your problem is because ultimately it’s your business transaction which leads to your experience–a journey that you’re going through, your business journey–and at the end of that rainbow is the actual business outcome that you’re looking for. And whether that’s a sale or moving money or buying a ticket or a new shirt, that’s the key, is having that business outcome.

    And the more we tie application performance to that business outcome, we’re better off as a business. At AppDynamics we look at all of the way to the edge, to the browser, to the device, to the data centers, to the cloud, but we also tack into that the ability to look at money or what the business KPI that’s being impacted. Because if you have a seasonal event like an inventory level dropping or a loss of money or you’re making more money because of a promotion, wouldn’t you want to know that your performance or your lack of performance–what the impact is to businesses?

    Ashley: It’s interesting too, because it wasn’t very long ago–I mean, maybe just a few years ago–where we’re almost solely reliant on things like an NPS measurement or call times–things that we do in the call center. And outside of that, you know, one person tells five people when they have a bad experience and you don’t really know those kinds of things. But now, in this age of data for everything, you know, the last time somebody used my Lyft app, “Oh, it was poor performance. We haven’t seen them in three weeks.” Maybe they went to Uber. I may not know that, but I know what their last–there’s data there to mine and to go after, where some of the telemetry information about performance could indicate some symptom that might’ve led to a consumer change in behavior.

    Long: Yeah, and it’s no longer–we talk about expectations and not meeting them, but the downside is that is your reputation risk. I mean, if you get an issue reported to you from internal or you know, somebody inside, that’s one thing, versus it being reported to your call center–that’s another thing–versus somebody switching to a competitor and posting it on a social media site. Now you’re not only worried about stock price and revenue risk; you’re talking about reputation risk. And that reputation risk really does tie back into those expectations not being met, at whatever level it is. So reputation and how, you know, media out there is picking up on that and promoting that when it’s negative–for a reason, because it’s a shared experience.

    Ashley: This may be a bit of an impossible question, so I recognize that up front. Is it possible to measure what customers’ expectations are? Is it too big of a moving target? Is it too many variables? Or is it something we can get a handle on? How do you address that?

    Long: Well, I firmly believe we can. And say that coming from the customer side of AppDynamics, but also–you know, businesses know their business, their KPIs, what outcomes they’re looking for. And if you don’t measure it, you can’t really make change in a positive way. So we see digital transformation–Agile–you know, we’re seeing BizDevOps–those are all things to really move businesses to become more efficient, drive the right decisions around the right business outcomes they’re looking for. And you know, I always say you do have to focus on the application performance. Being a past CIO, I tried to put myself in the customer’s shoes. I mean, if you don’t have a robust way to look at application performance then you don’t have that safeguard or those guardrails that you can put up to make sure that, you know, number one, your mission-critical applications are working, but number two, it’s the user experience. And being able to measure that and seeing that all the way from the edge to your cloud services.

    And the dependencies. I mean, think about it, the number of SaaS providers, cloud providers, third-party data that you’re touching just to execute a login on any application site, it could span hundreds of transactions. And if any one of those little things go wrong, you’re not focusing on application performance or you can’t measure it, and it could have a negative user experience.

    Ashley: Very good point. Because there are very few, maybe if any, closed ecosystems, to throw the “ecosystem” word in there. You really are reliant on a lot of other providers to come together to deliver that overall experience, that service, that functionality.

    Long: Yeah, and that’s how you get your business outcomes. So I do think that it’s achievable and you know, that performance and correlating it to the business performance and the business outcome–that’s really the objective companies are setting, and why they’re moving to APM and companies that are expanding out to give them business metrics on top of APM and consumer metrics on top of APM. Because it’s that trifecta. And if you think about it, there’s lots of ways you can look at logs or network flow data, but at the application, since it’s the central north star of the business, it’s the most granular and contextual data that you can get to see whether or not your business is performing. So that when you have that granular and contextual data down to the line of code maybe not performing, you’re able to look at the two most important metrics to any company, which is finding and fixing problems and reducing the time it takes to get to solutions on either side of that. So the faster you find them and fix them–whether it’s a ten-second issue that you can automate out of your system, or if it turns out to be a ten-hour outage, those are radical, you know, separations that we see in companies by just implementing the right tool set, the right performance measurements and really driving their businesses to reduce those risks in any company.

    Ashley: Okay. Maybe one last thing to kind of touch on–were there any other interesting findings, notable things? Maybe kind of a small little quirk thing that was “Wow, we didn’t expect to see that,” any other things to glean from the report?

    Long: No, I think we’ve covered it. At a high level I always suggest, put yourself in the consumer’s shoes. Really have a robust way to look at performance, and tie that back to the business outcomes. Because ultimately the business teams have a really hard time understanding IT and engineering, but the IT teams actually can make that leap. They can tend to look at the business challenges and situations, take that complex technology of all the ecosystems and translate that into showing the business what’s actually happening under the covers in a meaningful way. And that’s the basis of the report, is that the expectations are rising, the issues are out there. Finding them and fixing them, and really, put yourself in the consumer’s shoes, because it’s only going to grow, the dependency. And the flawless application wins.

    Ashley: Great. How do folks get a hold of this? How do they get to The Era of Digital Reflex report?

    Long: It’s posted on our AppDynamics website. It’s AppDynamics.com, and you can go ahead and get it there, and we’d be happy to share that.

    Ashley: Excellent. For you to download, I assume?

    Long: Yep, sure is.

    Ashley: Nice. Okay, well excellent. Thank you so much, Steve. Appreciate you being on DevOps Chat with us.

    Long: Thanks, Mitch. It was a pleasure, and I appreciate the invite.

    Ashley: Absolutely. I want to thank my guest today, Steve Long, regional CTO with AppDynamics. And of course we want to thank you, our listeners. We appreciate the time that you spend listening to us and ____ some feedback that you give. This is Mitch Ashley with devopsy.kinsta.cloud. Have a great day, and be careful out there.

    — Mitchell Ashley

  • DevOps Chats: Automic + CA + Broadcom, a Release Automation Journey

    DevOps Chats: Automic + CA + Broadcom, a Release Automation Journey

    The life of an acquired company can be an exciting one full of unexpected twists and turns, and that’s been true of Automic software. Scott Willson, a repeat podcast guest and longstanding member of the DevOps community, joins us again on DevOps Chats. He describes his acquisition journey as being akin to a “barracuda, eaten by a great white shark, eaten by a whale.” Scott is product marketing director, Release Automation at CA Technologies, a Broadcom company (the new proper name for Automic + CA + Broadcom).

    Scott shares what it’s like being a software DevOps company inside Broadcom, whose strong roots come from the chip manufacturing business. While software has its many differences from chips, of course, Scott shares how many of the DevOps principles and heritage from lean manufacturing have made the transition easier. There are valuable lessons here for all of us.

    Amid all the change, Automic shifted its products to the freemium model. Automic Continuous Delivery Director is now free to use for up to 10 active releases. We talk about the upcoming releases, which includes machine learning. Join us as we bob and weave through the acquisition story into the world of continuous automated delivery.

    Transcript

    Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud and you’re listening to another DevOps Chats podcast. Today, I’m joined by Scott Willson, he’s product marketing manager, Release Automation at CA Technologies at Broadcom—that’s a mouthful. I think we’re gonna explore a little bit of that.

    So, our topic today is really catching up with what’s been happening with the Automic product and CA’s, this part of the acquisition back into Broadcom and just kinda catch up with Scott. So, he’s been on the podcast before. Scott, welcome to DevOps Chats.

    Scott Willson: Thank you, Mitch. Glad to be here.

    Ashley: Appreciate you being here. Would you start just by introducing yourself for any of our audience that doesn’t know you, a little bit about what you do, and tell us a little bit about CA Technologies at Broadcom.

    Willson: Yes, yes, so my name is Scott Willson—two Ls in Willson. I think many listeners may know me. I have been, in the past, an author with some of Gene Kim’s DevOps Forum papers, and I’ve been at the DevOps Enterprise Summit several times. I’ve been fairly active on blogs and all that stuff.

    And where my kinda claim to fame was, I came from a company called Automic. That’s spelled Auto-mic for the phonetic spelling of it. My background is, just to throw this in here, too, my journey, as I always tell people, is I started from a development standpoint writing the code, migrated my career kinda doing the DevOps thing before anyone called it DevOps, like many others.

    Ashley: Mm-hmm.

    Willson: And yadda yadda yadda, here I am now with a software company speaking all things DevOps and trying to help people get better at releasing their software.

    Ashley: You’ve certainly been a part of the community for some time now, so we appreciate all your contributions. Well, you’ve been through quite a journey. I know being acquired by CA and then CA being acquired by Broadcom—why don’t you kinda catch us up? Maybe tell us a little bit about that journey. I think you described it as an 18 month history going through that.

    Willson: [Laughter]

    Ashley: You know, we don’t have too long on the podcast, [Laughter] but if you can kinda give us a little bit of that path that you’ve been on.

    Willson: Yeah, yeah. You know, I kinda looked at it like a, it was almost like a barracuda eating a great white who was eaten by a whale, right?

    Ashley: [Laughter]

    Willson: It was a really interesting, very quick journey. It was a couple years ago when I was with Automic Software. We were acquired by CA Technologies, and about a year or 18 months after that acquisition or so, Broadcom bought CA Technologies to the—not just our surprise, but of course, to the surprise of the entire market.

    Ashley: Mm-hmm, yeah, it was.

    Willson: There was a lot of press about it.

    Ashley: It was a surprise.

    Willson: And so, yeah, it was a very, very quick and interesting turnaround. We are now known as CA Technologies, a Broadcom company. As far as Automic goes, we are retaining the Automic brand, so you’ll see a lot of our old software and all that will have the Automic brand. And not just Automic, but our current leadership has really felt to really keep and embrace the brands that we’ve had. So, you know, Blaze—CT is still gonna be Blaze, and Rally is gonna be Rally, those type of things.

    Ashley: Mm-hmm.

    Willson: So, the brands are remaining, which I think is useful, because even though CA had remained Rally—I know I, anyways, always refer to it as Rally, because—

    Ashley: I do, too, yep. [Laughter]

    Willson: – right, because that’s what I always knew it as. [Laughter]

    Ashley: Especially knowing them from the beginning, it’s hard not to call them Rally, but yes.

    Willson: Right, exactly. So, Automic is still going to be Automic from a product brand positioning. And then from a company standpoint—yeah, we’re still CA, just a Broadcom company. Within Broadcom, our group is known as the Enterprise Software Division, so we are a distinct division within Broadcom.

    Ashley: Mm-hmm.

    Willson: And then, to add to that, to the little journey, what has been interesting to me is—I remember it was a year ago, the last Forum meeting I was going to or what many of us in the inside called those meetings, the Gene Kim’s Great Slumber Party.

    Ashley: [Laughter]

    Willson: Right? We kinda talked about being lean, and some of the people, I remember having discussions with and running papers, we just really got to talking about what it means to run lean and the lean manufacturing and the whole thing, right, that DevOps is based on and that The DevOps Handbook espouses and all these kinda things.

    And after that, right, three months after that meeting it was announced Broadcom was gonna buy us and then they actually did, and here I am all this time later. And I bring that up because what was interesting to me, Mitch, is that Broadcom embodies this. They run lean.

    Ashley: Mm-hmm.

    Willson: And it’s been an interesting journey to go from the short time at CA, which ran big, to now being with Broadcom, which runs lean. And they run even leaner than what Automic did, which was in ISV, right? You know, Independent Software Venture.

    Ashley: I was curious about that, because I know Broadcom, you know, I had worked with them as a chip supplier with companies like Broadcom ad Qualcomm and, you know, manufacturers of chips in telecommunications think about the world, they’re much more, you know, operational efficient. It’s all about fractions of pennies and the cost of their goods to maintain profitability, but that’s super high volume. So, it’s a different mentality than a software company.

    Willson: It is, and there are some significant differences, I suppose, from the business or sales aspects. But from a production aspect, there really wasn’t a whole lot of difference. When you think about taking a product to market, right? Now, we were doing this with software, they’ve done it with chips, but Broadcom is an engineering excellence company. They really pride themselves on having the best of fill in the blank of engineering, whether it’s engineering processes, engineering whatever. In fact, what has been interesting post acquisition is that our engineering staff for the bulk of our products have actually increased. Broadcom is very keen on ensuring that the products we deliver are best in class.

    Ashley: Mm-hmm.

    Willson: And so, they realized a way to do that in the market is to make sure they’ve got the top minds developing and producing and shipping these products, right?

    Ashley: That has to be refreshing to you—you and the team.

    Willson: Oh, yeah. It was very interesting to see that that was the case. It’s been a unique acquisition as far as that goes, and to see that—no, they wanted to increase the head count of R&D. [Laughter] It’s important for them.

    Ashley: Mm-hmm. What? [Laughter]

    Willson: Exactly. Like—wait, what? They want the products to improve, to get better, and to continue to be viable and market leading. And so, they embody the lean processes to do all of this stuff. And so, it’s been—there’s a little bit of a learning curve, I think, on both sides to adapt to this new way, but here we are. You name the things that are automated—I mean, Broadcom automates just about anything and everything. The very stuff you read about, Gene Kim talks about in his book, saying—and even in Nicole Forsgren’s book, Accelerate, you know, automation is at the key of a lot of these things, and Broadcom embodies that.

    Ashley: It is. It’s a much different kind of company. If you’re not a hardware person or haven’t worked at a chip company, what I said, it’s—fractions of a penny make a difference in cost, and that cost can be in materials, but it also can be in process. And when you multiply that by millions upon millions of chips that they may ship a particular line, that’s a lot of money. So, it makes a huge difference in the profitability of the company—

    Willson: That’s right.

    Ashley: – to be able to reinvest and give you more engineers to be part of your team.

    Willson: That’s right. And so, software is a little different, right? It’s not the pennies like that. Obviously, with software, the margins are different, because you’re not dealing with a physical thing, a commodity, right? But the principles to be efficient like that are absolutely espoused by DevOps. And one of the things that this transformation internally has occurred—because we’re transforming ourselves from a software company, right? I think we’ve made some press releases talking about our BizOps platform, a holistic platform—

    Ashley: Right.

    Willson: – that we’re offering the market. No longer should you be thinking of us like the CA of old where you had a bunch of different siloed tools. Now, they’re all coming together in a holistic way to provide real business value and create a true, interesting platform for everybody.

    Ashley: Mm-hmm.

    Willson: But just this way of doing things lean like that and embracing the agile way of turning things around has been a cool thing to behold.

    Ashley: That has to be really good, because I haven’t talked with somebody that’s had that experience—not that others haven’t, but to have that common lexicon frame of reference of thinking about producing software like a software factory just like a hardware based factory. You have a lot of synergies and commonalities there. Yeah, there’s differences, too, but instead of software being this mystery to hardware guys or vice versa, right, you can talk and work together and understand the efficiencies, improvements, speed, impact, quality—all the things that drives the manufacturing company also drives software.

    Willson: That’s right. And what’s been exciting, too, with this opportunity to change—really, the Broadcom offer has provided us an opportunity to change things and to really force these integrations and synergies between what used to be separate divisions of CA but are now different teams and the interoperability that’s happening between them to deliver this BizOps platform is really exciting. It’s cool to see that now it’s like—look, we have all these things, but what’s the point, unless we’re focusing on business outcomes, right?

    Ashley: Right.

    Willson: We can actually drive a business outcome if there’s harmony and cohesiveness between all these things, and that’s very exciting. In fact, one of the other things along these same lines that was announced is the free tier program—I don’t know if you see the PR a couple weeks ago, Mitch—

    Ashley: I did.

    Willson: – but now, a lot of our products are offered for free.

    Ashley: Yeah.

    Willson: So, for example, a continuous delivery director, or Automic Continuous Delivery Director, if you go to CDDirector.io, you can just use it for free, up to 10 active releases. And Rally has some free tier as well. So, basically, we’re now offering the freemium versions of what we had, and—

    Ashley: Have the products changed substantially to be able to do that? Are they less functional or are they time based or just volume, kinda getting folks to up to date?

    Willson: It’s really just volume, right? Yeah. They haven’t really—well, a lot of them didn’t really need the change. A lot of the products that are offering this are the ones that are already SaaS based already. There will be more that are coming, of course, as things are being migrated to be SaaS based, and that effort is under way.

    But an example of Automic Continuous Delivery Director, there’s not really a time limit. It’s 10—10 active releases. That’s a lot of releases.

    Ashley: Mm-hmm.

    Willson: And you can just use it, and use it at will for that. You wanna get more than that or grow it across the enterprise or make it the standard or something, same with the Rally and Blaze CT. Oh, well, yeah, then you can buy the licenses that let you use it more broadly, right?

    Ashley: Mm-hmm. That’s a great model. I mean, I think we’ve sort of disproven the crippleware approach. Nobody likes that.

    Willson: [Cross talk], yeah.

    Ashley: Exactly, and being able to use it on enough of software production; you talked about 10. You’ve gotta be able to get a pretty good feel for what the product does that’s gonna do what you want it to do and start to integrate it into your tool chain workflow platform.

    Willson: Right, and these are 10 active releases. In other words, 10 releases in parallel, right?

    Ashley: Mm-hmm.

    Willson: Well, that’s—like you said, you should really be able to get a feel for it.

    Ashley: Yeah.

    Willson: Being able to run that many releases in parallel to each other.

    Ashley: Well, that’s awesome. That’s great to hear. I’m excited to hear about more of that coming from you, too.

    Do you have a new release coming up, here?

    Willson: Yeah. So, speaking along those lines, we have a new release for Automic Continuous Delivery Director. We’re pretty excited about this release. It is our first release that includes some ML capabilities and heuristics that are involved. Basically, what we were looking to do with this is, amongst other improvements and advances we made in the product, right, to help with continuous delivery, one of the things we looked at is that there’s significant bottleneck in continuous delivery, the methodology, which is QA or testing.

    Ashley: Mm-hmm.

    Willson: Now, as you probably know, Mitch, continuous testing is technically a subset or a component of continuous delivery.

    Ashley: Right.

    Willson: And I might argue that unless you’re doing continuous delivery, you are not doing continuous delivery.

    Ashley: Right.

    Willson: You need to be doing—

    Ashley: You need to think of continuous delivery is plan, dev, test, deploy, operate, right?

    Willson: That’s right, and you’re testing continuously, right? But the problem is, QA wasn’t really built to be agile themselves. So, they kinda became the bottleneck, not just from the way that they run, but you, as a QA person, right, testers, you build out all these tests. And so, you release this small change per DevOps per agile per continuous delivery, right? And it goes into QA and you run, you know, 1,000, 10,000 tests—and I’m not making these numbers up. Literally, a lot of our large enterprises have thousands and 10,000 tests.

    Ashley: Oh, yeah. Totally.

    Willson: Alright, so then, we’re in the waiting game, right? And maybe it takes however long for the test results to come back and then you get feedback to development. So, we found with our, a lot of our large customers, you know, the Global 2000, that this was a significant bottleneck. So, now what we do, what occurs, is that when you do that release and you’re a pipeline, right, you’re a release pipeline and it gets pushed to QA, one of the things that we do is, we look at several heuristics.

    So, we come around and say, “Alright, well, what are all the tests? What is the historical responses or outputs of the tests?” Right? How many of them failed? Which tests are flaky? How many tests are new tests? We want to actually run these tests first. So, a flaky test, for example, it ought to run at the front, and if it fails—pfft, we’ll send that feedback immediately to development or to QA so that we’re identifying these things earlier, quicker, faster, rather than having to wait on the back end for this, right?

    Ashley: Try to get to that 80/20, right? Get the things—

    Willson: That’s right.

    Ashley: – upfront that are gonna cause the most issues, find the most bugs.

    Willson: Right. So, we’re looking at all the data and intelligently figuring out the tests for you. The other thing that we’ll do is that we’ll look at the code changes you’ve made and we will map those code changes. At least in the case of Java apps, we’ll be able to map the code change to specific test suites.

    Ashley: Mm-hmm.

    Willson: So, now, when you submit that one component change, you are actually getting identified just the test runs that affect or exercise that component. So, altogether, when you put it all together, Mitch, what occurs then—now, when a change gets pushed into your continuous delivery pipeline, when it hits QA, you’re in a position to not have to run all 10,000 tests. You’re in a position now to run the high risk tests, the tests that you have marked are critical and important, right? So, those are included, that’s taken into account, too. And the test that we know actually exercised the change that you put in.

    So, you’re able to run what you have to run to ensure you have a high quality and also order them in such a way where there’s high likelihood of failure that Development is fed back that feedback earlier and faster. And so, we’re very excited about this release.

    Ashley: That’s awesome.

    Willson: Yeah.

    Ashley: Boy, gone are the days, I can remember when I would judge the quality of the release by how severe the bugs are and if we haven’t found really big problems yet, we’re not through. There’s more to come. [Laughter] You know, that was the heuristic. [Laughter]

    Willson: That’s right. Exactly. That was the heuristic.

    Ashley: That’s right. We’re far from those days, aren’t we? [Laughter]

    Willson: Well, you know, because one of the other constraints, if you think about agile and continuous delivery, right—small change and you make more frequent small changes and the “State of DevOps” report just came out, right, a week ago or this week, and it again is showing how this way, this approach is actually reducing the bugs and reducing ________ quality.

    All those things are great. But that now means one of your biggest resource constraints is time. And what we’ve found with our customers, it was time in QA. So, by intelligently assigning the tests on a run with a particular change, we’re able to shorten that down. You’re not having to wait for all 10,000 tests, you’re just waiting for the five that matter to run against that little small change and—pfft, get it shipped out the door.

    Ashley: Exactly, right. Be smart about testing, yeah.

    Willson: That’s right—be smart about testing. Exactly right.

    Ashley: Now, do you have a date when this is going public? Are you in beta now? Where are you at?

    Willson: Yep. So, if you were an on prem user of the product, it is actually GA now. It actually was up on the download site yesterday and—

    Ashley: Oh, nice.

    Willson: – it can be downloaded and you can go through the upgrade process. If you’re on our SaaS platform, you know, October 2nd, it’ll just be online working and available for you in the beauty of SaaS, right?

    Ashley: Nice. Well, great. That’s fantastic. Awesome. Well, congrats on the great release. I hope it works out very well and also the free tier brings some new customers, new people onto your platform and expand that as well.

    We have a few minutes left. I know you’re gonna be at the DevOps Enterprise Summit, correct, in Las Vegas?

    Willson: Yeah, exactly. So, we’ll have a big booth there at the DevOps Enterprise Summit, so certainly looking forward to seeing everybody wearing our Broadcom wear. We’re gonna—not to give away too much, but I think we’re gonna have some pretty cool giveaways at our booth at various times, so, you know, definitely come check us out.

    Ashley: Come early, come often for the swag.

    Willson: What—get the swag? [Laughter] That’s half of what you go for. Especially if you have kids, you know? I see some of these people, they’ve got bags full of this stuff, right?

    Ashley: Mm-hmm.

    Willson: And I’m like—yeah, you’ve got kids at home. [Laughter]

    Ashley: You don’t wear a Small—you don’t wear a Small. What are you—what are you doing, there?

    Willson: [Laughter] Exactly right. Exactly right. But we’ll have some things, a couple of the giveaways we’ll have I don’t think you wanna give to the kids, let’s just say that.

    Ashley: Okay.

    Willson: And if you’re going to the Gartner Symposium, we’re gonna be at the three upcoming Gartner Symposiums in Orlando, São Paulo, and Barcelona as well.

    Ashley: Fantastic.

    Willson: So, yeah.

    Ashley: Well, good. We’re gonna have to check in every six months or so just so we know what swag wear, logo wear to look for you in.

    Willson: [Laughter]

    Ashley: This time it’s in Broadcom—I’m kidding you, of course. But almost—it’s almost true.

    Willson: It’s almost true. Feels true. [Laughter]

    Ashley: Scott, it’s been great having you on the podcast. Appreciate you being here.

    Willson: Thank you so much, Mitch. Appreciate it and look forward to seeing you and Alan at the DevOps Enterprise Summit.

    Ashley: Absolutely. Look forward to it as well, and great to catch up with you about everything happening. Well, I’d like to thank you. I’d like to thank Scott Willson, Product Marketing Manager—or sorry, Product Marketing Director of Release Automation at CA Technologies, a Broadcom company—I think I got that right—

    Willson: You got it right.

    Ashley: – for joining us on the podcast. [Laughter] And thank you—you, our listeners, for being here as well. This is Mitch Ashley with devopsy.kinsta.cloud. You’ve listened to another DevOps Chats podcast. Be careful out there.

    — Mitchell Ashley