Author: Alan Shimel

In this DevOps Chat we speak with Damon Edwards, co-founder and chief product officer of Rundeck. What I like to say about them is, Rundeck putting the Ops in DevOps. But Damon needs no introduction to those who have any familiarity with DevOps. He is one of the early leaders of the DevOps community and continues to evangelize DevOps around the world. In fact, he is speaking at our DevSecOps event at RSA APJ in Singapore July 25, which you can find here.

It has always been a hope of me personally to have Damon as a guest on DevOps Chat. Besides being a great interview, Damon and Rundeck are a great story of an open-source project growing organically, spawning a great commercial business. This is harder than you think and Damon tells us a bit about it in this interview.

As usual, the streaming audio is immediately below and the transcript is below that. Enjoy!

Audio

Transcript

Alan Shimel: Hey, everyone, this is Alan Shimel, editor in chief of devopsy.kinsta.cloud, and we’re here for another DevOps Chat. We’ve got a really, really special guest for this DevOps Chat, it’s one of my heroes in the DevOps movement from when I first started researching what DevOps was and it’s about. I’m thrilled to have him appear on our DevOps Chat today, none other than Damon Edwards. Damon, welcome.

Damon Edwards: Thank you, Alan. Thank you for having me and thank you for those kind words, I’m excited to be here.

Shimel: Thank you. So Damon, I’m going to guess a lot of people in our audience have heard of Damon Edwards, instrumental in putting together DevOps Days, especially DevOps Days at Silicon Valley kind of drove that for a number of years. You had several different companies, both open sources and consulting, and probably maybe even more than all that, Damon, you have appeared—I don’t even know if you keep count any more—at how many different DevOps Days events or other DevOps related events where you’ve spoken at. I mean, do you know? Have you kept track?

Edwards: I don’t know, [Laughs]. I know it’s less than John Willis, because he reminds me of that on our podcast, but DevOps Café, I’m going to give a cross plug for that. We just had you on, Alan, as a guest, and you were fantastic. Yeah, I kind of lucked into just sort of right place/right time. We were really interested in how do you move fast—we called it Dev2Ops ourselves internally, our consulting business we used to call DTO Solutions that we have. And it was really all about how do you help organizations move faster? For a long time, scale was the problem. Hey, how do you deploy, hook into a thousand servers, right? A thousand servers used to be a big deal. And then it suddenly came to be that, hey, we can deploy anything to anything, that’s not really—deployment scale is not really the problem, speed is the problem. That organizations can’t move fast enough, that the pipeline is jammed, that the organization is kind of stuck in cement. And the organizations that are able to free themselves and decouple themselves and move in this new and fast way had a strategic advantage, were really kicking the butts of the big traditional companies that were kind of locked in place.

So we really started asking ourselves, why is that? Our customers started asking us, why is that? And that really got us into lean and all those things. And at the same time, John Allspaw gave his big talk at Velocity, I saw that, you know, fell in with John Willis and Thatcher Dubois and the rest of the crowd and the DevOps movement was born. And we kind of became known as the folks who are great at taking the design patterns that you see on stage at the big high flyers and translating them to the big enterprises out there. So that was DTO Solutions.

Then along the way we had a side project called Rundeck that was an open source tool, I’ll get to that in a second, but it really just took off. And after awhile, enough people called us about Rundeck, we really got excited about it, and there are two reasons why we decided to jump with both feet into Rundeck. The first being that I feel like operations is a DevOps frontier that’s not truly explored yet. I think in a lot of big organizations the dev side, they’ve got dev talking better to QA, they’ve got the build pipeline going, the little dev test cycles are going a lot faster. Maybe deployment’s happening a lot faster. But these DevOps principles haven’t really entered into full operations mode yet. A lot of big operations organizations just haven’t been able to transform themselves, and we saw DevOps, especially we saw Rundeck as a tool being used just to do that. So as a way to, number one, I think we were always product guys at heart, we wanted to see where we could take it. But number two, I really feel it’s helping push the DevOps movement to a new place.

Shimel: Yeah, I agree with you. And Damon, just again, adding a little color, you actually have a video up somewhere, it’s kind of a brief history of DevOps. For any of our listeners who really want to know kind of how DevOps got started, Damon has a great video on that that I’d highly recommend. But let’s jump into Rundeck. Damon, as you mentioned, it was a side project to what you were doing with DTO. It was open source, wasn’t it?

Edwards: Yeah, Rundeck’s core is it’s an open source product, an open source community. I Think last time we did some sort of estimate. There’s around like 25,000 different open source users out there. So it kind of became this silent—for us, we woke up one day and said, wow, a lot of people are really using Rundeck, and they were calling us saying, “Hey, we really want to do more with Rundeck,” and that led to the formation of what’s now Rundeck Inc.

And if you want to talk about what Rundeck actually is, fundamentally, from a technology perspective, it’s an orchestration and scheduling platform. Right now a lot of orchestration happens within different silos, so there’s container orchestration, there’s configuration management orchestration, all kinds of different things like that. There’s enterprise job schedulers. But we noticed that as folks are transforming their operations organizations there was this layer that needs to span all of those different silos of tools, all the different islands of automation that they have, all the different systems that they have. They want to create procedures, operational procedures that span all of those different silos. And they want a place where they can define those procedures using whatever scripting language that they want. They want to be able to define the work flow, they want to be able to set security around it, that’s huge, because what a lot of people are doing in those procedures is not just helping their operations teams be more efficient and effective but they’re starting to open up more of a self-service model. Self-service operations is the big design pattern that I think is really driving us forward.

Which is, all these people that need to do operational things throughout my organization, and as I start to transform my organization, how do I safely let them do things that could be deployment but it’s often remediation activity, diagnosing things, just doing operational things that I need to do, how do I let an ops team define those and be in charge of those and say who can do what, when and where, but allow other people to participate? And by participate, I mean two things. One is give them the button or the API to hit themselves to do the operational things that they need to get done, but also give them a mechanism to which those different teams, often in different business units, can define those operational procedures for the things they’re creating themselves, then have a standard mechanism to hand it off to operations where ops and security can vet them and then that ops and security vetting process can then turn around and give access to those other teams. So it’s like how do we let the operations decouple, let more people participate in operations, and do it all in a safe, sane and secure and audited and logged way? That’s where Rundeck shines.

Shimel: Got it. I mean, Damon, you opened a bunch of different things in here I want you to kind of flesh out a bit. So number one is this whole idea of ops, right? My personal opinion is for too long with DevOps there was a very developer-centric focus of DevOps, and really, at the end of the day, ops is where it’s at for a lot of different reasons, especially in the world we’re living in. And you’re clearly working in there. But we’re not looking to recreate silos, right? There’s a place for the development team, right? They interact within Rundeck, if you will, with the ops folks, right? This isn’t for ops only kind of thing.

Edwards: Right, right. If you look at organizations today, again it kind of goes back to our statement before about how scale is no longer the problem but organizational speed was the problem that caused the DevOps movement. And if you look at what’s going on with operations it’s a very similar thing. It’s not scale of the infrastructure as much as it’s complexity and scale of the organization that causes the problems. So in a lot of large enterprises, most every large enterprise, you have this central place called operations. Just kind of put the teams out of the way at this point, just think about the action of running all these things. You have all these different streams of activity that come into this central place, and that has to all hang together to form this enterprise. That is operations. That is the business. That’s where all the money is made. All these streams of development activity that come into that, essentially they’re part suppliers, so the next version, the next cogs in the machine that has to run, but it all has to hang together or we aren’t making any money, unless you actually sell software, which very few companies do these days, your running the service is the business. So operations is very important. But in that same vein, that operations, because things are so tightly coupled and it all falls to this one often central team, that becomes a huge bottleneck and a big problem, and it really ends up stunting or blocking the DevOps. That decoupling transformation that these organizations want to go to, they get blocked by the sort of calcification of the classic operations domain.

And so we see these high flying organizations, what they’re doing now, the high performers, what they’re doing is a frame of how to uncap and decouple all of that operational activity. And in doing so, they’ve kind of realized there are three key pieces to what you’re working with. There’s the ability to—who defines the procedures? Who defines the procedures for the things that they’re going to be running. The second is, who can push the button for those procedures, button meaning through a GUI or through an API. And then third, who has the secondary or the operational, the management control of that action?

Traditionally that all kind of fell to one group or one team and they’re grossly outnumbered by the rest of the organization and that became the big ops bottleneck. Their whole mindset was all about let’s protect ourselves from being overrun. Let’s protect ourselves because people are pushing things at us that they don’t really know how it’s all going to hang together, they don’t have that responsibility for defining things. So ops was in a very, very tough position and doing their best. We see these organizations doing a bit of a Tom Sawyer paint my fence or a jujitsu move by saying, hey, let’s redefine who does what where. Let’s allow that definition. Let’s allow those teams. They created these things, let’s allow them to participate in the definition of these operational procedures. And then let’s also, once it comes back to us, let’s let them push that button as well—for some things, not everything—maybe just certain non-destructive things, or maybe certain teams can deploy but certain teams we don’t want to deploy because it’s a different type of system or different security requirement. And you can move those things around and ultimately you can keep that third piece, the management and the compliance and the audit security control, with that traditional operations organization, or traditional operations skillset or management skillset who wants to control those things.

So in this self-service design pattern it’s about dividing those three things up—the definition of the automation, the execution of the automation, and the management security compliance control over it—and allowing those to be moved to the parts of the organization where it makes the most sense for the flow of the business, speed, taking advantage of the labor that you have. It’s that mindset that these operations organizations are using to transform their organizations. When we saw that happening we were like, wow, that’s something really cool, and we jumped in with Rundeck Inc. with both feet.

Shimel: Absolutely. Let’s talk a little bit about Rundeck Inc. as you called it. So Damon, was it about nine months, maybe more, maybe a year ago now, you and your—I’m drawing a blank, I apologize –

Edwards: Alex Honor.

Shimel: Right, Alex Honor, you and Alex have been doing Rundeck for as long as I know you. But about nine months or a year ago you brought in sort of a professional CEO, someone who I’ve known for years, a tremendous person based out in the Valley, Stephanie Fohn, as your CEO. And it really went from being Rundeck’s a side project of DTO Solutions, to Rundeck Inc.

Edwards: Yeah, we jumped in full feet. We luckily landed Stephanie, it was a big coup for us. I think she described our situation “an embarrassment of riches,” and I wasn’t sure if she was emphasizing on the embarrass or the riches part. We started what’s now Rundeck Inc. as kind of an experiment to see, hey, what’s going to happen? It was Alex, myself, Greg Schuler, as lead developer of Rundeck. Alex is the founder of the Rundeck project. We had a small team going, we had some engineers scattered about, and things were working. But next thing we looked up and we had a whole bunch of large enterprises that were paying us for the early version of Rundeck Pro, which is sort of an enterprise enhanced version. There’s all kinds of features and extra plugins that really matter if you’re doing this stuff at scale. The open source is really defined for if you’re just a team and you need to define buttons and push buttons, then the open source is fully featured and there for you. Rundeck Pro is really about kind of adding things around and on top of it that the boss cares about, that big organizations care about.

So we looked up and things were doing well. Stephanie came in and said, hey, we’ve really got something here and I want to be a part of it. So it was a big score for us. She is very well known on the security side of the house, a number of great successes there. She’s kind of found that operations and security is a very similar mindset, very similar set of people, and she’s really taken to it. Yeah, it’s been great, she’s brought in even more professional people.

Shimel: It’s grown up to a company.

Edwards: Yeah, we’ve got over 50 enterprise customers so far and we’re growing fast, so it’s a pretty cool, pretty fun ride.

Shimel: Damon, your role now in Rundeck Inc., if we can call it that, is you are CTO and obviously cofounder, but CTO and evangelist, but your hands are still getting dirty talking to customers and helping.

Edwards: Yeah, I’d say that’s mostly what I’m doing. Actually, Alex is the CTO, I’m the, I guess, Chief Product Officer, or head of product, these names that, you know, it’s a typical start-up, right? I focus a lot on the evangelism, a lot on the product side, a lot on helping customers figure out, A, well, figure out what they’re doing with Rundeck, and then also, B, help them figure out what they want out of Rundeck and driving that forward. So I’m still intimately involved with the DevOps community and out on the road doing that because I believe this is the next frontier, this is where you have to go. Straighten out your deployment pipeline and getting automated testing in there is a great thing, but if you’re any organization of any scale, this is the next major issue. You need to transfer operations otherwise you’re not really meeting the full DevOps vision. And the companies that are, have a distinct advantage over you.

Shimel: You know, Damon, to me, DevOps is still very much like right out of the goal, where you solve one bottleneck only to find the next bottleneck. And that’s what we do is we keep knocking them down. And as you knock one down, another one further down the road shows itself. And I think that’s what we’re seeing in DevOps, is as we knock down bottlenecks in the CICD arena and configuration management arena, ops becomes where the next series of bottlenecks are, and that’s what obviously Rundeck is hitting in there.

I wanted to talk a little bit, Damon, about a particular subset of this, and that’s DevSecOps. I think you almost inadvertently maybe backed in to the wonderful world of as we call it now cyber security or InfoSec or whatever you want to call it. Inherent in that ops mission is the idea of deploying securely and keeping a secure infrastructure up and securing your apps. And this is something that Rundeck is spending more and more time, more and more resources, more and more of the focus that I’ve seen anyway is involved around this DevSecOps mission. Comment on that?

Edwards: Yeah, I’d say it’s sort of the intersection of security and operations. Really security, operations and governance. That if you think about what’s going on now—especially, well, two big trends. There’s the digital transformation trend, which is really just saying all of my systems need to be interconnected now for a consistent customer experience, that’s something the business wants. And then there’s all these new server lists, containers, the new API world where everything’s an API. So now we’ve got to define these procedures that span all of these different systems that used to be stand-alone stovepipes. And, everything’s turning into an API, so I’ve got to have a way to secure that, I’ve got to have a way to figure out who’s who and who can do what. So we kind of fell into that in terms of people use Rundeck as that tool, to say, hey, I’ve got my Chef automation over there, I’ve got some Kubernetes stuff over there, I’ve got some legacy stuff over there, I’ve just got a bunch of scripts I need to run over there, I’ve got these different Amazon services I need to hit, Akamai and these kinds of things. So they’ve got all these different things, buttons they need to push, and they each have their own logins, each of their own sort of way of managed security policies.

So they look at Rundeck and say, hey, this is a place where we can enforce policy, or enforce this authentication and enforce the security controls. By defining these Rundeck jobs, it becomes a nice abstraction for that to say they can just call all this stuff, it’s real easy to string it together, and then I can define who can do what and where, and I’ve got the full access. In a lot of ways we call it like the key card and the camera. You can define who can do what through Rundeck, sort of the key card in your office building. But then there’s also a bunch of cameras, right? Not that we don’t trust you but we have to prove that you didn’t do anything bad with your keycard. So the kind of keycard and camera metaphor, we talk about that internally, but we should probably talk about that more externally. It’s something that we see a lot of people implementing with Rundeck in this new world. It’s a way to satisfy the security and governance demands but also do the new ops stuff.

Shimel: Damon, we’re a little overtime already but I wanted to just quickly mention that you’re going to be appearing at RSA APJ, which is the largest info sec or cyber sec related show in the Asia Pacific region. We’re putting on a full—we meaning Sonatype, DevOps Connect and a few others—are putting on, Rundeck is a sponsor, is putting on a DevSecOps day there, and you and John Willis are headlining, and for any of our listeners who are in Singapore or around Singapore, that’s on July 25. And on July 26 I think we’re going to try to do a mega meetup group, several meetups, different meetup groups within Singapore. Also at RSA APJ, and you and John will be presenting there as well.

Edwards: Yes, we will. You guys do a very good event. We’ve done the one here in San Francisco, excited to do the one in Singapore. Singapore is actually a great DevOps community out there with all the banking and finance and whatnot. It’s good work being done there. And the event’s great. It’s not just for security, it’s really looking at the end-to-end pipeline I’d say. It’s more like how do you bring these enterprise concerns into the DevOps story. And I don’t want people to think it’s just, hey, if I’m not in security, I don’t need to go to this. I think it’s really good for anybody who works in the enterprise and wants to think about the nuts and bolts of how do we get this stuff done so we can move fast and be secure or sure in what we do.

Shimel: Absolutely. And you mentioned RSA, Damon, actually I’ll have some news about that hopefully within the next week—RSA San Francisco next year is April in San Francisco and I think we’ll be doing a DevSecOps day there as well. I think you presented, we’ve done three or four of these, I think you’ve presented at all of them, actually.

Edwards: I’m a supporter. I’m a supporter of the Alan Show.

Shimel: Well, you know what, to me it’s about bringing these two tribes together. It was something that Gene Kim and Josh Corman tried to do and I’ve tried to help, and Mark Miller and the Sonatype folks. And Damon, you know what, it’s paying off because I spend a lot more time in security just because of my background, and I’m hearing from—I’m not getting the pushback I used to get from people about DevOps any more. They get it. Or they’re starting to get it. God hopes. Anyway, Damon Edwards, we’re way past our time, but that’s okay, it was worth every minute. Thank you for being this episode’s guest on DevOps Chat. We will see you in Singapore. You know what though? For people who want information about Rundeck, where can they get more information?

Edwards: Rundeck.com. There’s a free trial of Pro. You can always use the open source. We’re on IRC, we’re available, so we’re around. Feel free to reach out to us and we love talking about this stuff.

Shimel: Cool. Damon Edwards, co-founder, chief product officer of Rundeck Inc., and also DevOps evangelist extraordinaire. Thanks for appearing on DevOps Chat and we’ll speak to you soon.

Edwards: Thank you, Alan.

Shimel: All right. This is Alan Shimel for devopsy.kinsta.cloud, DevOps Chat, and we’ll see you soon on the next chat.

— Alan Shimel