I had a chance to speak with Julian Dunn, product manager for Chef ahead of the upcoming ChefConf conference in Austin Texas next week. Julian and I spoke about Habitat, Chef’s new open source application automation project.
Many in the industry are hailing Habitat as something really special. Our own Tony Bradley says it could be a game changer. Julian gives us some of the background and reasons why Habitat is important for you. He also gives us an advance look at the three Habitat related sessions scheduled for next week at ChefConf.
If you are already going to ChefConf, be sure to hit all three of these if you can. If you are in the Austin area next week, try to make the show. We will be there covering it live. Also the opening keynotes for Tuesday and Wednesday are being live streamed over the web.
In the meantime here is my conversation with Julian with the printed transcript below, enjoy!
Alan Shimel: Hi, this is Alan Shimel, devopsy.kinsta.cloud, here for another DevOps Chat, and I’m happy to have Julian Dunn, Product Manager of Chef joining us today. Good morning, Julian, how are you?
Julian Dunn: Good morning, Alan, I’m doing very well. I’m melting a little bit because it’s very hot in New York, but you know, other than that—perfect summer day.
Alan Shimel: Yeah, it’s July in New York, it happens. Julian, we could talk about weather, I’ve certainly been traveling enough to see different ones, but let’s focus in a bit on what I wanted to really talk to you about today, and that is Chef’s new Habitat offering. Let’s assume a listener doesn’t know what Habitat is at this point. Give them a quick kind of elevator pitch on it.
Julian Dunn: Yeah, absolutely. Now, if you’ve read any of the materials and things like that, we’re using this tagline called Application Automation. And, you know, what on Earth does that actually mean? Well, what is the problem that Habitat solves for people?
So, you’re an application developer today, and what you end up doing as an application developer is—well, what you want to do is, you want to spend a lot of your time doing the business logic and writing the business code that is actually gonna differentiate you from your competitors, right? So if you’re working at JCPenney, let’s say, you’re trying to write features for JCPenney’s website or mobile application or whatever to differentiate you from Macy’s and Target and Nieman Marcus or whoever your competitors are, right?
Well, the experience of being a developer today actually ends up being that, before you write any of that program code, you have to spend a lot of time in the weeds and dealing with what I call sort of the plumbing oriented aspects of doing the programming, right?
So just imagine if you were an interior decorator and you want to simply decorate your bedroom to whatever you wanted in there—you know, damask curtains and what have you—but you’ve spent most of your time dealing with the electrical and the plumbing. Not a particularly nice experience.
Alan Shimel: Mm-hmm.
Julian Dunn: The other thing is that, when you make those choices about what plumbing and what electrical systems you’re gonna use, that actually changes how your room is decorated. I mean, how absurd would that be, right?
Alan Shimel: Mm-hmm.
Julian Dunn: And so the notion of trying to let the application develop or be able to concentrate mostly on developing their program code and make a lot of those infrastructural choices later led us to develop Habitat, which is something that takes a little bit of a different approach than how we’ve built IT systems in the past, right?
It’s kind of back to my analogy of the plumbing. In the past, we’ve built sort of up from the bottom towards the application, right? So we choose an operating system, we choose a bunch of stuff that sits on top of the operating system. You eventually at some point, we reach the application, right?
And that’s an incredible waste of time for the application developer, who actually wants to start up here and define what their program logic is, right, and how that application should behave in a lot of different scenarios. And if you sort of start at the top and move towards the infrastructure, that’s the surface area that you actually end up owning as the application developer—we believe that that is Habitat.
Alan Shimel: Very cool. And hence the name, then, right? When you explain it that way, the name Habitat actually makes a heck of a lot of sense.
Julian Dunn: That’s right.
Alan Shimel: Yep. So, Julian, this is, some would say, you can see where Habitat is analogous—not analogous, adjacent to kind of the space that Chef plays in already. It’s a much bigger mission, if you will, than just what Chef has done to date. Is that a fair statement, you think?
Julian Dunn: It sure it, it sure is. You know, Chef, I think, initially started out, obviously, right, with a configuration management product that was very infrastructure oriented. And, over time, you’ve seen that our product portfolio and what we’re trying to do is a lot larger than just infrastructure configuration management, right?
We kind of describe ourselves today as an automation company. We solve a bunch of automation problems, wherever automation is needed to help reduce people’s time to market or burdensome activities that they need to do in a regular ________, right? To really free people up, no matter what their problem domain is to being able to do the really intelligent parts of their job—the thinking parts of their job, right? And you’ll see that as we released Chef Delivery, which is a workflow system that allows people to basically implement, if you will, the principles that are in a lot of the books out there like Jez Humble’s Lean Enterprise that tell you, “This is the shape of how you should be delivering the software out to production.” Basically something like Chef Delivery is an automation arrayification or that technology—or those ideas, right?
And then when we made the acquisition of VulcanoSec, which is the German security company, that gave us in-spec and Chef compliance that’s automating the work of compliance officers, security officers—right, folks that spend a lot of time, their job today and their pay and their experience is they spend a lot of time manually poking around systems and putting up fires manually because they don’t have a good framework and a system for describing the correctness of a particular system, right?
And so now, as we move into—right, we’re, as a product organization, you want to be one that’s trying to address people’s pain points, right? When we started looking at what the pain was for an application developer, and they spend so much time dealing with the nuts and bolts of actually writing the code, we’re like, “How—why are systems built in this way?” We really went down a path of inquiry and we came up with Habitat as a way more application developer-centric than the perhaps infrastructure-centric things that we’ve built in the past.
Alan Shimel: Sure. Just two thoughts on that, Julian, I’d like to explore with you. One is—look, I’m not, I don’t hold myself out to be a Product Manager at Chef or anywhere near that integrated into it. But it feels to me, anyway, that the Chef compliance and the Chef delivery are very tactical pieces that embrace and extend Chef’s core configuration management capabilities, right?
Where Habitat to me is more of a strategic rather than tactical type of play here, where you’re looking to set the game—the grounds, right? The stadium that we play in, whatever you wanna call it, the field, right? And so it’s a different—and again, I might be wrong, I’ll defer to you—but it feels different to me.
Julian Dunn: It is different in the sense that it’s a little bit iconoclastic, you’re right, in that it challenges some of the approaches that folks have been doing to try and deploy applications and get the sort of modern application characteristics that they want out there. And those characteristics are really great. I mean, one thing that’s really—as we developed this product, what really caught for us was to focus in on, “What does a modern application look like? What characteristics does it have?” And those characteristics are things like, it should be deployed immutably. It should be deployed atomically in that it’s either installed or it’s not, right? It should have certain characteristics built in. It should be aware of—it should know how to behave when it’s deployed in different scenarios, right, in different, what we call topologies.
Alan Shimel: Mm-hmm.
Julian Dunn: So you can think of, on a develop laptop, you might just have one instance. In a production environment, you probably have a number of instances, and they would have some relationship with one another, right? They might have one where you send all the rights two and the other ones are the reads, right? There might be some that are—like, think of a caching system, right? They right be hierarchical where there’s sort of a first, second, and third layer cache, like Squid or something like that, right?
So it’s relationships between these entities, these application components, and one thing that we realized as we’re developing Habitat is, you know, as folks develop more complex applications and those applications are actually distributed systems, folks haven’t really spent a lot of time figuring out how to describe those relationships in a way that it imbues the applications themselves with the understanding about how they’re related to other components, right? The ways in which people have tried to solve the relationship problem between these components is to create sort of a central system, an orchestrator, if you will, that defines those relationships.
And the trouble with an orchestrator is that it becomes a single point of failure, and as we know, that’s not really compatible with the cloud, right? Where in the cloud, they tell you, “You really shouldn’t have single points of failure, because you’re on an unreliable fabric,” right? So you really should have a system that knows how to recover and continue to function in the face of failure one or additional components, right?
And so that’s where we started to come about with the architecture and the design of Habitat and how aware—how self-aware these individual components should be.
Alan Shimel: Absolutely. Just one other kind of comment or observation, and that is, you’re right—configuration management and compliance seem to be operational, or at least lead with operational, where Habitat clearly is a pay in the developer specter. But it’s all supposed to be one under DevOps, right? I mean, we don’t want to—I’m not into creating divisions between developers and operators, and I’m sure you guys aren’t, either, Julian.
Julian Dunn: Right, but I’ll tell you
Alan Shimel: But clearly they’re different focus—go ahead.
Julian Dunn: I was about to say, Alan, you know, the dirty of secret of DevOps, my observation, you know, I’ve been doing this for a number of years, and I’d love to hear your thoughts on this. But my feeling about the dirty secret of DevOps, and I go to DevOps Days and these conferences, and you survey the audience and you look at who the folks in the room are, right? And it’s something like, if you’re lucky, it’s still 80 percent Ops and 20 percent Dev, right?
Alan Shimel: It is, it’s OpsDev.
Julian Dunn: Yeah, right. So DevOps has been very much about Ops taking a lot of the good practices from Dev and learning a little bit of coding and learning that there’s a proper workflow and that you should have tests for stuff, but even if you’re not doing automated tests like Chef does, at least you should have some system for testing stuff before you just go log into a box and make changes in production and whatever.
It’s almost like there’s this information flow that’s gone one way, but the information flow hasn’t really gone in the other way, even though we’ve had a number of iterations and generations of this technology. And so when we started asking the question, why—why have we kinda gotten from the brick wall to sort of more an English hedgerow, if you will, between Dev and Ops? How can we further develop capabilities to break down even the English hedgerow, if you will?
Alan Shimel: Makes a lot of sense. So, Julian, unfortunately, we’ve only got 12 or 14 minutes here and we’re running out of time. There’s a couple other things I wanted to hit. We’re recording this just the week before ChefCon, which is next week down in Austin, Texas—July 12th and 13th, or 11th, 12th, and 13th, technically. And, of course, I’m sure you’ll be there. Will there be a lot of Habitat kind of information and presentation?
Julian Dunn: There will. There’s gonna be three major talks, and we’re sort of billing them as the Habitat 101, 201, 301 kinda talks, right? We’re gonna have an intro and then sort of how Habitat fits into the ecosystem is sort of the 201 level talk, and then there’s a deep dive into why we made the technological choices that we did, the system design, and things like that, and that’s the 301 level talk.
At ChefCon, we’re also gonna have something called the Habitat Zone, and we’ll program that with a few talks as well. The speakers from the 101, 201, 301, they’re gonna answer the Q&A in the Habitat zone, and that’s gonna be near the registration area. We’ll also have sort of community building—that’s one thing I didn’t mention, I don’t know, Al, if you mentioned it, but Habitat is a fully open source project.
Alan Shimel: Yeah, under an Apache 2.0 license.
Julian Dunn: Under the Apache 2.0 license, and we believed that that, we needed to make it open source to really engender a great community, similar to how created a huge community around Chef and, you know, just ways in which folks can contribute to that Habitat community is by writing plans that can build packages that live within the Habitat ecosystem.
There’s, you know—just think of how many packages are out there within a typical kinda Linux distribution, right? There’s a lot of surface area that we would want to cover with people creating Habitat plans. That’s a great way for folks to get started. We’ll have some programming in the Habitat Lounge to talk about that with folks.
Alan Shimel: Yep. Julian, I’m gonna do my best to at least get the audio of this up before, in the next day or so if I can’t get it today.
Julian Dunn: Great.
Alan Shimel: And anyone hearing this, if you are in Austin or near Austin and you’re not already going to ChefCon, you really should look into it. You can get information, I guess, at getchef.io. I’m sure it’ll be listed there, just Google ChefCon.
Julian, unfortunately, we’re just about out of time, but you know what, maybe we’ll pick this up down in Austin and we’ll do a part two or something.
Julian Dunn: Great.
Alan Shimel: Absolutely. I want to thank you for joining us today on DevOps Chat, and great—you know, I’d love to see Habitat really take off, here, and continued success to Chef and all the gang there.
Julian Dunn: Absolutely. Thanks very much, Alan.
Alan Shimel: Okay. Julian Dunn, Product Manager of Chef, thanks for appearing on this episode of DevOps Chat. We’ll see everyone next time. This is Alan Shimel for devopsy.kinsta.cloud.
