Tag: it automation

In front of next month’s ChefConf in Seattle, Chef has announced a major refinement to its business model. Affirming and clarifying its commitment to open source business models, Chef will now and in the future release all of its software as open source.

In this DevOps Chat, we speak with Corey Scobie, SVP of Products and Engineering at Chef. He explains that Chef will be adding services, training and support to the open source software, as well as packaging all of this together.

For the full story, be sure to listen to the podcast and check out the video version of this interview below, along with coverage on devopsy.kinsta.cloud.

[videopress nUXwRMxU]

Transcript

Alan Shimel: Hey everyone, It’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to a DevOps Video Chat. Today’s DevOps Video Chat features Corey Scobie, SVP of Engineering, is it?

Corey Scobie: Product and engineering, yep.

Shimel: Product and engineering now at Chef. Hey, Corey, welcome.

Scobie: Thanks, Alan, happy to be here.

Shimel: I’m happy to have you here. Today is a red letter, big day in the history of Chef. You guys are announcing some pretty significant changes to the model is I guess what I would call them. Though, I don’t think the changes are as big as some may initially feel, but let’s go over them. I’m going to let you take the wheel here. Tell our audience.

Scobie: It is a big day today for us at Chef. We are announcing some changes to the way we both produce and ultimately distribute our software, which I think are really exciting. There’s a lot going on in the industry right now. As you probably know, Chef has been one of the marquee players in the open source software development community for some time. Part of the announcement that we’re making today that I think is most exciting, is we’re announcing that what we’re going to do is going forward, produce 100 percent of our product-oriented software in that open source model.

For us, that’s a change because we, over time, and through the addition of additional projects and additional software at Chef, had started following a little bit of an open source, open core path, which means that we had some proprietary software and some open source software, but when we got down to it and we thought about the history of Chef and the roots of what has made Chef such great software in the past, it really came down to collaboration. We realized the best way to make software is to make it in collaboration with the people that actually use the software, at the end of the day.

For us, that’s the big first talking point about the announcement is that we are committed to 100 percent open source software development going forward. We’re committing all of our software to open source repos and GitHub, and then ultimately attaching it and giving it to the commons with the Apache 2.0 license.

Shimel: Sure, so Corey, you know – so you’re talking to someone who has a bit of a history in open source, 20 years. I remember when the whole open core movement first came out. Look, Richard Stallman and that crew, they were really against open core, because they felt it was – it was almost heresy to the whole cathedral and bizarre open source kind of crowd, where if it’s open, it’s open. You can’t be a little bit pregnant. But, nevertheless, we’ve seen companies succeed with this open core or premium, where the core is free and there are premium modules on top of it, but it’s a slippery slope. In all honesty, I think Chef got caught in a little bit with that slippery slope with what’s truly open, what’s free, what is our proprietary that we make money from? I don’t know if the market really got it.

Scobie: Yeah, one of the things that is really tough when you’re following an open core strategy is drawing a very clean line between what is free and what is not free in the world of your technology offerings. For us, that got complicated and then even conflated further as we added more projects to the list. When we opened the InSpec project for security and compliance, when we opened Habitat for application automation, what ended up happening is we had multiple projects that were governed different ways that had different lines between the free and the core part of it. I think ultimately at the end of the day, had I been at Chef during that part of the journey, I probably would have made the same decisions, I’m guessing, but at some point you get back to this reconciliation point where it’s like, “Look. The best thing that you can do for customers if number one, you can collaborate with them, and number two, you can have a really, really clean line between what is free and what is not free in your world.”

For us, it becomes super simple with this announcement. What is free is all of the intellectual property that we’ve contributed to the commons in the world, and will contribute to the commons going forward. What isn’t free is all of the work we do to make a turnkey, enterprise-class, package that enterprise-class customers can pick up and distribute at hyperscale in their organizations, from that point.

Shimel: That brings us to the next thing, and of course this is always the issue. So if you’re going to make all the software and the IP open, well how the heck do you make money?

Scobie: Sure. As it turns out, we’ve got venture capital funding, investors that want to see us turn that into dollars as well. For what it’s worth, we’re feeling pretty good about the shape of the business right now. We just came off the best quarter of revenue we’ve ever had at Chef, prior to this business model, this model change around how we distribute software whatsoever. So we’re feeling really good about the shape of the business, but just the inability to have that simple conversation about what’s free and what isn’t. What we decided was as we’ve matured as a company, and as the space has matured in enterprise automation, what we found is the majority of our customers actually do want to have a very material commercial relationship with us.

Not only do they want the software, but they want the support. They want the content. They want the expertise. They want the warranties and indemnities that go with all of that. So for us, it was very simple. We realized at the end of the day, that the most — that there was a huge constituency of our customers who what they wanted from us was a very curated commercial offering out the other end. So we’ve decided that’s where we’re going to put our effort and our energy, and we’re going to trust in the open source world to fill in the gaps where that isn’t the offering that they want as well.

Shimel: We look at this model, Corey. Obviously, Red Hat’s the poster child for much of the open source world in that most of their software, if not all of it, was in fact open source. It was all in the packaging and the support, and that kind of thing. It sounds very much like that’s going to be the model, or am I wrong?

Scobie: It is – no – there’s a strong analog to what Red Hat has done in the industry as well. What’s interesting is I think that that is the antithesis of what we are seeing from a lot of the more recent moves in the open source industry where there’s a desire to become more proprietary in some cases, or in other cases to try and somehow bend, I think, the definition of what is open source software. For us, the core of it, we get back to the four freedoms, and freedom zero is about the freedom to use the software, but it’s free as in freedom. It’s not free as in dollars.

Shimel: Free as in freedom, or free as in beer, as we used to say, but Corey, to be fair to these companies that are – that we do see going another way, it’s because they didn’t cut it. They tried the Red Hat model, and for whatever reason, they didn’t cut it and they had too much pressure from their boards and the VCs and investors, and so forth. So they tried to take a path that was quicker to money. I’m not here to judge anybody’s – as someone in the Godfather once said, “Far be it from me to disparage what someone does for a living” –

Scobie: I hold the same opinion, Alan, by the way. I sit in judgment of nobody on the decisions that they make about what’s right for their business. I just know that for us, what’s right is to try and both honor the open source roots that we came from, and to also serve the commercial customers that were really knocking at the door and saying, “Look, this is what we want from you. We want even more engagement. We want even more value out of your commercial subscription.”

Shimel: Agreed. Before we jump into the next part of it, which is how it manifests itself within the Chef universe, I wanted to just tie a knot on this one, and mention that when we talk about this model, and you said, “Oh, we’re going to count on the open source community to fill in the gaps,” and one might say, “Well, what are those gaps? I think people have to realize what made Red Hat great was not just distributing Linux. It was that RHEL, and Fedora even before that. It was that package, that distro, that had everything you needed and wanted in there in a really easy to use – you didn’t want for anything else. It was all right there, nicely done for you, instead of running out and having to download different packages and distros from all over the internet, which is what held back Linux on the desktop for so many years. There wasn’t that one package that you could do that.

I think that’s what you’re referring to is Chef’s strategy going forward is to have that killer distro, with all the right packages in it. If you don’t want to pay Chef for that, and I don’t think at this point, though people like software that is free as in beer, and they get free as in freedom with it, no one denies that a company should be able to make a living, that a company should be able to make a profit, pay its employers, pay its developers, and go forward. So for people who either don’t have the money, or don’t have the inclination to buy the packaged product, they can get the open source components, and the community can work on giving you the glue, in essence, that puts these all together. Correct?

Scobie: Absolutely, for sure. Santos came out of that exact scenarios. The scientific community needed something that wasn’t going to be necessarily nearly as curated as Red Hat and Linux was, but they needed a platform for scientific computing, so CentOS was born and has a place in that world. I think our vision for that is exactly the same. If there’s a downstream project that is born of Chef InSpec or Habitat, we welcome it into the community, and we look forward to whatever innovations they bring, and we want to collaborate with anybody that wants to come and take that work out of the commons and create the derivative work out of it, that allows them to meet whatever their goals are. That is the freedom in free and open source software.

Shimel: Absolutely. I think one of the other things though, Corey, is when we look at Chef over the last couple years, we’ve seen the whole InSpec line of security functionality, and we’ve seen Habitat. Look, Habitat was an ambitious piece of – well, it was more than a piece of software. Habitat was an ambitious play, but because there was so many balls in the air and going off in different directions and open/closed, proprietary/not, people never – it wasn’t explained well enough to people. I spent the time at ChefConf so I understood Habitat well, but there’s maybe a thousand people at a Chef conference as opposed to millions of people in the world. I think it was really hard getting InSpec and Habitat, carving out their niche in the world, amidst this whole confusion around the closed/open, supported/paid for, free/not free.

Scobie: Yeah, that’s fair, and I think this really helps us simplify the messaging around that too. I will openly admit I think InSpec is an easier basic value proposition to parse, which is “Look, I need security and compliance in my organization and the right way to do that is express it as code”, so that should be fairly straightforward, and relatively noncontroversial. I think, with respect, to Habitat, what is interesting about Habitat is I agree with you, super ambitious project right from the get-go to redefine the abstraction of packaging and orchestrating the delivery of software in the world.

That’s – it’s huge that there’s a huge surface area associated with that. I think what’s interesting is that we’ve learned much – we’ve learned a lot over the last couple of years about how to help people in the mental journey of how to think about Habitat in the world, and how it relates to Chef, and having that foundation of understanding and knowing Chef, which tens of thousands, if not hundreds of thousands of people already have, the ability for them to jump to where Habitat plays becomes much easier in the world where we can talk about them collaterally and not independently of one another.

Shimel: Fair enough. I think that’s a great segue, Corey, to the next part of our conversation I wanted to have with you, and that is okay, how does this manifest itself down the line of Chef product and services? So we’re going to put all our IP out as open. I guess we should mention also, I think everything is being released under an Apache 2.0 license.

Scobie: That’s correct. Apache 2.0.

Shimel: Industry standard, GPL, Apache are probably the two biggest. How does this manifest itself? What changes does this mean for Chef’s products?

Scobie: On flag day when we release new products with new packaging and new terms and conditions, it’s an evolution that we’ve been going through for some time now, which is really to understand what the interface parts are between the three core engines that we have. So we’ve created an engine for infrastructure automation, which is the Chef project, and going forward, just from a purely – from a branding simplicity perspective, it’s going to be known as Chef Infra, which is I think is fairly straightforward.

Shimel: I’m sorry. Can you repeat that though? Chef…?

Scobie: Infra, as in infrastructure.

Shimel: I-N-F-R-A. Okay.

Scobie: Yeah, and then we’re going to have Chef InSpec, for security and compliance, and the state relationship between infrastructure and security and compliance are pretty – it’s pretty intuitive, which is that if I deploy an operating system, and I want it to be locked down in a certain way and secure and meet a certain set of compliance standards, then testing that on a continuous basis, managing the drift, and being able to automatically remediate any drift that’s happening there seems like a really obvious relationship. What’s interesting though is the relationship between infrastructure and application automation. This is where things get really murky, even for longtime Chef customers who have been wildly successful in using Chef as an infrastructure automation tool. If you go and talk to those customers, what you’ll find is that they’re doing about – 30 percent of what they’re doing is about infrastructure deployment provisioning and shaping, locking down configuration management.

70 percent of what they’re doing is deploying software on top of the operating systems in service of some application that’s going to come along at some point in time and need that – those software dependencies to be there. What we discovered, and this is really the genesis of Habitat, which I’m not even sure – we knew that what was happening in the space of Chef was not working at scale for us. That’s the reason that some folks at Chef went off and decided to create Habitat, but what we didn’t realize is why it wasn’t working as it was at the time. The reason is that there is a natural boundary for building systems from the bottom up in a layer cake approach. It doesn’t matter whether it’s Chef or Antival or Puppet or Kubernetes clusters or whatever, that there is a natural boundary layer, if what you’re doing is building systems from the bottom up.

So you have an operating system, and then you have system libraries, and then you have packages, and then you have middleware, and eventually you have an application. At some point that dependencies graph, the modularity of everything that you lay down, and the dependencies graph gets so complicated that there’s no possibly way you can manage understanding what the implication of changing component X is on all the other components that rely on X. So the beauty of Habitat and where Infrastructure, and where Chef and Habitat come together is that Habitat takes the exact approach. It says, “Look, what you should do is you should define an application, all of the application’s dependencies and all of the orchestration steps for that application to run successfully in any environment regardless of where it is.”

So by doing that, and by removing that layers – that bottoms up layers dependency, what we’ve really done is created an application stack, an enterprise automation stack is what we’re calling it where infrastructure and application automation marry together at a very, very clean abstraction level. The abstraction level is essentially, if you’re running software on the operating system that runs above the kernel, it should be defined by application packages and orchestration, and not by infrastructure packages and orchestration. Now, it’s architecturally it all makes sense, and it actually works really well in practice. Where it’s complicated is that organizationally, INO – traditional INO operators and other infrastructure groups have been tasked with more and more software deployment on top of their stacks. So this is a very paradigm-shifting idea. I think what is helpful for us is the enterprise automation stack allows us to create a framework and a template for how to do that in a way that helps customers adopt that methodology much, much quicker.

Shimel: Fair, fair. So we got Chef Infra. We have Chef InSpec.

Scobie: Yes, and Chef Habitat, Chef Habitat.

Shimel: Chef Habitat sits – I wouldn’t say it sits on top of them.

Scobie: Look, the stack is – we have trouble with this with our marketecture diagrams. If, what happens in the podcast now is you’ll throw up the marketecture diagram you’ll have, what you’ll see is a linear stack, but the state relationship between those three things is much more complex than can be defined in a diagram like that.

Shimel: _____ diagram, sure.

Scobie: Then on top of that, then there’s two other pieces to the portfolio. So we have Chef Automate, which is the observability layer on the top. Of course, if you have automation on all of these things, you want to be able to see across all of your pipelines, across all of your applications, across all of your systems. What is the state of everything that’s running? How has it been deployed? Is it in compliance or not in compliance, etcetera? So we’ve got this whole enterprise dashboard with automation up top. Then on the bottom end of the stack is we need to make it easier for developers to get started. So all of these engines are representative of either infrastructure compliance or application automation as code. What do you do as a developer? You download a bunch of tools. You get started writing code that are the artifacts that actually drive that automation. So Chef Workstation is the compilation of all of those tools and an easy way to set up your development environment to get going. That’s the entirety of the enterprise automation stack. It’s Chef Workstation, Chef Infrastructure – Chef Infra, Chef InSpec, Chef Habitat, and Chef Automate on top.

Shimel: And repeat for me, what all of these packaged together is the –

Scobie: Yeah, the Chef Enterprise Automation Stack.

Shimel: That’s the whole enchilada, so to speak. This is all packaged up into one coherent type of product, if we can call it that. Fair?

Scobie: Exactly. Yeah, one coherent distribution, absolutely.

Shimel: Corey, you know the news broke today around this. Some are saying, “Well, Chef is pivoting. Chef is this.” How do you view this? How do you and the other folks at Chef view this? Is this really a pivot? To me, it’s – I don’t see it as a pivot.

Scobie: Yeah, it’s not a pivot. I mean, first of all, I don’t think it’s – we’ve – you can question the effectiveness or the efficacy of our open source strategy in the past as we’ve meandered through different phases of that, but this is us staying true to the fact that we are an open source company, and so I think we’re leaning into the business model by which Chef was founded, and by which we’ve been operating for quite a while.

Shimel: I agree with you.

Scobie: I think from a technology perspective, we just have more clarity now in how to deliver the components and technology that have become critical to customers who are trying to solve large-scale enterprise automation problems. I don’t know that I would really describe it as a pivot. I think what we’re trying to do is use this as an opportunity to reintroduce people to Chef who maybe haven’t heard of the latest of our innovation in InSpec and Habitat, as an example, who maybe are not familiar with the work that we’ve been doing in the last three or four years. For us, it’s a little bit of a rebirth, but I wouldn’t call it a pivot.

Shimel: I don’t know if I’d call it a rebirth. So here, look, you’re in the forest.

Scobie: I am in the forest. There’s no question.

Shimel: [laughter] I’m outside the forest. To me, obviously a pretty – I’m more than a casual observer. To me, Chef was always an open source company.

Scobie: Always.

Shimel: I think in some ways, what we’ve seen with Chef over the last five years, I would characterize as growing pains for an open source company trying to come to grips with remaining true to its open source roots, while at the same time recognizing you took other people’s money, and that’s how – I lived that life for 20 years. You take other people’s money, they want – it’s a funny thing. They want to get a return on their investment. It’s like trying to serve two masters. It’s a hard – it’s a hard –

Scobie: Not only that, but look, the successful commercialization of any open source project or software company is how you actually create sustainable good in that community for the long-term. Obviously, you need participation from the community, but you also have to inject capital into that community if you want sustained innovation.

Shimel: You have to extract revenue as well though. That’s where I think it gets dicey. So when I look at InSpec and Habitat, I almost feel at some level, they were somewhat hobbled by having to live in this schizophrenic, open/not open, making money/not making money. Well, if it’s totally open, we’re not making money. How much do we put into this? There was just too much of this all over the place, where to me, this is just a clarification. Look, Chef is and will remain an open source company, and our value to our investors is that No. 1, we have this tremendous community around us who are users of our software as well as contributors to the community, but No. 2, we are now offering a proprietary, a for-profit solution that takes the best of this and packages it up nicely.

Scobie: Absolutely. Look, Alan, you make a great point. I would love to give a shout out to the community, because the Chef community is absolutely amazing. I know you’ve been to ChefConf. It’s an experience in and of itself. It’s hard to think of another tech conference that is as authentic and as welcoming and as diverse as ChefConf is, and it’s a bunch of people that come together every year, because they are passionate about solving the challenges of infrastructure management at scale, and application deployment at scale, and all of those things that Chef is interested in as well. So the community is a really key factor of it, but to your point is that there’s just this growing demand in the enterprise. We are the experts in figuring out how to do that in a really, really tight curated way. Of course, we’re going to take advantage of that market opportunity as well.

Shimel: Sure, we got the news out. Is there any part of the news we’re missing, Corey?

Scobie: No, I don’t think so. That really covers the announcements for today. I think the only other thing is we’ve got ChefConf 2019 coming up in Seattle May 20^th – 23^rd in Seattle. It’s going to be a great event. We also have a ChefConf for the first time ever, ChefConf coming to London at the end of June. It’s June 19^th and 20^th, I believe are the dates in London.

Shimel: Really?

Scobie: Yeah.

Shimel: You know I’ll be in London those dates. I’ll have to talk to you about it.

Scobie: Yeah, absolutely. We’d love to see you there, and we’d love to see others there as well. Yeah, so we’ve got a couple big events coming up where we’re going to showcase the latest of the innovation that we’ve had on all of the fronts, innovation, compliance, application automation, our enterprise consoles, all of that kind of stuff. so those are going to be fantastic events coming up. So other than that, we’re looking – we’re really looking forward to getting back to those community roots and really everybody being there together in the trenches together building the next generation of what this is all going to be.

Shimel: Cool. So now let me ask you a question. Put on your wizard’s hat. If we’re talking about ChefConf 2020 now, a year from now, this strategy, this road has been – this strategy has been implemented now. We’re going down this road. How does it manifest itself over the next 12 – 24 months you think?

Scobie: Look, we’ve been talking about that from a product strategy perspective, and I think we’ve got – there’s a lot of opportunity for us to do great work in pulling the pieces of the puzzle together, even more explicitly. We’re doing a lot of work with partners, so I anticipate that the ecosystem partners will be a big player in ChefConf 2020 as well. So we’ve been working with the likes of everybody from Microsoft, Azure DevOps, and the workflows there. We’ve got – we’ve been doing some product work, and continue to do product work with Amazon in the ops work for Chef Automate, managed and hosted to platform there. We’re quite engaged with GitHub because we’re seeing both GitHub and Artifactory and a lot of our customers in the DevOps toolchain and pipeline there. We’re doing some work with that. We’ve been doing a lot of integration with services now. So I actually think that the next phase of this is not just about the Chef parts of the stack, but how those parts of the stack connect with the rest of the ecosystem that is fast emerging in the enterprise automation space as well.

Shimel: Sounds cool. Well, Corey, we’re about out of time. I want to thank you for taking time on what’s a crazy busy day for you guys, to brief our audience on this. Congratulations to the whole Chef team. Tell Barry and everyone else, looking forward to ChefConf. Unfortunately, I won’t be at ChefConf in Seattle. I’m in Europe, but we’re going to have a full team there doing video interviews there and everything. John Willis, our partner at Digital Anarchist will be there, and Chris Reilly. So we’ll have plenty of Chef coverage from ChefConf. Like I said, I didn’t even realize the London one. We’ll try to get over there. I think that’s Tech Week in London actually, which is a pretty big week.

Scobie: Yeah, fantastic. Well, thank you so much, Alan. I appreciate you taking the time today as well. Look forward to seeing your colleagues at ChefConf in Seattle, and then hopefully we’ll see you in London in June.

Shimel: Absolutely. Hey, Corey Scobie, SVP Engineering and Product. I got it right.

Scobie: You got it right.

Shimel: You’re talking to us about some news coming out of Chef today, a little bit of a change in how they package stuff, but they’re still an open source company. Corey, thanks very much.

Scobie: Thank you, Alan.

Shimel: Take care. This is Alan Shimel for devopsy.kinsta.cloud, everyone. Have a great day.

— Alan Shimel