devopsy.kinsta.cloud

Tag: Chef

  • Austin & Chef Give ChefConf 2016 That Warm, Fuzzy Feeling

    Austin & Chef Give ChefConf 2016 That Warm, Fuzzy Feeling

    I have written before about the DevOps community in Austin. It is one of the most welcoming and warm groups of people I have ever interacted with. Combine that with the HugOps operation of the friendly folks of Chef and you wind up with a great experience. That pretty much sums up ChefConf 2016 for me. Beyond the warm and fuzzy though, there was also a lot on display on what the future of business is all about. To paraphrase a line from the movie, The Graduate, I have one word for you: AUTOMATION.

    Automation was on full display at ChefConf 2016. And not just your grandpa or dad’s automation either. For those who think Chef is still all about configuration management and automating servers, think again. I am talking about automation extending into every nook and cranny of the business. You only had to see Alaska Airlines CIO, Veresh Sita’s presentation to understand what automation really means. But truthfully that was just one of many talks and sessions that demonstrated the power of automation and how Chef is pushing automation up and down the business.

    There was also the announcements regarding Chef Automate and Habitat. While we have covered Habitat already, a quick word on Automate. Automate which is years in the making, is actually the combination of Habitat, Chef Delivery which was released last year and defines the DevOps workflow, Chef itself and Chef InSpec, the compliance and security project based on the technology from the acquisition Chef made last year of VulcanoSec. All of these are combined in one enterprise class package that Chef CEO Barry Crist called “our flagship product”.

    Beyond that though to really give you a feel if you couldn’t make it to ChefConf this year, there was a palpable buzz around the place. I think the feeling is that so many good things are starting to come together. While many label many of these under the DevOps umbrella, it is automation, DevOps, Agile and more combining to enable businesses to do more, faster and better than they ever have before. In addition, as these trends continue to evolve and accelerate the pace of innovation also continues to accelerate.

    I had a chance to speak briefly with Dr.Nicole Forsgren, who works at Chef but is also one of the key people behind the State of DevOps survey. Forsgren said that is why we are seeing the gap between high performing IT led companies and those who are not continuing to widen. The gulf will continue to grow as companies adopting these practices pull away from those who do not.

    From my point of view it is amazing to see that since I first became involved in IT we have seen IT move from a cost center as viewed by many an executive to the engine that drives the business.  As I remarked to several people at ChefConf, we certainly live in interesting times!

    I was also able to use the Chef Newsroom to do a few interviews there. We have put them up in our DevOpsTV Youtube channel under ChefConf 2016 playlist. There are about 8 or so total but I am embedding two to give you a taste. You should check out the rest if you have a chance, as they are pretty good.

    The first is with George Miranda of Chef and we talk about Habitat and a little about Automate.

    This second interview is with well known DevOps journalist, writer and community member, Ben Kepes, enjoy!

  • DevOps Chat: Habitat by Chef with Julian Dunn

    DevOps Chat: Habitat by Chef with Julian Dunn

    julianI had a chance to speak with Julian Dunn, product manager for Chef ahead of the upcoming ChefConf conference in Austin Texas next week. Julian and I spoke about Habitat, Chef’s new open source application automation project.

    Many in the industry are hailing Habitat as something really special. Our own Tony Bradley says it could be a game changer.  Julian gives us some of the background and reasons why Habitat is important for you. He also gives us an advance look at the three Habitat related sessions scheduled for next week at ChefConf.

    If you are already going to ChefConf, be sure to hit all three of these if you can. If you are in the Austin area next week, try to make the show. We will be there covering it live.  Also the opening keynotes for Tuesday and Wednesday are being live streamed over the web.

    In the meantime here is my conversation with Julian with the printed transcript below, enjoy!

    Alan Shimel:                Hi, this is Alan Shimel, devopsy.kinsta.cloud, here for another DevOps Chat, and I’m happy to have Julian Dunn, Product Manager of Chef joining us today. Good morning, Julian, how are you?

    Julian Dunn:               Good morning, Alan, I’m doing very well. I’m melting a little bit because it’s very hot in New York, but you know, other than that—perfect summer day.

    Alan Shimel:                Yeah, it’s July in New York, it happens. Julian, we could talk about weather, I’ve certainly been traveling enough to see different ones, but let’s focus in a bit on what I wanted to really talk to you about today, and that is Chef’s new Habitat offering. Let’s assume a listener doesn’t know what Habitat is at this point. Give them a quick kind of elevator pitch on it.

    Julian Dunn:               Yeah, absolutely. Now, if you’ve read any of the materials and things like that, we’re using this tagline called Application Automation. And, you know, what on Earth does that actually mean? Well, what is the problem that Habitat solves for people?

    So, you’re an application developer today, and what you end up doing as an application developer is—well, what you want to do is, you want to spend a lot of your time doing the business logic and writing the business code that is actually gonna differentiate you from your competitors, right? So if you’re working at JCPenney, let’s say, you’re trying to write features for JCPenney’s website or mobile application or whatever to differentiate you from Macy’s and Target and Nieman Marcus or whoever your competitors are, right?

    Well, the experience of being a developer today actually ends up being that, before you write any of that program code, you have to spend a lot of time in the weeds and dealing with what I call sort of the plumbing oriented aspects of doing the programming, right?

    So just imagine if you were an interior decorator and you want to simply decorate your bedroom to whatever you wanted in there—you know, damask curtains and what have you—but you’ve spent most of your time dealing with the electrical and the plumbing. Not a particularly nice experience.

    Alan Shimel:                Mm-hmm.

    Julian Dunn:               The other thing is that, when you make those choices about what plumbing and what electrical systems you’re gonna use, that actually changes how your room is decorated. I mean, how absurd would that be, right?

    Alan Shimel:                Mm-hmm.

    Julian Dunn:               And so the notion of trying to let the application develop or be able to concentrate mostly on developing their program code and make a lot of those infrastructural choices later led us to develop Habitat, which is something that takes a little bit of a different approach than how we’ve built IT systems in the past, right?

    It’s kind of back to my analogy of the plumbing. In the past, we’ve built sort of up from the bottom towards the application, right? So we choose an operating system, we choose a bunch of stuff that sits on top of the operating system. You eventually at some point, we reach the application, right?

    And that’s an incredible waste of time for the application developer, who actually wants to start up here and define what their program logic is, right, and how that application should behave in a lot of different scenarios. And if you sort of start at the top and move towards the infrastructure, that’s the surface area that you actually end up owning as the application developer—we believe that that is Habitat.

    Alan Shimel:                Very cool. And hence the name, then, right? When you explain it that way, the name Habitat actually makes a heck of a lot of sense.

    Julian Dunn:               That’s right.

    Alan Shimel:                Yep. So, Julian, this is, some would say, you can see where Habitat is analogous—not analogous, adjacent to kind of the space that Chef plays in already. It’s a much bigger mission, if you will, than just what Chef has done to date. Is that a fair statement, you think?

    Julian Dunn:               It sure it, it sure is. You know, Chef, I think, initially started out, obviously, right, with a configuration management product that was very infrastructure oriented. And, over time, you’ve seen that our product portfolio and what we’re trying to do is a lot larger than just infrastructure configuration management, right?

    We kind of describe ourselves today as an automation company. We solve a bunch of automation problems, wherever automation is needed to help reduce people’s time to market or burdensome activities that they need to do in a regular ________, right? To really free people up, no matter what their problem domain is to being able to do the really intelligent parts of their job—the thinking parts of their job, right? And you’ll see that as we released Chef Delivery, which is a workflow system that allows people to basically implement, if you will, the principles that are in a lot of the books out there like Jez Humble’s Lean Enterprise that tell you, “This is the shape of how you should be delivering the software out to production.” Basically something like Chef Delivery is an automation arrayification or that technology—or those ideas, right?

    And then when we made the acquisition of VulcanoSec, which is the German security company, that gave us in-spec and Chef compliance that’s automating the work of compliance officers, security officers—right, folks that spend a lot of time, their job today and their pay and their experience is they spend a lot of time manually poking around systems and putting up fires manually because they don’t have a good framework and a system for describing the correctness of a particular system, right?

    And so now, as we move into—right, we’re, as a product organization, you want to be one that’s trying to address people’s pain points, right? When we started looking at what the pain was for an application developer, and they spend so much time dealing with the nuts and bolts of actually writing the code, we’re like, “How—why are systems built in this way?” We really went down a path of inquiry and we came up with Habitat as a way more application developer-centric than the perhaps infrastructure-centric things that we’ve built in the past.

    Alan Shimel:                Sure. Just two thoughts on that, Julian, I’d like to explore with you. One is—look, I’m not, I don’t hold myself out to be a Product Manager at Chef or anywhere near that integrated into it. But it feels to me, anyway, that the Chef compliance and the Chef delivery are very tactical pieces that embrace and extend Chef’s core configuration management capabilities, right?

    Where Habitat to me is more of a strategic rather than tactical type of play here, where you’re looking to set the game—the grounds, right? The stadium that we play in, whatever you wanna call it, the field, right? And so it’s a different—and again, I might be wrong, I’ll defer to you—but it feels different to me.

    Julian Dunn:               It is different in the sense that it’s a little bit iconoclastic, you’re right, in that it challenges some of the approaches that folks have been doing to try and deploy applications and get the sort of modern application characteristics that they want out there. And those characteristics are really great. I mean, one thing that’s really—as we developed this product, what really caught for us was to focus in on, “What does a modern application look like? What characteristics does it have?” And those characteristics are things like, it should be deployed immutably. It should be deployed atomically in that it’s either installed or it’s not, right? It should have certain characteristics built in. It should be aware of—it should know how to behave when it’s deployed in different scenarios, right, in different, what we call topologies.

    Alan Shimel:                Mm-hmm.

    Julian Dunn:               So you can think of, on a develop laptop, you might just have one instance. In a production environment, you probably have a number of instances, and they would have some relationship with one another, right? They might have one where you send all the rights two and the other ones are the reads, right? There might be some that are—like, think of a caching system, right? They right be hierarchical where there’s sort of a first, second, and third layer cache, like Squid or something like that, right?

    So it’s relationships between these entities, these application components, and one thing that we realized as we’re developing Habitat is, you know, as folks develop more complex applications and those applications are actually distributed systems, folks haven’t really spent a lot of time figuring out how to describe those relationships in a way that it imbues the applications themselves with the understanding about how they’re related to other components, right? The ways in which people have tried to solve the relationship problem between these components is to create sort of a central system, an orchestrator, if you will, that defines those relationships.

    And the trouble with an orchestrator is that it becomes a single point of failure, and as we know, that’s not really compatible with the cloud, right? Where in the cloud, they tell you, “You really shouldn’t have single points of failure, because you’re on an unreliable fabric,” right? So you really should have a system that knows how to recover and continue to function in the face of failure one or additional components, right?

    And so that’s where we started to come about with the architecture and the design of Habitat and how aware—how self-aware these individual components should be.

    Alan Shimel:                Absolutely. Just one other kind of comment or observation, and that is, you’re right—configuration management and compliance seem to be operational, or at least lead with operational, where Habitat clearly is a pay in the developer specter. But it’s all supposed to be one under DevOps, right? I mean, we don’t want to—I’m not into creating divisions between developers and operators, and I’m sure you guys aren’t, either, Julian.

    Julian Dunn:               Right, but I’ll tell you

    Alan Shimel:                But clearly they’re different focus—go ahead.

    Julian Dunn:               I was about to say, Alan, you know, the dirty of secret of DevOps, my observation, you know, I’ve been doing this for a number of years, and I’d love to hear your thoughts on this. But my feeling about the dirty secret of DevOps, and I go to DevOps Days and these conferences, and you survey the audience and you look at who the folks in the room are, right? And it’s something like, if you’re lucky, it’s still 80 percent Ops and 20 percent Dev, right?

    Alan Shimel:                It is, it’s OpsDev.

    Julian Dunn:               Yeah, right. So DevOps has been very much about Ops taking a lot of the good practices from Dev and learning a little bit of coding and learning that there’s a proper workflow and that you should have tests for stuff, but even if you’re not doing automated tests like Chef does, at least you should have some system for testing stuff before you just go log into a box and make changes in production and whatever.

    It’s almost like there’s this information flow that’s gone one way, but the information flow hasn’t really gone in the other way, even though we’ve had a number of iterations and generations of this technology. And so when we started asking the question, why—why have we kinda gotten from the brick wall to sort of more an English hedgerow, if you will, between Dev and Ops? How can we further develop capabilities to break down even the English hedgerow, if you will?

    Alan Shimel:                Makes a lot of sense. So, Julian, unfortunately, we’ve only got 12 or 14 minutes here and we’re running out of time. There’s a couple other things I wanted to hit. We’re recording this just the week before ChefCon, which is next week down in Austin, Texas—July 12th and 13th, or 11th, 12th, and 13th, technically. And, of course, I’m sure you’ll be there. Will there be a lot of Habitat kind of information and presentation?

    Julian Dunn:               There will. There’s gonna be three major talks, and we’re sort of billing them as the Habitat 101, 201, 301 kinda talks, right? We’re gonna have an intro and then sort of how Habitat fits into the ecosystem is sort of the 201 level talk, and then there’s a deep dive into why we made the technological choices that we did, the system design, and things like that, and that’s the 301 level talk.

    At ChefCon, we’re also gonna have something called the Habitat Zone, and we’ll program that with a few talks as well. The speakers from the 101, 201, 301, they’re gonna answer the Q&A in the Habitat zone, and that’s gonna be near the registration area. We’ll also have sort of community building—that’s one thing I didn’t mention, I don’t know, Al, if you mentioned it, but Habitat is a fully open source project.

    Alan Shimel:                Yeah, under an Apache 2.0 license.

    Julian Dunn:               Under the Apache 2.0 license, and we believed that that, we needed to make it open source to really engender a great community, similar to how created a huge community around Chef and, you know, just ways in which folks can contribute to that Habitat community is by writing plans that can build packages that live within the Habitat ecosystem.

    There’s, you know—just think of how many packages are out there within a typical kinda Linux distribution, right? There’s a lot of surface area that we would want to cover with people creating Habitat plans. That’s a great way for folks to get started. We’ll have some programming in the Habitat Lounge to talk about that with folks.

    Alan Shimel:                Yep. Julian, I’m gonna do my best to at least get the audio of this up before, in the next day or so if I can’t get it today.

    Julian Dunn:               Great.

    Alan Shimel:                And anyone hearing this, if you are in Austin or near Austin and you’re not already going to ChefCon, you really should look into it. You can get information, I guess, at getchef.io. I’m sure it’ll be listed there, just Google ChefCon.

    Julian, unfortunately, we’re just about out of time, but you know what, maybe we’ll pick this up down in Austin and we’ll do a part two or something.

    Julian Dunn:               Great.

    Alan Shimel:                Absolutely. I want to thank you for joining us today on DevOps Chat, and great—you know, I’d love to see Habitat really take off, here, and continued success to Chef and all the gang there.

    Julian Dunn:               Absolutely. Thanks very much, Alan.

    Alan Shimel:                Okay. Julian Dunn, Product Manager of Chef, thanks for appearing on this episode of DevOps Chat. We’ll see everyone next time. This is Alan Shimel for devopsy.kinsta.cloud.

  • Chef Has Potential Game-Changer with Habitat

    Chef Has Potential Game-Changer with Habitat

    Chef is an established leader in the DevOps world. Automation is a keystone of DevOps tools and processes, and Chef’s platform simplifies and streamlines automation. Chef is branching out, though, with the recent unveiling of Habitat—an open-source project to automate the applications themselves.

    As companies continue to embrace DevOps and developers continue to create custom applications, it is becoming increasingly difficult to manage it all. Many organizations have infrastructures that span on-premises and cloud—some using multiple cloud platforms—and the applications have to be crafted for the specific infrastructure or platform they’re designed to run on. Habitat seeks to change that. It ostensibly allows developers to focus on the features and capabilities of the app rather than the unique constraints of a specific infrastructure or particular runtime environment.

    “We must free the application from its dependency on infrastructure to truly achieve the promise of DevOps,” said Adam Jacob, co-founder and CTO of Chef. “There is so much open-source software to be written in the world and we’re very excited to release Habitat into the wild. We believe application-centric automation can give modern development teams what they really want—to build new apps, not muck around in the plumbing.”

    A blog post from Chef describes the premise of Habitat:

    Applications packaged with Habitat have the intelligence to self-organize and self-configure. Habitat allows the application to both be portable across independent infrastructure environments and have the intelligence to select the infrastructure features that benefit that application. This makes it easy to run applications across increasingly diverse environments such as containers, PaaS, cloud infrastructure, and on-premise data centers, while also accelerating hybrid cloud environments.

    It sounds promising to Michael Glenney, solutions architect at New Context. Glenney said Habitat—if it lives up to what Chef has promised—may represent a paradigm shift in how applications are managed in a data center where configuration management (CM) is prevalent.

    “Traditionally, you would use CM (Chef, Puppet, Salt, etc.) to not only lay down the bits an application needs to run, but to manage, redeploy, reconfigure and upgrade that application as well,” explained Glenney. “This type of work is not particularly well-suited for a CM and it mixes workloads unnecessarily. For example, an application deployment could fail because you have some data incorrect concerning log rotation on the host system. This is a manufactured relationship that needs to end.”

    Glenney noted that with features such as application-to-container conversion, universal package management, audit and traceability of both buildtime and runtime application dependencies, and advanced management and deployment functionality, Habitat is an exciting addition to the world of DevOps and lean security.

    Habitat combines the automation, configuration and management of an application into a single package. “When used in combination with CM it will deliver an unprecedented level of control over applications in all of your company’s environments and provide what I believe has been missing since the beginning,” Glenney said.

    Habitat is an open source project under the Apache 2.0 license and is available for immediate download. Check it out and let us know what you think of it.

  • Chef Launches Habitat, New Open Source Project to Automate Applications

    After Nine Months in Stealth Development, First of Its Kind Open Source Project Packages Automation with Application for Deployment Anywhere, from Bare Metal to Containers

    SEATTLE – June 14, 2016 – Chef<https://www.chef.io>, the leader in automation for DevOps<https://www.chef.io/solutions/devops>, today announced Habitat<https://habitat.sh>, an open source project that introduces a new approach for application automation. Applications packaged with Habitat have the intelligence to self-organize and self-configure. Habitat makes applications independent of underlying infrastructure, so it is easy to run applications across increasingly diverse environments such as containers, PaaS, cloud infrastructure and on-premise data centers.

    Companies today have extensive digital presence with many custom applications. As these applications have grown in complexity, so has the burden of managing them. Traditionally, infrastructure has dictated the design of the application. Habitat takes the opposite view and, for the first time, puts the application front and center. As a result, companies can focus on business value and planning features that will make their products stand out rather than on the constraints of infrastructure and particular runtime environments.

    Whether greenfield or legacy, any application that is deployed using a Habitat package has the intelligence to be aware of and react to its environment. Habitat’s novel approach to packaging allows the application to be independent of any particular infrastructure environment. Habitat also provides a well-defined interface that simplifies common tasks such as monitoring, safe deployment of new features and the creation of peer relationships that are required for production systems.

    From a developer’s perspective, by wrapping applications in Habitat packages, application teams don’t have to worry about a particular runtime until they’re ready to deploy. Because Habitat flexibly handles the management of the application, teams are no longer responsible for creating their own management solutions over and over again. With Habitat, developers can focus on new products and features that move the business forward.

    Chef’s co-founder and CTO Adam Jacob will unveil Habitat via livestream<https://bignews.chef.io/> at 8:30 a.m. PT today. The presentation will include a live demo and be followed by live Q&A on Twitter. Tweet questions to @chef using the tag #habitat.

    News Highlights:

    Habitat is a first of its kind open source project that offers an entirely new approach to application management. Habitat makes the application and its automation the unit of deployment. When applications are wrapped in a lightweight “habitat,” the runtime environment, whether it is a container, bare metal, or PaaS, is no longer the focus and does not constrain the application. Features of Habitat include:

    •   Support for the Modern Application: Habitat packages include everything that an application needs to run, throughout its lifecycle. Habitat’s core components are:

    •   Packaging format. Applications in a Habitat package are atomic, immutable and auditable.

    •   Supervisor. The Habitat supervisor runs application packages with awareness of the packages’ peer relationships, upgrade strategy and security policies. The Habitat supervisor configures and manages the application for whatever environment is present.
      The Habitat ecosystem also provides a build service that takes a Habitat build plan, creates the Habitat package, and publishes it to a depot.

    •   Run Any Application, Anywhere: With Habitat applications can run, unmodified, in any runtime environment, from bare metal and virtual machines, to containers like Docker and cluster-management systems like Mesosphere or Kubernetes, and PaaS systems like Pivotal Cloud Foundry.

    •   Easily Port Legacy Applications: When legacy applications are wrapped in a Habitat package, they become independent of the environment for which they were originally designed. They can quickly be moved to more modern environments such as the cloud and containers. Also, because Habitat packages have a standard, outward-facing interface, legacy applications become much easier to manage.
    •   Improve the Container Experience: Habitat reduces the complexity of managing containers in production environments. By automating application configuration within a container, Habitat addresses the challenges developers face when moving container-based applications from development environments into production.
    •   Integrate into Chef’s DevOps Workflow: The Habitat project is sponsored by Chef. Habitat leverages Chef’s deep experience with infrastructure automation to bring unprecedented automation capabilities to applications. Chef will offer commercial support for Habitat and ensure seamless integration between Habitat and Chef Delivery to fully automate the application release cycle, from development to deployment.

    Availability:
    Habitat is an open source project under the Apache 2.0 license. You can download Habitat immediately athttps://www.habitat.sh/docs/get-habitat/. Chef offers commercial support for the Habitat supervisor through its Chef Enterprise subscription.

    Supporting Quotes
    “We must free the application from its dependency on infrastructure to truly achieve the promise of DevOps. There is so much open source software to be written in the world and we’re very excited to release Habitat into the wild. We believe application-centric automation can give modern development teams what they really want — to build new apps, not muck around in the plumbing.”

    •   Adam Jacob, co-founder and CTO, Chef

    “Each year we’ve seen steady progress in taming the challenges of application management at scale,
    but I think we have to pay particular attention to technologies that continuously deliver desired end-states,
    and across diverse production environments, otherwise we’re just speeding up over-the-wall procedures.”

    •   Mark Burgess, Technologist, author, creator of CFEngine, Emeritus Professor of Network and System Administration, Oslo University College

    “Open source projects like Habitat embrace the ability to help DevOps teams run their applications in containers everywhere. The community can take advantage of Habitat’s application-centric automation on CoreOS Linux, the lightweight OS designed for containers, and Tectonic, the premier infrastructure platform for running distributed applications using Kubernetes, across all their environments while benefiting from security, reliability and scalability.”

    •   Wei Dang, head of product, CoreOS

    “Traditional enterprise applications have limited proliferation into the hybrid cloud. By making it easy to build applications that run anywhere, the value of hybrid cloud platforms, containers, and their management can be fully realized. Habitat is an important step forward in enabling enterprise IT to fully benefit from the portability and efficiency of cloud computing.”

    •   Jonathan Donaldson, vice president of Software Defined Infrastructure, Data Center Group, Intel

    “Open source is at the core of the modern application principles powering today’s digital economy. Together, Habitat and DC/OS enable organizations to apply these principles — including autonomy, portability, and scale — to new and legacy applications alike, making it even easier to automate the application stack in any environment.”

    •            Tobi Knaup, co-founder and CTO, Mesosphere

    “Modern application teams want the freedom to make infrastructure choices without worrying about the impact of those choices on their applications. Habitat’s application automation combined with our container management platform gives development teams the ability to easily build, deploy, and manage their containerized applications on everything from bare metal to the cloud.”

    •            Sheng Liang, CEO, Rancher Labs

    Additional Resources

    About Chef
    Founded in 2008, Chef Software is changing the way companies build and deliver software to delight customers. Chef is the leader in automation for DevOps, a cultural and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners. Built on a strong open source foundation, we have distilled the proven patterns and practices for fast and scalable software development into our leading IT automation platform. Chef automates the continuous and unified delivery of applications and infrastructure, enabling enterprises worldwide to deliver software much faster with minimal risk. With hundreds of commercial customers and tens of thousands of open source community members, we are leading one of the strongest technology movements in the world today. Join us atwww.chef.io<http://www.chef.io>.

    Shelby Lichliter
    Revere | Shelby.Lichliter@revereagency.co<mailto:Shelby.Lichliter@revereagency.co> | 925.216.5303

    Attachments area
  • DevOps Chat: Nathen Harvey, Chef

    DevOps Chat: Nathen Harvey, Chef

    I recently had the pleasure of catching up with Nathen Harvey of Chef. And I do mean catch up. As VP of community development, Nathen keeps up an amazing pace of visiting Chef communities around the world. In this role who better to understand what DevOps practitioners are seeing? Nathen and I spoke about several things including Chef’s security and compliance capabilities.  Below is the streaming audio of our conversation and below that is a transcript of our discussion to follow along.

    Enjoy!

    Alan Shimel:                Hi. This is Alan Shimel for another DevOps Chat here on devopsy.kinsta.cloud. Today’s guest is Nathen Harvey, VP of Community Development at Chef. Hi, Nathen. How are you?

    Nathen Harvey:           Great, Alan. How are you doing today?

    Alan Shimel:                Good. Nathen, I’m going to bet that most of our audience already knows who Nathen Harvey is and what your role is at Chef, but maybe for the few who are unfamiliar, why don’t you give our audience a little bit about what it is you do as a VP of Community Development at Chef?

    Nathen Harvey:           Yeah. I get asked that question a lot, Alan, and I’ll tell you that the real definition is this. There are two parts to it. The first being VP of Community Development means that none of my code runs in production and that makes the world a much safer place, if we’re being honest.

    Alan Shimel:                Okay.

    Nathen Harvey:           The second thing is before I joined Chef I did actually have code that ran in production. I actually managed a bunch of production infrastructure. So when I moved from an operations role into a community management role, the thing I like to say about that is I put down the pager and I picked up the bar tab. Oddly though, I’m up at 3:00 AM either way.

    Alan Shimel:                Yeah. Well, I think that’s just part of today’s work environment, where everyone is up it seems at 3:00 AM, depending where you are in the world.

    Nathen Harvey:           But joking aside, it’s really about going out and meeting with our community and helping fuel the love of Chef, helping folks in companies large and small get better at automation, get better at dev-ops, get better at great software development and infrastructure practices.

    Alan Shimel:                Yeah. And, Nathen, you’re not kidding about seeing people the world over, too, as part of your role, because I follow you on Twitter, I follow where Nathen is speaking, and whether it be a local meet-up group or a Chef users’ group or a conference or customers, you literally are traveling the world constantly.

    Nathen Harvey:           Yes indeed.

    Alan Shimel:                Yes, preaching the Chef message. Nathen, I wanted to focus our talk a little bit today around one of my favorite subjects, security, infosec, whatever you want to call it. Chef has become one of the leaders and has been for some time – one of the leaders in security within the dev-ops movement, if you will.

    I had the pleasure of becoming friends with Justin Arbuckle a year, year and a half ago. Justin was out preaching – what was it – compliance at Velocity or something? I may be mistaken.

    Nathen Harvey:           That’s right, compliance at Velocity. You bet.

    Alan Shimel:                You know, for a long time and it resonated with me. Then of course I think Chef made an acquisition, a European company who sort of had a compliance vulnerability scanning for code.

    Nathen Harvey:           Yes.

    Alan Shimel:                And now you guys have really kind of laid down the gauntlet on this and really put the pedal to the metal in talking about why it’s important to do this and why it’s important to do it earlier. I don’t want to steal your thunder. Can you give our audience a little bit of your opinion on why it’s important and what should be done?

    Nathen Harvey:           Yeah, sure. Look, Alan, in a traditional way of developing our software we all fall under some sort of regulatory burdens. We all have auditors or other security controls that we need to make sure are implemented within our production systems, and the traditional way of doing that, at one point in your development workflow you might introduce a security scan into the development workflow itself before you go to production, but usually that ends up just being a blocker for moving flow, moving change across your organization.

    So you’ll oftentimes pull that security scan out and then just start scanning your production systems. But for me, scanning the production system is kind of like the oil light on your dashboard. When it comes on, it’s too late. You have to have solved that before you get into the production environment.

    So at Chef, really we look at our roots with automation and our ability to fully customize and fully automate the buildup and the maintenance, configuration management of your infrastructure over time, your infrastructure and your applications. We say that we need to make sure that we’re always in compliance, but it’s also very important for us that we start to adopt more of the development workflow, more of the software engineering principles.

    One of those principles of course is that we should do test-driven development. Well, why not take those controls and those audit controls that you have to enforce and move those directly into the development workflow? You certainly see many cases where you’re doing, say, static code analysis during your continuous delivery pipeline or during your continuous integration. But what about moving that even further back and doing not only static analysis of your code, but also actually testing what does the running application actually look like? Is it exposing ports that it shouldn’t? Is your infrastructure configured properly? And can you do that on the developers’ workstations, move that -? We like to say you’re moving compliance to the left. If you think about your change flows from left to right, from your developer’s workstation all the way out to production, how do we get and start thinking not about how to scan the production environment better, but instead how to start making those controls into your developer’s workflow?

    Alan Shimel:                Yep. Nathen, as a security person that’s almost radical to me. I will tell you, having spoken to many security people about this, we actually put on a DevSecOps day, a Rugged DevOps day at the RSA conference last year in San Francisco and Justin spoke there. A lot of security people, this makes them very nervous, believe it or not, Nathen, because it’s not them doing the scan. We’re going to ask developers to scan their code on their workstation, while I’m not there to see it, right, or not there to make sure that the code is right?

    Nathen Harvey:           Right, right.

    Alan Shimel:                Go ahead. I’m sorry.

    Nathen Harvey:           Let’s look at that. In the past, and I understand where these security folks are coming from, but if we look at today’s enterprise and the way that software is built today, security folks want to be the ones that are doing all of the scans, that want to be the ones that are making all of those assessments. On the one hand that’s fine. They should absolutely be involved in those assessments. But, frankly, if it’s the security team only that’s responsible for that, it’s simply not going to get done. The security teams aren’t big enough; don’t have enough capacity to keep up with the rate of change. They’re going to become a blocker in the enterprise and that’s going to become a real problem.

    So it’s really that security professional’s goal or the way that they need to look at this is, first, automation makes their job much, much easier. Not only can we remediate things that are found much, much faster, but we also gain consistency and things like that.

    But if we can then partner with the development team, with the engineers that are building out those systems and get them to think of security as a first level concern and compliance as a first level concern and start being partners with us, like we’re partnering with them, we’re not abdicating our responsibility to them. Frankly, some of the scans that the security team is going to run in the production environment, they’re still going to run those, but what we’re going to find by giving developers better tools to run those scans sooner, to ensure compliance earlier in the process, that those scans are going to find fewer and fewer vulnerabilities, and those vulnerabilities that are found we’re going to be able to address and remediate much, much faster.

    Alan Shimel:                And I might add, Nathen, not only much, much faster, but much, much cheaper. I mean I’ve seen studies where it’s something like 10x to 100x cheaper to take care of a vulnerability or a compliance issue at that shifted left stage, where it’s still on the developer’s workstation rather than on some cloud instance. Right?

    Nathen Harvey:           Absolutely. That may feel radical, but it’s really just we’re going back to our roots. I mean we’ve known for decades that fixing a bug in the development process is much less expensive than fixing it after that bug has been released into your production environment. The same is true for a security control or a compliance control.

    Alan Shimel:                I agree. I agree with you. Nathen, I have another issue I wanted to touch on with you, and that is some people get hung up on the compliance, the C word, so compliance versus security. They’re not synonymous, right?

    Nathen Harvey:           They are not. They are not at all synonymous, no. I think that compliance – Alan, let’s think about this. You have a CEO at a company who has to sign off and agree to – have auditors come in and assess whether or not they are compliant. Frankly, when an auditor walks in the building, there are many, many organizations who say, “Our number one objective when the compliance person walks in, when the auditor walks in is to get them to sign off on our audit and leave as quickly as possible. I don’t want them meddling in my business. I want to get them out of here.”

    Alan Shimel:                Yep.

    Nathen Harvey:           So with that approach, what we really do is try to make sure that we can run compliant systems, and compliance is there so that we can assess whether or not we are secure, but security really needs to – it’s a different mindset than compliance. Compliance sometimes is the easier place to start. I want to make sure that my audits are passing. When I have audits that are passing, I can sleep at night knowing that my audits are going to pass. Then I can start to look broader at the system in terms of: am I actually building a secure system? But, frankly, security is the foundation and compliance is what sits on top of that.

    Alan Shimel:                Yes, absolutely. Nathen, the other thing I think that really has enabled – and maybe it’s a little bit of a chicken or an egg kind of issue – does this enable them or they have enabled the compliance issue? Developers are taking to heart the fact that security is everyone’s responsibility, and the fact of the matter is it starts with them. Right?

    Nathen Harvey:           Absolutely. Alan, we’ve seen similar things in the past. Look at QA teams, where developers have said, “You know what? We’re going to start doing test-driven development. We’re going to adopt some of those practices.” And the QA teams get nervous, “That’s my job.” The problem with having it be someone else’s job is you’re not going to do it. You’re not going to be concerned with it.

    So, really, just like our QA professionals are now partnering with the devs, the security professionals and compliance officers are now partnering with the development teams. Frankly, that’s what dev-ops is all about. It’s about this working together towards those common goals and getting there as quickly as we can.

    Alan Shimel:                Yeah, absolutely. And it’s good. It’s good on so many levels to have the security team integrated with the devs, to be working closely with both dev and ops, dev-ops, and making it happen because, frankly, that’s what attracted me to dev-ops to begin with, was the promise of being able to do that instead of being the appendage at the end of the Rube Goldberg kind of –

    Nathen Harvey:           Absolutely. Alan, what you’re really looking at there is we’re talking about a partnership. So there are some cultural or some workflow changes, some partnering that we have to do across the organization. The thing that helps enable that also is the tooling. The tooling definitely matters and the two, culture and tooling, are there to reinforce one another.

    So with Chef, we have a new open source language called InSpec and that’s the tooling that allows the developers to actually write these compliance controls, execute them locally. Frankly, it allows them to write them in a way that’s delightful, in a way that a developer understands, that a business analyst can look at those tasks and understand exactly what’s happening.

    Then with Chef compliance we also introduce those same compliance controls into our own scanner capability, where you can scan against your production environment, find vulnerabilities, go back into your automation code, remediate those, and watch that remediation happen across the entirety of your pipeline.

    Alan Shimel:                Fantastic. Nathen, as I had told you earlier, before we started recording, our time is limited, but for folks who maybe want to find out more about compliance scanning, more about Chef’s capabilities in this area, where can they go to get more?

    Nathen Harvey:           For sure the place that they can go is www.chef.io/compliance. Again, I just want to stress that the scanning capabilities are absolutely important, but really what we need to do is shift our mindset and think about: how do we start writing tests for those controls and do those early on in the process? Not that scanning should be an afterthought, but we really do need to focus on that beginning stages of our development process.

    Alan Shimel:                And automation, automating that is key. Nathen, we’re almost out of time here. Unfortunately, it always goes too quickly. But I wanted to ask you one last question, which we usually ask our guests here on DevOps Chat. That is one book, if you had to recommend one book to our listeners that they should read maybe over the summer or in the near future, what book would that be?

    Nathen Harvey:           I would recommend Lean Enterprise: How High Performance Organizations Innovate at Scale. It’s a book by Jez Humble, Joanne Molesky, and Barry O’Reilly.

    Alan Shimel:                It’s actually on my bookshelf. Cool. I love it when someone suggests a book I’ve read actually. Very cool, Nathen, and it is a great book. Jez’ Lean Enterprise book is great as of course is the Continuous Delivery book.

    Anyway, Nathen, we’re about out of time here. So I wanted to thank you for being our guest on DevOps Chat. Continued success with your globetrotting VP of Community Development role at Chef, and continued success to Chef in general there. It’s one of our favorite companies in dev-ops and we love what they’re doing.

    Nathen Harvey:           All right.

    Alan Shimel:                And maybe we’ll get you back on again soon.

    Nathen Harvey:           Yeah, that sounds great. Thanks so much, Alan.

    Alan Shimel:    Okay. This is Alan Shimel with Nathen Harvey, VP Community Development of Chef for DevOps Chat. Thanks and we’ll see you next time.

  • DevOps and ITaaS: Two Sides of the Same Coin

    DevOps and ITaaS: Two Sides of the Same Coin

    It seems IT practitioners haven’t realized yet the scale of digital disruption. Mired in the old IT thinking, they firmly believe that sophisticated tools are all businesses need to survive the ongoing digital disruption waves. Most of them warn, “Digital transformation is changing the way we do business, the way we deliver IT services must change, businesses need to adapt the way they work.” The problem is, there’s nothing new in what they offer to help businesses work the “right” way.

    Indeed, the same old IT thinking’s practices are still applied with the illusion that deployment automation and configuration management tools on their own will make businesses agile, responsive to market opportunities, and therefore in line with the digital economy’s competitive challenges. That belief misses the point for two reasons: First, it’s an incorrect understanding of what business agility is, and second, it’s a misunderstanding of the global competitive environment’s transformation.

    Turning your IT organization lean and agile is the competitive differentiator your business lines desperately have been calling for. Authentic DevOps implementation is the way to go. That’s the price to pay to keep up with the pace imposed by new leading-edge companies such as Google, Apple, Uber and the like.

    The changing competitive environment forces businesses to focus on innovation, market responsiveness and timely revenue

    A digital economy threatens the existence of all businesses. Yes, you read it right, their existence is at stake. And that fact is underestimated by IT vendors and consulting firms.

    As illustrated, we’re shifting from business environments where established businesses (incumbents) control a large percentage of market segments to competitive contexts where, attracted by the amazing digital opportunities and the adoption of cloud which significantly reduces infrastructure expenses, a growing number of startups disrupt entire industries:

    The changing competitive environment (Credit: Philippe Abdoulaye ITaaS Now LLC)
    The changing competitive environment (Credit: Philippe Abdoulaye ITaaS Now LLC)

    The proliferation of startups created by highly qualified unemployed seniors and creative entrepreneurs, coupled with the propagation of highly innovative digital services (e.g., Internet of Things [IoT] applications) substituting traditional services, have been disrupting the vast majority of industries.

    Several examples substantiate that fact: the taxi industry shaken by the emergence of technology-driven companies such as Uber, the banking industry facing new breed of competitors including PayPal, Google and Apple; and the healthcare industry increasingly disrupted by companies such as GlowCap.

    To grow and prosper in the digital era, your scale, brand equity or market capitalization aren’t enough anymore. To succeed, your business lines must be on top of the digital economy’s competitive standards: competitive innovation (the ability to continuously improve customer experience), market responsiveness (the ability to deliver added-value services to the right market segments at the right moment) and timely revenue (the ability to continuously increase customer value).

    The digital economy has taken IT to an inflection point—deploying sophisticated tools isn’t enough; being lean and agile is the way to go

    The obligation for business lines to focus on innovation, market responsiveness and timely revenue is unprecedented for techies; it forces them to think beyond tools and address issues as unusual as organizational and operational effectiveness.
    Indeed, innovation, market responsiveness and timely revenue assume greater cross-functional collaboration, IT focus on business priorities, organizational flexibility and accelerated time to market. Make no mistake, you’ll never achieve them by only deploying tools, but by also making your IT operating model flexible through proactively eliminating the organizational and operational dysfunctions that prevent cross-functional collaboration, fast problem-solving and decision-making.

    That’s what is meant by making the business agile.

    The new IT thinking is summarized in that simple  arithmetic: Digital Competitiveness = Lean and Agile Operating Model + Effective and Efficient Tools. Without the lean and agile operating model, innovation and timely revenue are wishful thinking. Without effective and efficient tools, innovation and market responsiveness are wishful thinking as well.

    The organizational and technological dimensions are complementary. Adam Jacob, Chef’s CTO, confirms that fact in a very informative article, “The Secret of DevOps: It’s Always Been About People, Not Technology.” This is where DevOps can help businesses succeed.

    DevOps is the IT asset businesses need to succeed in the digital economy

    Contrary to a now (unfortunately) established belief, DevOps isn’t a technology; Chef, Puppet, Jenkins, OpsWorks, alike aren’t DevOps, they’re only pieces of it. It’s important to be clear about it.

    As I’ve been implementing it, DevOps is a system of principles, practices, roles and responsibilities, governance structure and tools purposely arranged to establish cross-functional collaboration, enable accelerated applications delivery and, in turn, make the business responsive to market opportunities:

    DevOps Model (Credit: Philippe Abdoulaye ITaaS Now LLC)
    DevOps Model (Credit: Philippe Abdoulaye ITaaS Now LLC)

    As illustrated, DevOps as a business capability is made up of three components: the Rules of the Game, the IT Operating Model and the Continuous Delivery Pipeline (CDP). Let’s explore them.

    DevOps Rules of the Game create the conditions for lean and agile IT environments

    Rules of the Game are a fundamental dimension of DevOps: They enable the establishment of lean and agile environments. I refer to them as the DevOps non-technological piece.

    In practical terms, they’re a set of principles, practices, roles and responsibilities and tools institutionalized to ensure accelerated applications delivery.

    They serve as catalysts to specific organizational and operational effective practices. Let’s take the principle, “Facilitate with system thinking (involvement of all relevant stakeholders)”—it forces application development stakeholders to adopt a collaborative mindset, which in turn breaks down organizational silos to eliminate the impediments to operational flexibility and institutionalize cross-functional collaboration.

    They’re the poor cousin of today’s DevOps practices, which ignore them in favor of more technical and technology issues. That’s a terrible mistake that reduces DevOps influence vs. the digital economy’s challenges.

    The agile IT Operating Model establishes the lean and agile work environment

    The IT Operating Model reflects the interactions that take place across your application development value stream. In practical terms it represents the work environment derived from the application of DevOps’ rules of the game.

    I always remind my clients that DevOps’ rules of the game aren’t theories, they’re the foundations of a tangible strategic capability: the lean and agile IT  environment. They demand a thorough transformation of your application development work space to put together business, application development, and operations:

    The lean and agile IT work space illustrated (Credit: O.C. Tanner)
    The lean and agile IT work space for the digital era illustrated (Credit: O.C. Tanner)

    The picture illustrates what’s meant by lean and agile IT environment, it substantiates Adam Jacob’s point that “Fundamentally DevOps is about taking the behaviors and beliefs that draw us together as people, combining them with a deep understanding of our customers’ needs, and using that knowledge to ship better products to our customers.”

    The IT operating model is the poor cousin of today’s DevOps practices which ignore it in favor of more technical and technology issues. Once again, it’s a terrible mistake that reduces DevOps influence vs. the digital economy’s challenges.

    Make no mistake, technology matters, the CDP helps to meet digital technologies challenges

    The importance of the rules of the game and the resulting IT operating model don’t suggest at all that technology doesn’t matter. Make no mistake about it, the CDP is essential; I refer to it as the DevOps technological piece. Through deployment automation and configuration management tools, it significantly accelerates application development, testing, resources provisioning and deployment.

    However, one thing to be aware of is, digital economy raises technological challenges that transcend speedy delivery issues. They include:

    • The importance of IoT
    • The pivotal role of predictive analytics
    • The increasing importance of APIs

    As a matter of fact, IoT devices generate massive data and making use of it to generate revenue requires big data technologies to integrate, automate and analyze them.

    On the other hand, this ocean of IoT data has generated an API business; the idea is, to monetize their data and generate revenue, businesses must provide their customers and partners with APIs that easily give access to their systems.

    Implementing the CDP as part of PaaS cloud services can help to prosper in the digital economy; it allows businesses to benefit the cheap and on-demand computing, storage and network resources provided by the cloud.

    Takeaway

    In hindsight, with a business-oriented perspective, the ITaaS delivery model and DevOps follow the same goal: Provide business lines with the lean and agile IT environments they need to be flexible enough and responsive to meet the competitive challenges of digital economy:

    The ITaaS Delivery Model (Credit: Philippe Abdoulaye ITaaS Now LLC)
    The ITaaS Delivery Model (Credit: Philippe Abdoulaye ITaaS Now LLC)

    Like DevOps, the ITaaS delivery model (…) takes advantage of an agile IT operating model to help businesses succeed in the digital era. It implements consumer-provider relationships across the IT value stream to ensure organizational flexibility and operational agility.

    CIOs must realize that what matters to the business is to continuously generate revenue through improving customer experience and increasing customer value. Deploying applications at the speed of light for the sake of speed doesn’t make sense from a business perspective; as long as expected business benefits aren’t achieved, speed is a useless gadget.

    Implement DevOps as what it is: a three-dimension IT capability including rules of the game, IT operating model and, of course, continuous delivery pipeline.

  • In DevOps, Standardize to Minimize Architectural Debt

    In DevOps, Standardize to Minimize Architectural Debt

    I recently reading a great Q&A with the Etsy CTO John Allspaw over on The New Stack, which contained some nuggets of good advice that increasingly are important as we continue the journey toward DevOps adoption. Of particular note were statements on the introduction of “new” things to solve problems. In a nutshell, Allspaw reinforced the importance of standardization as a means to reduce architectural debt.

    If you aren’t familiar with the concept of architectural debt, it’s similar to technical debt, in which choices made today incur longer term investments of time and money when future changes are introduced. If you settle on a caching solution such as Redis, for example, there are technical and architectural implications. You can’t simply switch out Redis for memcached later without significant changes to both code and the infrastructure.

    Even your choice of load balancer today implies architectural debt because of the growing inclusion of scalability as part of the application infrastructure itself. The result is integration with automation and orchestration systems that are not easily changed later on if you decide you want a different solution. The style of service registry you choose is an architectural decision that impacts clients, servers and systems across the microservices architecture. Switching from client-side to server-side will later be costly.

    That’s architectural debt.

    Allspaw noted the need to not reinvent the wheel when possible, and instead encourage the collaborative aspects of DevOps as a means to avoid reinventing the wheel and potentially increasing the architectural debt load on the organization.

    This is not a new concept. There’s a reason enterprises in particular proudly proclaim themselves “Java shops” or “Microsoft shops” and primarily require development in one or perhaps two languages, but no more. Maintainability is a component of technical debt, just as middleware decisions are a component of architectural debt.frameworks in use soad2016

    So as we continue to see wider DevOps adoption, it’s important that we carefully consider the impact of our choices in terms of the automation, orchestration and integration required to implement continuous delivery and deployment. Once you’ve invested time and energy (and probably budget, too) in Puppet or Chef, VMware or Cisco, OpenStack or SaltStack, you’re going to find an unavoidable amount of technical and architectural debt in development as well as production across infrastructure and networking teams.

    Each of these systems, solutions and frameworks comes with its own components. It isn’t just a language, it’s a comprehensive system with management and repositories and agents and reporting components. All of these impose certain architectural requirements on organizations, from needing certain software running to prescriptive details on networking that must be enforced, lest the system break down.

    We’re already starting to see that fragmentation in enterprises that are engaged in automation and orchestration efforts. When asked the number of automation frameworks (Cisco, VMware, Python, OpenStack, Chef, Puppet, etc…) in use, one-third today reported using three or more.

    That’s not necessarily “bad,” as it could be indicative of standardization across silos (networking wants to use Cisco, infrastructure is already on VMware, and Python is being used to cobble it together by ops), but it also could be an early sign of the kind of fragmentation that causes exponential growth in technical and architectural debt.

    Certainly there is room for experimentation in these early days, but it behooves us to maintain a watchful eye on the use of frameworks and systems and languages we employ for automation and orchestration to ensure we don’t end up buried in technical and architectural debt.

    Standardization, even loosely enforced, can help minimize the impact of that debt before it spirals out of control.

  • Chef Makes DevOps Workflow Faster and Safer for Enterprises Amid Rapid Business Growth

    New Chef Delivery Product Enhancements Come as Company’s Annual Recurring Revenue Grows 94 Percent Year-Over-Year in 2015 and Chef Nears 1,000 Customers

    SEATTLE, WA–(Marketwired – Mar 10, 2016) – Chef, the leader in automation for DevOps, today introduced Dependency Management, a new feature of its Chef Delivery product, that enables faster and safer developer collaboration by allowing Chef Delivery to know which services and applications in the pipeline depend on one another. With Dependency Management, not only must changes pass local tests, but related, downstream components or services must pass those tests, as well. Only service changes that pass are promoted, so deployments to production are safe. Dependency Management enables a reliable, repeatable and scalable process for cross-team collaboration in large enterprises. It doesn’t matter if there are two developers or thousands of staff — teams can create and safely deploy interconnected services on diverse runtime environments, including containers.

    Chef also today announced enhanced capabilities for Chef Compliance to enable automated management of compliance policies that are based on the broadly used Center for Internet Security (CIS) benchmarks. Users can now implement CIS and other compliance policies as code, called “compliance profiles.” In addition, Chef Compliance offers an enhanced Windows experience by allowing users to import Windows policies from the Microsoft Security Compliance Manager.

    By using compliance profiles in conjunction with Dependency Management, users can enforce these policies with Chef Delivery to ensure that changes to infrastructure and applications meet all requirements across the pipeline. Together these two new features enable users to easily input, track, and test service runtime dependencies. Simultaneously, users can incorporate CIS benchmarks profile into Delivery’s automated testing pipeline to create a foundation for meeting Payment Card Industry (PCI) standards.

    Software is now the competitive currency of the enterprise. However, software is also becoming increasingly complex and composed of interdependent services that reside in the cloud. Businesses must be able to develop and maintain these intricate applications, along with the infrastructure that supports them, while moving at high velocity. It is critical that any changes to applications and infrastructure be versioned and tested before they are deployed to production. Because multiple teams often work on a single application, cross-team collaboration and transparency are critical to the success of a project. The DevOps workflow, which uses both culture and technology to foster these values, is becoming the most effective approach to developing software quickly and safely. Chef has distilled its customers’ successful workflow patterns into Chef Delivery, where service dependencies can be predefined and compliance requirements become part of the development pipeline. Chef Delivery eliminates silos and project segregation, enabling developers and operations personnel to begin committing code to the primary development pipeline in their first week on the team, instead of waiting weeks or even months in traditional environments.

    “Enterprises need comprehensive solutions to safely implement DevOps in the push toward ever-greater development velocity. Unlike other frameworks and tools, Chef Delivery doesn’t just help you build a change management pipeline, it is the pipeline,” said Alex Ethier, vice president of product, Chef. “We’re committed to advancing our product in a way that enables developers, operations and IT to collectively manage the speed and scale of enterprise business.”

    Company, Customer and Community Growth
    The new product enhancements reflect Chef’s rapidly growing footprint in the enterprise. Today, more than 80 percent of the company’s revenue comes from enterprise organizations, and more than half of the Fortune 50 use Chef. Chef has tens of millions of machines under management on any given day. To date, the Chef client has been downloaded more than 27 million times, with more than 13 million client downloads in 2015 alone. Among the increased demand from the enterprise, Chef’s annual recurring revenue (ARR) grew 94 percent year-over-year in 2015.

    Regionally, Chef’s EMEA business continues to rapidly expand. Q4 2015 was a record quarter for Chef EMEA, accounting for nearly 20 percent of the company’s total ARR for the quarter. Chef’s EMEA business now serves more than 150 customers and ARR for 2015 grew 91 percent year-over-year.

    Chef is closing in on 1,000 customers worldwide, with Chef’s commercial offerings used by organizations including Nordstrom, Facebook, Disney, Target, Standard Bank, and Etsy.

    Critical to Chef’s success is its open source community, which has more than doubled in size – at a clip of more than 100 percent year-over-year for three consecutive years – to nearly 70,000 registered members and thousands of active constituents contributing code daily. As adoption of open source technologies rise and more developers adopt Chef skills, the value of those abilities is reflected in enterprise careers. Dice recently noted that developers who cultivate Chef skills see return in a more than $130,000 annual salary.

    Recently, Chef announced a new Partner Cookbook Program to allow even greater collaboration and integration for its open source community. Some of our partners include 3Scale, Alert Logic, CloudPassage, Dynatrace, Infoblox, and NetApp.

    Availability
    Chef Delivery’s new Dependency Management capability is now available to all current Delivery customers. Read more on accessing and utilizing this new feature. Register for a webinar demonstrating Chef Delivery with Dependency Management on April 7, at 10 am PT.

    The CIS and Microsoft Security Compliance Manager profiles for Chef Compliance will be available on March 31 and demonstrated via a webinar that day at 10 am PT. Register here for this webinar and for more information on the profiles’ availability.

    Additional Resources

    About Chef
    Founded in 2008, Chef Software is changing the way companies build and deliver software to delight customers. Chef is the leader in automation for DevOps, empowering customers to build and operate high velocity organizations. Built on a strong open source foundation, we have distilled the proven patterns and practices for fast and scalable software development into our leading IT automation platform. Chef automates the continuous and unified delivery of applications and infrastructure, enabling enterprises worldwide to deliver software much faster with minimal risk. With hundreds of commercial customers and tens of thousands of open source community members, we are leading one of the strongest technology movements today. Our company is based in Seattle, WA, and financed by DFJ Growth, DFJ Ventures, Battery Ventures, Citi Ventures, Ignition, and ScaleVP. Learn more about Chef at www.chef.io.

    CONTACT INFORMATION

  • DevOps in 2016: Mainstream Momentum Ahead

    DevOps in 2016: Mainstream Momentum Ahead

    I believe that 2016 will be the year that DevOps takes mainstream foothold in the enterprise. And I base this assertion on seven sweeping changes that — from my vantage point — are taking hold and dramatically changing the digital world.

    1. The Job of IT Will Change From Infrastructure to Innovation. In the past, IT has been about providing compute infrastructure and applications to run the internal workings of a company. But this has been radically shifting as digital transformation penetrates nearly every company of significance across the globe. As a result, IT is increasingly about driving innovation and customer value through software. We see this every day from our work with large enterprises; indeed, almost half of the global 500 use Chef in some manner. And, in our interactions with enterprise customers, over 75 percent cite innovation, velocity and agility as top priorities for IT — and the company. The common ingredients behind this are almost always automation, cloud and DevOps. In fact, while these three elements are, in many ways, distinct, they form a magic IT trinity in service of innovation-led organizations. If you need further proof that IT is on its way to becoming the pointy end of the innovation-creation spear inside large-cap companies and organizations, just look at the auto industry, where CES may actually be more important than the Detroit Auto Show, where GM is investing in Lyft, and where Google is doing state-of-the-art R&D on a self-driving car.
    1. Continuous Delivery (CD) Will Become Ubiquitous. CD is the practice of rapid software development and iteration through fast, small-batch releases. In its purest form, developers can drive small, incremental software releases quickly, sometimes as often as every few minutes for some very well known Web sites. Not long ago, CD was exclusively the domain of big Web innovators such as Facebook, Amazon, Google and Yahoo! Not long ago, making even several releases a day sounded insane. And not long ago, a CIO told me that it was ridiculous to contemplate bringing CD into the enterprise. All this has changed, and today it makes absolutely no business sense to shy away from CD because it’s one of the keys for unleashing an organization’s software development innovation engine. The point is that the whole stack — infrastructure, compliance, security, microservices and applications — is being expressed as code now, and  it’s the same as you deliver with work-flow. In the end, whether your company needs to release frequently all the time, or not, every aspect of IT is charged with a business requirement to do something quickly some of the time. CD is a very important piece of the puzzle.
    1. The Public Cloud Will Triumph in the Enterprise. You may think the cloud is old news, but it’s not. In 2016, the enterprise is going to move beyond public cloud experimentation and project-based work and start migrating entire data centers to the public cloud. We’re also going to see more hybrid cloud initiatives (whatever the composition) because dual adoption helps companies avoid vendor lock-in. As for private cloud projects, there are definitely some good teams and technologies out there. But, having said this, I believe that the public cloud will win out. Many of the lingering public cloud issues, like security and governance, have begun to evaporate. And the ease of use, affordability and effectiveness of the public cloud will simply triumph in the enterprise over the next 12 months at the expense of the private cloud.
    1.  We Will Enter a New Era of System Administration. The world of systems administration has changed forever. In the IT world that I grew up in, a human being logged onto a system to manage, update or fix it manually. Since then, compute scale has grown exponentially for most organizations, thanks, in part, to both server virtualization as well as the cloud. Most of our enterprise customers have, excluding desktops, tens of thousands of compute nodes. Some have over 100,000. A few have over 1,000,000. Administering systems manually by having humans act on systems is no longer viable or tenable. In the new era, humans act on code and code acts on machines. In fact, code is the system. In other words, humans act on compute resources through code. And this reality creates a whole new universe for the world of system administration. The bottom line? The job and role of the system administrator will change significantly. Watch this trend play out in 2016.
    1.  Peace Will Break Out Between Developers and Security / Compliance Officers. This has been a long-running battle inside the enterprise, and there are several theories behind the conflict. One is miscommunication — the  security team doesn’t think the dev team gets it, and the dev team doesn’t think the security team gets it. Another hypothesis is that developers just want to be left alone to write code for the business, but the business is increasingly worried about security — not just IT security. And, lastly, there’s this explanation for the hostility — developers want to go fast, and security teams want to slow everything down to a safe speed. Whatever the reasons, I believe that developers and security / compliance officers will find much more common ground in 2016 because we’re learning that compliance and velocity aren’t necessarily in conflict with each other.  In fact, high-velocity IT organizations can deal with security threats more quickly by assessing vulnerability, creating a fix, testing it and then deploying it rapidly before the damage is done. We’re also seeing the early beginnings of what I believe will be a very important trend in our industry: compliance defined as code. Today, compliance is all too often defined in binders or text documents; what’s a developer going to do with this? But we’re seeing a new idea emerge: define compliance — either to your own security standards or external, regulatory standards in code. If you can define compliance as code, you can then use that code to create a test that can simply be moved into the software release process and managed like any other automated test. Forward-leaning compliance officers will shift their gaze and embrace their developers because it’s through their software developers that they’ll be able to take their craft to a new level.  Keep an eye on this in 2016.
    1.  Executives Will Move Into the DevOps Tent. The DevOps tent will expand to include the executive suite in 2016, and executives are excited to join the practitioner-born DevOps movement. “DevOps” makes one think of only “development” and “operations,” but it’s really a movement focused on how we build and operate high-velocity organizations. Frankly, DevOps is meant for the entire enterprise. The results are both clear and definitive. Mature DevOps organizations see increases in both innovation and quality that flow through to business results, including profit and shareholder value — and, quite possibly, security and safety as well. Executives have seen what code can do, and they’re fervently embracing the far-reaching organizational changes that have accompanied this technology revolution. The good news for DevOps practitioners is that they’re going to find strong support in the executive suite and many will see good things happen to their careers in 2016
    1.  Container Technology Will Be Improved and Grow Even More. Containers — which bundle entire run-time environments into packages that make it much easier to move applications from a developer’s laptop to the test, production and end-user stages — are becoming invaluable, especially in homogeneous green-fields, because they abstract so many problems away. But most enterprises aren’t homogeneous green-fields, so the challenge is how to deal with a variety of legacy technologies without re-writing all the applications that run. I believe that this problem will be addressed in 2016, and container usage will expand by leaps and bounds as a result.

    There will be other big changes in IT during 2016, but, when combined, I believe these seven represent a truly significant tipping point that will catapult enterprises toward the future with a velocity and vigor that can only enhance and enrich growth.

    Abouth the Author/Barry Crist, CEO, Chef

    Barry-Crist-620x930Barry Crist is CEO of IT automation company Chef. Barry has more than 20 years of experience in driving enterprise customer success and adoption with both open source and DevOps software solutions. Prior to joining Chef, he was CEO of Likewise, a company with a strong open source to enterprise go-to-market model that had wide industry reach including OEM agreements with VMware, Dell, HP, Riverbed, and EMC. Likewise was acquired by EMC Isilon in September 2012. Prior to Likewise, Crist served as Vice President of the Application Management group at Mercury, now a division of HP, where he grew North American sales from less than $1 million to more than $75 million. Crist led enterprise sales organizations at Network Associates (now McAfee) and was in various sales, technical, and international roles at Apple.

  • Agile Infrastructure: Because your Infrastructure Deserves Tests, Too

    Agile Infrastructure: Because your Infrastructure Deserves Tests, Too

    Editor’s Note: This post was updated Aug. 4, 2018, to reflect an updated slide presentation at the end of the article.

    Infrastructure as Code is a truly powerful concept whose significance is best described with the words of Adam Jacob, CTO at Chef, in Web Operations: Keeping the Data On Time: “Enable the reconstruction of the business from nothing but a source code repository, an application data backup, and bare metal resources.” And so it’s not for nothing that treating infrastructure as code is regarded a key principle for reducing cycle time and maximizing flow in Continuous Delivery and DevOps, respectively. Even if your plans are less bold as you read this article, wouldn’t you agree that consequent validation of what you automate on the infrastructure level — long before it goes into production — would be extremely powerful?

    So, bear with me for a while and please don’t rush head over heels into automation by simply picking one of the established infrastructure automation tools such as Chef or Puppet, getting everyone trained in Ruby, and blindly start to “automate all the things”. If that’s how you do it, then you are most probably doing it wrong. In You’re Doing DevOps Wrong. Automation in the Enterprise., author Alan Sharp-Paul suggests that, before doing any automation, you first have to understand precisely the requirements and how to validate them to gauge success and maintain quality of your automations over time:

    • Are the configuration files in place? Do they contain the right settings?
    • Is this port open? Is that port closed?

    Why Agile Infrastructure?

    Honestly, you and I wouldn’t expect anyone to come up with reliable software merely by choosing an arbitrary programming language and a nifty IDE without having some sort of engineering process in place, right? Likewise, the (automated) provisioning of a solid production environment demands a no less methodical approach. Luckily, Infrastructure as Code enables us to treat the provisioning of infrastructure as well as automating deployments and configuration as an Agile engineering discipline. Although the concepts of Agile Infrastructure are not entirely new, testing against live infrastructure has only recently been facilitated with the consolidation of tools such as Ansible, Chef, Puppet, Docker, Vagrant and Serverspec under the hood of Test Kitchen (see more on that below). Here are some of the key implications of applying Agile software development practices to the infrastructure level:

    Alignment of Devs and Ops

    Agile software development teams are driven by customer demand and aim to produce outcome in short, regular intervals. Enabling IT Operations and Infrastructure departments to work in small iterations and react to changes quickly allows them to align with and effectively become driven by software development. The outcome of these joint efforts will be both working and deployable software at the end of each iteration.

    Fail Fast: Feedback Loops

    Executing infrastructure tests as part of your Continuous Delivery build pipeline (before an application is deployed and long-running tests are executed) establishes another important feedback loop that enables you to “fail fast” and get the right people involved. The underlying principle is simple: by expressing your intentions twice — once in the code and once in a test — you’ll know you’ve caught a bug whenever these assumptions don’t match.

    Regression Safety

    Believe it or not: whether you are using Ansible, Chef, Puppet or any other — you’re running on code! Their respective issue lists are full of unresolved issues of various severities with new issues coming in regularly. Having a sufficient safety net of tests for your infrastructure in place allows you to quickly identify bugs as they creep in (from either side) and protects you against unwanted regressions. After all, a bug in your infrastructure can be very hard to track down and can be much more severe than a bug in your application.

    Refactoring

    Code refactoring becomes a breeze when you have tests in place that immediately alert you when you broke the desired state of your infrastructure (and all you wanted was to make your code simpler and more maintainable). This is invaluable when you want to recompose your existing infrastructure from 3rd-party components offered on public hubs such as Ansible Galaxy, Chef Supermarket or Puppet Forge, and especially when you are migrating away from your existing automation stack.

    The Red, Green, Refactor Cycle of Test-Driven Development

    Test-Driven Development (TDD) is a popular software development process that involves writing code in short, repeating development cycles with the goal to enforce good design and enhance confidence. While it is certainly no silver bullet and has been controversially discussed in the past, the Red, Green, Refactor Cycle, which is at the heart of TDD and whose principles I want to quickly summarize here, hopefully gives you some inspiration on how to incrementally build up your infrastructure:

    1. Red: Understand the requirements and the state you want to implement. Then think about how to capture this state in a test. Implement the test, then run your entire test suite and see the added test fail — and if it doesn’t, something is really wrong.
    2. Green: Implement just enough functionality to make the test pass. Don’t care much about readability, simplicity and design for now — just make it work. Run your tests again and watch all tests pass. If they don’t, either fix your test or your implementation, or take a step back (revert to a point in your code that’s known to work) and start over with a smaller increment.
    3. Refactor: Now that your tests pass, improve your implementation and make sure, by running your tests again, that you didn’t break anything. Once you are done refactoring, start the cycle over again.

    Keep in mind that each repetition of the cycle refers to a small increment. In terms of infrastructure, such an increment could be the installation of a user, a package or the placement of a configuration setting. When you arrive in a spot where you broke something, you can easily revert to a point in your code that’s known to work. Leveraging such practices really makes the difference between Infrastructure Automation and Infrastructure as Code, as described in Infrastructure as Code – Automation Is Not Enough by Keith Morris.

    Test Kitchen, Terraform, AWS and awspec

    Now, how to get there? Test Kitchen is an easily extensible test harness that allows you to test your code written in Ansible, Chef, Puppet (and others) on various cloud providers, virtualization providers and operating systems with tests written in a variety of test frameworks.

    In the following presentation, I give hands-on examples for you to quickly dive into the topic of using Test Kitchen with awspec as a framework to test Terraform deployments on AWS: