devopsy.kinsta.cloud

Tag: automate

  • ChefConf 2018: Chef Grows Beyond Infrastructure to Apps

    ChefConf 2018: Chef Grows Beyond Infrastructure to Apps

    chefconf keynoteHere I am in Chicagoland for my third ChefConf. I caught the keynotes today with Nathan Harvey, Barry Crist, Adam Jacob, Corie Scobie and more. The big news at this year’s event is that Chef is focused on more than just infrastructure and compliance—Chef is bringing automation to apps and the entire software delivery life cycle.

    I spoke with Chef CEO Barry Crist about this and he said there is a sea change taking place with software is delivered. Chef has seen this wave coming and, over the last few years, has been retooling its offerings to match this. Today, Chef Automate and Chef Habitat represent the heart of Chef’s arsenal in its mission to offer the premier platform to automate the software development, deployment and security life cycle.

    I personally have seen this shift over the last few years. I think back to when we launched devopsy.kinsta.cloud. Chef was already a popular solution within the DevOps community; however, most people will tell you that they used Chef for configuration management. It was great to be able to make sure that as you spun up new instances in the cloud, your infrastructure was configured exactly as you wanted. By automating and using a Chef-type of solution, you could spin up new instances at-will and their configuration would be exactly the same or exactly as you desired.

    At the time, that was pretty cutting-edge. People were writing Chef recipes for all kinds of tasks—again, usually around configuring infrastructure.  With the advent of microservices, containers, Docker and Kubernetes, this type of automated deployment has become commonplace. But the need to automate, especially with security and compliance and app deployment, is perhaps an even bigger mission.

    Crist acknowledged these facts during his keynote. In speaking about Habitat, Crist noted that not only did many of Chef’s customers not understand the real story behind Habitat, but that half of Chef’s own employees didn’t, either. One person who did was Julian Dunn, director of product marketing, who I interviewed on DevOps Chats about Habitat when it first launched. Tony Bradley wrote that it could be a game changer back then, as well.

    Habitat has come a long way since then and still has a ways to go to grow into its ambitious potential, but it is very functional right now. Chef has several clients here at ChefConf talking about their Habitat deployments and plans. Chef now tags Habitat with Automate Any App Anywhere – Build, Deploy, Manage.

    Perhaps further along than Habitat, however, is Chef Automate. In fact, Chef Automate 2.0 was announced ChefConf, and it seems to be an almost complete rewrite of the program, as it is now built on Microsoft Go. Here is Chef’s take on Automate 2.0:

    Chef Automate has been rebuilt from the ground up to provide performance, scale and advanced analytics required by enterprises using DevOps at scale to meet the demands of post-digital transformation. Chef Automate now provides a single control plane for infrastructure and compliance automation to enable a closed-loop ‘detect, correct, automate’ process to make infrastructure effortless.

    New features include:

    • Enhanced operational visibility and debugging – New tools and visualizations, including a streaming event feed, trend graphs and a rich query language, help users analyze infrastructure and compliance automation data from a unified dashboard.
    • Compliance scanning and reporting in any environment – Integration with cloud compliance capabilities of InSpec 2 enhances scanning, which can now be used to evaluate cloud configurations (AWS, Azure, Google Cloud) via APIs for compliance on either an ad-hoc or ongoing basis. Chef Automate 2 also supports scanning of Cisco IOS equipment as a significant start on extending modern compliance capabilities to network devices.
    • Re-architected for speed and flexibility – A Go-based microservices architecture, with a modern, performant web UI atop a REST API, enables Automate 2 to support tens of thousands of nodes from a single installation. In-place upgrades with automatic data migration from Chef Automate 1 ensure that customers retain all historical information.

    Additionally, Chef has released something called Chef Workstation. From the release:

    Chef Workstation enables DevOps teams and practitioners to complete ad-hoc DevOps tasks using all of the familiar Chef capabilities, without agents. This provides a simple and effective way to use Chef to complete tasks and then seamlessly scale those tasks through Chef Automate.

    Capabilities include:

    • New standard desktop/laptop experience for Chef – Includes a new ‘target mode’ for executing Chef recipes or entire cookbooks remotely and Introduces ‘chef’ command as the single point of entry.
    • Foundation for a better workstation experience – Bundling all the tooling previously included in the ChefDK provides an improved user experience and cohesion between tools to deliver a seamless experience for any infrastructure automation task.

    Finally, new enhancements to Inspec, Chef’s compliance and security solution, were announced. Besides security and compliance being built into most everything Chef does, Inspec now sports:

    • Cloud configuration compliance – Verify AWS and Microsoft Azure policies, with Google Cloud Platform support in beta. More than 10 new AWS resources bring it to CIS Level 1 benchmark completeness.
    • Improved out-of-the-box experience – More than 30 new resources, including support for Cisco IOS network devices as agentless targets, are built in.
    • Drastically improved performance – Users now see up to 90 percent performance gains on Windows and 30 percent on Linux/Unix.

    I will report more as it comes here in Chicago and ChefConf 2018.

    — Alan Shimel

  • DevOps Chat: Chef Update with Ken Cheney, Chef CMO

    DevOps Chat: Chef Update with Ken Cheney, Chef CMO

    There are always new developments going on at Chef. We caught up with Chef CMO Ken Cheney to give us an update. Last time we spoke with Ken was right after ChefConf. Since then, Chef continues to move forward on several fronts including Habitat, Automate and INSPEC. We speak to Ken every other month or so, as it is always good to know “what’s cookin’” at Chef.

    As usual, the streaming audio of our conversation is immediately below, followed by the transcript of our chat. Enjoy!

    Audio

    Transcript

    Alan Shimel: Hey everyone, Alan Shimel, editor in chief of devopsy.kinsta.cloud here, for another DevOps Chat. Today’s guest on DevOps Chat is our friend Ken Cheney, CMO of Chef. Ken, welcome back to DevOps Chat.

    Ken Cheney: Alan, it’s great to be here. Thanks for having me.

    Shimel: Thank you. Ken, I apologize, between your schedule and my schedule, it’s been a while since we’ve had a chance to catch up. I think the last time we left off it was – was it right after Chef Conference, ChefConf?

    Cheney: It was actually right before ChefConf. So it’s been a couple months, right? Given that was at the end of May. 

    Shimel: Ken, I don’t think I’ve been home for more than two or three days at a time since then. So yeah, it has been. But for our listeners who maybe aren’t lucky enough to be down in Austin and ChefConf this year, can you fill us in a little bit about maybe what we missed?

    Cheney: Yeah, so ChefConf this year, as you mentioned, was in Austin, Texas. And you know, I think the highlights of ChefConf, on the product side, were some announcements we made around Chef Automate and bringing in our open-source tooling fully under that umbrella of Chef Automate. So Chef Automate is our commercial offering. And first and foremost, we brought in InSpec fully, so now, from one spot within Chef Automate you can see both how you’re doing in terms of your infrastructure and how your infrastructure automation is going with Chef, but also from a compliance perspective you can see what’s happening in terms of policy through the Chef Automate UI. So it’s really – the product has evolved from how it was working before, where we essentially had two servers for and now you have a single, common UI for looking across all the various data points

    In addition we brought in the Habitat data so now you can see the state of your Habitat instances that are running in the wild.  And you know, what I find really interesting about what’s happening with the compliance front is that, you know, we’re really evolving quickly from just being focused on sort of core compute elements to now expanding InSpec out to cover up on cloud-based APIs. So we can now go at AWS and Azure and be able to scan those, as well as VMware environments. So you know, we’re expanding InSpec to be a language that’s not just appropriate for compute, but also for broadly managing all of your infrastructure as code. And so that was announced as well at Chef Conf. And I think that’s a bi move forward. You’re going to see that continued expansion of that InSpec language and capabilities to go after a broad set of other types of infrastructure in the near future.

    And it’s an area where we see just a tremendous interest and uptake from our customers and users. So anyone who hasn’t tried out InSpec, I really encourage them to try it as part of Chef Automate if you want to actually see that InSpec data roll up into common policy views and really understand how you’re doing at an infrastructure level, both in terms of the state of your infrastructure but also how your infrastructure is conforming to policy around compliance and security. You get that from one place. And in this world of WannaCry and other exploits that are happening, it’s amazed me how many organizations, when tools like InSpec exist, still have giant vulnerabilities, right? And here we have this open-source tool that can solve this problem of really understanding what is your exposure. And then you can use a tool like Chef to actually remediate and fix it. It just blows my mind that it’s still a common occurrence.

    Shimel: It blows your mind, Ken? For 20 years it’s blown my mind. For those of us in the security space, it’s no secret that well over three-quarters—actually, probably over 80 percent or 85 percent of attacks and successful attacks actually take place not because of what we used to call Zero Day or unknown vulnerabilities, but because known vulnerabilities were not patched.

    Cheney: That’s exactly right. That’s exactly right, yes.

    Shimel: And that’s the key. I mean even today –

    Cheney: Yeah, and –

    Shimel: It’s crazy. I’m sorry, go ahead, Ken.

    Cheney: Oh I was just going to say, one of the common things, you know, as many organizations haven’t really fully automated – and so they reintroduce the vulnerabilities into the wild all of the time. And what I love about what you can do with InSpec is you can actually – because you’re managing as code, every time you test your software, every time you push a configuration with Chef, you can verify that you’re not reintroducing to file that or something on a keyboard the reintroduces that potential hole in your organization.

    Shimel: Yep, you know – I’m sorry, can you hear?

    Cheney: Yeah, yeah, go ahead.

    Shimel: I’m getting a little audio back. I hope this is recording well. Ken, we did a survey a while back where only 75 percent of organizations are actually scanning their code for vulnerabilities prior to deployment. And you know, what are the other 25 percent thinking?, is kind of what I think. But anyway, on the topic of security, Ken, I wanted to quick mention that Matt Ray of Chef is actually speaking at a conference I’m doing as a part of RSAAPG on DevSecOps next week. And Matt, I think, is based in Australia, correct?

    Cheney: Yes, he is. Yeah, Matt is based in Australia and he will be talking about InSpec, and folks who are there can attend his session and learn more, for sure. 

    Shimel: So maybe it’s because I’m traveling to Singapore and so it’s kind of right in my face, but to me, it’s this global – I mean, DevOps has really become a global – I don’t want to use the word “phenomenon” – but a global force. And we’re seeing it

    Cheney: Yes, it is.

    Shimel: – you know, certainly in EMEA, but we’re seeing it in AP. And then just this week, Ken, I think Chef just announced their first Latin American partner, right?

    Cheney: Correct, correct. Yeah, we are definitely seeing DevOps at a global level and seeing broad interest. And what’s interesting is that – Alan, I’ve been in the industry a while, and it used to be a little bit of a lag – you know, a several-year lag before you’d see markets such as Latin America or parts of Asia pick up technology that would originate in the United States. Or process ideas. But things move much more quickly now and you’re seeing global companies with operations in India and operations in China. And so we as a company do a lot of activity at a global level. You know, if you look at how we’re doing meet-ups and where we’re doing meet-ups, they’re happening all over the place. And our partner ecosystem, which is well over 300 partners, covers the globe, because we see that demand.

    Now how we enter any given market varies depending on what’s going to work best in that market. So some markets we address through partners and some markets we actually put our own physical presence in that market.

    Shimel: Yeah, we see the same thing at devopsy.kinsta.cloud, Ken. You know, our readership when we first started was almost three-quarters U.S.-based, and now it’s only about 37 percent, with the rest being global. But it’s more than just spreading through the world. It’s what you said – there’s not as great a lag time. At one point, Latin America was three – we used to tell people, “Oh, it’s three years behind what we’re doing here.” And that’s not the case anymore, certainly.

    Cheney: Yeah, I’ll tell you, the highest volume of leads that we get come from the Asia Pacific region.

    Shimel: Really.

    Cheney: Yeah, so there’s just a tremendous amount of demand coming out of there. I’ll tell you, if we do a meet-up in Bangalore or Delhi, we’ll have 300 people show up to that meet-up.

    Shimel: Wow, that’s fantastic. You know, we’ll talk more about that off-mic. But Ken, a lot of what’s driving strong numbers like 300 people at a meet-up and stuff like that is training and availability of training, right?

    Cheney: That’s right.

    Shimel: And that’s something Chef has always done really well. But you guys recently rolled out – not Pilot, excuse me – what is the name of the –

    Cheney: Chef Automate. The Chef Automate Pilot, yeah. We actually rolled out a whole new online learning site called Learn Chef Rally and it’s gotten very fast adoption and has a lot of users hitting it every day. And as part of that we have what’s called the Chef Automate Pilot, which is really targeted at an individual who wants to really understand Chef over the course of maybe a lunch, right? So that kind of lunchtime evaluation where they want to get a feel for how Chef works and what it does for them. But as a whole, Learn Chef Rally constitutes a whole set of modules that will walk you through the various parts of our technology. So if you want to learn about compliance, there’s an InSpec module and a Chef Compliance module. If you want to learn even about DevOps concepts, you can go and learn about them there on the Learn Chef Rally site.

    And we’ve had such strong uptake. You know, we’ve seen a lot of customers who would like to bring that learning site in-house and customize that learning environment for their own company. There’s been such a strong interest. So it’s a great resource and I encourage people to go look at it.

    Shimel: Sure. Ken, well, the logical question then is, how do they get there?

    Cheney: Yeah, so it’s very easy to get there. You can just go to learn.chef.io and you can get started.

    Shimel: Excellent. That’s a good one. So that’s Chef Rally and it’s highly recommended. It sounds like there’s a lot and I’m sure you guys are planning to add more into that. So Ken, we are kind of in the dog days of summer here but it’s certainly not slow, it doesn’t seem, anywhere. Any other news, events?

    Cheney: Yeah, well we just passed the one-year anniversary of Habitat and I’ll note that, you know, we at ChefConf made some Habitat news. So we announced a Habitat build service and we made improvements to the Habitat scaffolding, which are essentially build packs to make it easier for people to package up their applications, as well as we’re writing a set of enterprise-ready Habitat plans or common languages, common platforms, such as like BigData, Cassandra, Spark and real common kind of web apps. And so Habitat continues to evolve, and just in the past couple of weeks we announced the pre-release of Habitat for Windows. So along with what we had for Linux support, people can now use Habitat to package up their Windows-based applications. And from our view, Habitat is now the premiere packaging solution for containers. It’s the fastest, easiest way to package an application and have it run in a container and be able to manage it across the life cycle.

    Shimel: That rocks. I mean, because certainly, Ken, the one thing that we’re seeing here is the – you know, we talk about the Internet time crunching or compressing time. The pickup on containers? I’m old enough, and probably you are too – remember when VMware first came out, right?

    Cheney: That’s right.

    Shimel: And through virtualization, it went really quick. The move to virtualization went a lot quicker than let’s say the move off of client server or something like that, from peer-to-peer, or peer-to-peer to client server or what have you. But the move to containers – man, I’ve seen several surveys recently that showed container in production numbers upwards of 50 percent of organizations. And I thought frankly this wasn’t true, but it says something. Right?

    Cheney: Yeah well, you know what’s interesting though about that move? What’s interesting about that move, Alan, is that the majority of containers that are running out there are being used almost like a VM. You know, 75 percent of containers right now are running a full OS.

    Shimel: Yep.

    Cheney: And they’re basically just using it as a way to kind of attempt a higher level of abstraction and remove a little bit more of the infrastructure complexity that they were encountering when they were using VMs. So what I think the next wave is going to happen, I think people are going to actually figure out how to actually better package their applications to actually take advantage of what they can do to remove unwanted dependencies and remove a lot of the weight around the application that they’re currently doing today when they just package up the full OS and all of the broad set of unassociated dependencies that they’re packaging right now. So I expect that’s going to be the next wave and we’re there to help them when they go there.

    Shimel: Agreed, agreed. Ken, any shows or conferences coming up, other than of course the DevSecOps that I mentioned, where Chef will be presenting or a part of?

    Cheney: Yeah, well yeah, we will be at VM World, coming up. We’ll be at Microsoft’s big event that’s coming up. And of course as we look toward November we have our own community summits that happen in the fall timeframe. So those are a great way that we bring our community together. So the first one is going to be in London Oct. 10 and 11. So that’s coming right up as well. So expect to see Chef and our brand showing up at a lot of different places in the coming months.

    Shimel: Great. Very cool. Well Ken, we’re about out of time here. 15 minutes goes quick. But hopefully it won’t be a few months before you’re back on –

    Cheney: Yeah.

    Shimel: – and we can pick up where we left off here.

    Cheney: Yeah, Alan, enjoy your travels and thanks so much for the time today.

    Shimel: Not a problem. Ken Cheney, CMO of Chef, here on DevOps Chats. This is Alan Shimel for DevOps Chat and devopsy.kinsta.cloud. Thanks for listening, everyone, and we’ll see you on another DevOps Chat soon.

    — Alan Shimel

  • All Day DevOps: System Hardening with Ansible

    All Day DevOps: System Hardening with Ansible

    The DevOps pipeline is constantly changing.  Therefore, relevant security controls must be applied contextually.

    We want to be secure, but I think all of us would rather spend our time developing and deploying software. Keeping up with server updates and all of the other security tasks is like cleaning your home: you know it has to be done, but you really just want to enjoy your clean home. The good news is you can hire a “service” to keep your application security up to date, giving you more time to develop.

    At the recent All Day DevOps conference, Akash Mahajan (@makash), a founder/director at Appsecco, discussed hardening your system’s security with Ansible. In addition to his role at Appsecco, Akash is involved as a local leader with the Open Web Application Security Project (OWASP).

    Misconfiguration

    During his presentation, Akash mentioned the OWASP Top 10 Security Vulnerabilities list, zeroing in on No. 5: Security Misconfiguration. To determine if you comply with the guidelines, No. 5 on the list asks:

    • Is any of your software out of date?
    • Are there any unnecessary features enabled/installed including ports, services, accounts, pages or privileges?
    • Are default accounts and their passwords enabled/unchanged?
    • Are security settings and libraries not set to secure values?

    I am sure no one reading this article still uses the default administrator password, but can we say the same of your peers? Have you gotten around to installing the latest software patches on your server?

     

    Automation

    If a task can be automated, developers automate it. So we should automate our security tasks too, where we can. OWASP provides guidance here, suggesting you should:

    • Have a repeatable security hardening process.
    • Ensure your development, QA and production servers are configured identically but with different passwords.
    • Automate the process to minimize the effort required to setup a new secure environment.
    • Implement a process for deploying all new software updates and patches in a timely manner to each deployed environment.
    • Run scans and audits periodically to help detect future misconfigurations or missing patches.

    This is all part of security hardening, which is, “the process where we identify default configuration present on a system and apply changes that will change the configuration to secure values.” This can be applied to your network, transport, application, and kernel networking parameters.

    Ansible Playbooks

    Ansible is one of the solutions Akash likes to work with, but there are others solutions on the market that provide similar value. Without trying to endorse or evaluate one solution over another, let me share perspectives from Akash’s experience with his toolset.

    Why does he like it? It boils down to playbooks. An Ansible playbook is a codified security document, allowing you to describe the desired state of a system, rather than the specific steps of how to get to that state. As Akash points out, things change—it is better to have the end state described rather than have to change commands when the system changes.

    Other advantages of playbooks include:

    • Playbooks are written in YAML providing us with structure that we can learn and train on
    • Playbooks are text files, so we can use Git for version control
    • Managing playbooks is just like managing any software project
    • Playbooks are infrastructure as code but for security
    • Playbooks consist of roles, a key aspect of security
    • Numerous playbooks are available as open source

    The bottom line is you can—and you should—automate your security hardening process. Your users and other stakeholders will thank you and, most of all, you will thank yourself because you can spend more time on the things you love to do.

    Ansible is just one example of a solution that can be used to automate your security tasks. If you want to know more, Akash goes into further detail on getting started with Ansible in his full All Day DevOps conference session (just 30 minutes). The other 56 presentations from the All Day DevOps Conference are also available online, free of charge here.

    This blog series is reviewing sessions from the All Day DevOps conference from November which hosted over 13,500 registered attendees. Last week I discussed, “DevOps at Massive Scale.” Next week, look for “Operationalizing a Red Team for Fun and Profit,” delivered by Intuit’s own Ian Allison.

    — Derek E. Weeks

  • DevOps Chat: Ken Cheney, Chef on AWS OpsWorks for Chef Automation

    DevOps Chat: Ken Cheney, Chef on AWS OpsWorks for Chef Automation

    devopsy.kinsta.cloud catches up with Chef’s busy CMO Ken Cheney. Ken give us the inside look at AWS’s OpsWorks for Chef Automate. While this is an AWS offering it is based on Chef and is a pretty big bet by AWS on Chef. In addition Ken brings us up to speed (pun intended) on other things regarding Chef. Ken is always a great person to chat with and full of great info. We are going to have Ken on every month or so, we can count on hearing more Chef and DevOps talk soon.

    In the meantime, as usual the streaming audio of our conversation is below and the written transcript of our chat follows. Enjoy!

    Streaming Chat

    Transcript

    Alan Shimel:                Hi, everyone. This is Alan Shimel, devopsy.kinsta.cloud, here for another DevOps Chat. I am happy to be joined in this episode of DevOps Chat by Chef CMO, Ken Cheney.

    Ken, welcome.

    Ken Cheney:                Alan, nice to hear your voice again.

    Alan Shimel:                Nice to hear from you, Ken. Actually, I think the last time you were on our podcast or the last time we spoke you had maybe just been appointed CMO, or maybe it was even before you were appointed CMO at Chef, but it’s certainly been a busy time for you I’m sure.

    Ken Cheney:                Yeah, it has. There’s been a lot going on in the past two months and some of it we’re going to talk about today, the work we’ve been doing with AWS.

    Alan Shimel:                Yeah. I think the last time I spoke to you was right before AWS re:Invent, and there was some big Chef news coming down the pipe that we weren’t able to talk about then. It was still kind of under embargo, but it certainly is fair game now.

    So, Ken, why don’t you share with our audience what was big for Chef at AWS re:Invent?

    Other Great DevOps Chats to listen to:

    DevOps Chat: Nigel Kersten, Puppet, 2017 DevOps Survey
    DevOps Chat: Huw Price, Smart Automation Is Key
    DevOps Chat: InterConnect 2017 w/ Randy Newell, IBM DevOps
    DevOps Chats: Aruna Ravichandran, Co-Author, DevOps for Digital Leaders

    Ken Cheney:                First off, AWS re:Invent, a super-impressive show to go to. So if folks haven’t gone, I’d recommend this next year that they pencil it in. But at re:Invent we announced – actually AWS announced, it’s their offering, a new offering called AWS OpsWorks for Chef Automate. It really brings for the first time the full capabilities of Chef’s commercial platform to AWS as a managed service offering. So they provide support and they backup the offering 100 percent.

    So it’s a new offering that makes that job of configuring, deploying, scaling, both in the cloud as well as on-premise infrastructure simple and secure. We get there by really doing what Chef has done best over the years, which is automating infrastructure as code.

    I think to really highlight what I said, Jassy at re:Invent went out of his way to make the point that hybrid is the new normal in their view. So what’s interesting about what we are doing with AWS is you can manage both in the cloud with AWS as well as your on-premise workloads, and you can have that consistency in how you manage your resources across both.

    Alan Shimel:                Absolutely. Now, Ken, our audience may not be aware, but I happen to have known this, that Chef was sort of the dominant tool in use for AWS customers now for a few years; I mean an outrageous amount. If I’m not mistaken, wasn’t it like 65 or 70 percent of AWS customers who use some CM are using Chef?

    Ken Cheney:                Yes. A large proportion of AWS customers use Chef, and AWS themselves use Chef for many of their core services. It’s a platform that is really well suited to the job of: how do you get your workloads to the cloud, and how do you manage those workloads on an ongoing basis in the cloud?

    Alan Shimel:                Absolutely. And of course, Ken, when we talked about hybrid – I actually had a conversation yesterday, actually with maybe some IBM folks. We were talking about mainframes aren’t going away any time soon, obviously, the mainframe market. It’s probably bigger than ever in terms of dollars.

    But the idea of working with your systems of record and systems of engagement, and one is on a mainframe and one is in the cloud, is becoming more and more the norm for large organizations, right?

    Ken Cheney:                Yeah. It really is.

    Alan Shimel:                Yep. I mean they’re not going to just walk away from billions of dollars invested in mainframes and datacenters, but in order to compete with stuff that lends itself to it, it’s hard to beat the economics of the cloud. So for the foreseeable future anyway, probably the rest of our working lives, Ken, hopefully, hybrid cloud is going to be the dominant form there.

    Ken Cheney:                Right. Then if that is the case, then manageability and that kind of consistency in your manageability becomes front and center, and that’s where Chef plays a big role in being that bridge that really allows you to have a platform where you can store those configurations in one place, where your resources are going to work perfectly every time at scale, and you make changes in that single place and propagate those changes across those environments. So whether you’re running under a mainframe environment or you’re doing dev, test, prod in the cloud you get that repeatability.

    Alan Shimel:                Absolutely. Before we leave the AWS stuff though, Ken, I just want to, for our audience that may be interested, this is available now, today, through the AWS marketplace?

    Ken Cheney:                No. It’s a core AWS offering.

    Alan Shimel:                Okay.

    Ken Cheney:                So if you’re an AWS customer, you login through the AWS console and it’s one of the offerings that you can select. So it bills through AWS. It’s managed by AWS. It’s supported by AWS. So it’s very easy if you’re an AWS customer to get started, and you can get started for free, up to ten nodes for free per month.

    Alan Shimel:                Great.

    Ken Cheney:                Yeah. You can get started and get all those advantages.

    Alan Shimel:                That’s cool. All right, so people can go check it out right now. If you’re using AWS, it’s a core offering. And I guess that kind of makes sense because it is an AWS offering after all, isn’t it?

    Ken Cheney:                That’s right. We’re partnering very heavily with them to make sure it gets the kind of traction and success that we have envisioned.

    Alan Shimel:                You know what? That really is big news, Ken. Great job by the Chef people. Give my kudos to the business development guy. He did a hell of a job there.

    Ken Cheney:                Will do. Thanks, Al.

    Alan Shimel:                Weren’t you the business development guy, Ken?

    Ken Cheney:                I was. I’m weird, so –

    Alan Shimel:                Yeah, I know, just a little inside Chef joke. Ken, we talked a little bit about hybrid cloud and this new offering at AWS. Let’s look at the bigger picture. Here we are in January of 2017 and it’s not too late yet for – where do you see DevOps in 2017? Where do you see Chef in 2017? What big picture item should we be looking at?

    Ken Cheney:                What’s interesting right now in the market is that – you and I both see this, Alan – the market is still very young. I saw some Gartner data where you can extrapolate out that 90 percent of the market is still untouched. Even that 10 percent that has started down the path of adopting is only partially adopting.

    Yet what’s very clear, both at an industry level, if you look at industry data, as well as what we see with our own customers, is that DevOps has a significant impact on the bottom line. Companies that adopt DevOps have been shown again and again to outperform their competitors.

    Alan Shimel:                Yep.

    Ken Cheney:                And I think that DevOps – we’ve been talking about this for some time – but it’s really a full-on move from the early adopters, those organizations who figured out how to be disrupters, to the rest of the market who are being disrupted.

    Alan Shimel:                Yeah. It’s DevOps or die, as I like to say.

    Ken Cheney:                That’s a great statement.

    Alan Shimel:                Yep. Ken, I saw some interesting statistics from a study that CNA had commissioned, 1,700 respondents, and it said something like only 34 percent, so just a third of organizations had adopted DevOps.

    Ken Cheney:                Right.

    Alan Shimel:                Which to me and probably to you too, right, because how many organizations do we talk to that either haven’t adopted DevOps or don’t want to adopt DevOps? We live in a DevOps bubble. We think everyone does, but yet the fact is it’s only about a third, 34 percent. When you look at it that way, it’s barely mainstream, right?

    Ken Cheney:                That’s right. And even those who have adopted it typically adopted it in a small silo.

    Alan Shimel:                Right, the pockets of DevOps, as they call them. I continually refine that, Ken. What I mean by that is you’ll have a team – a team – using DevOps, maybe a team here and a team there, but only when those multiple teams come together under a common DevOps sort of processes, framework, manager, whatever, do I really consider the organization doing DevOps. So for me, an organization that’s doing DevOps is doing it with multiple teams.

    Ken Cheney:                Yeah. I totally agree, and I think this is the year where we start to see organizations really at an executive level begin to put their shoulder into what that means. How do you actually begin to go automated scale?

    And at the same time they’re going through the tech modernization investments around embracing the cloud, embracing microservices, containerization. So I think this is going to be a really interesting year, where I expect many traditional shared services organizations are going to be under a lot of pressure to change.

    Alan Shimel:                I agree 100 percent. It goes back to the DevOps or die thing. All kidding aside, if these organizations don’t change you’re going to start seeing them falling by the wayside.

    But you know what else I think, Ken, and I’m interested in your opinion on this. DevOps tools have to change, too. They have to continue to mature. They have to continue to be enterprise-ready. They have to continue to deliver and meet the challenges that organizations are faced with.

    Ken Cheney:                I totally agree. I think that there’s been a Cambrian explosion of tooling.

    Alan Shimel:                Yes. That’s a great phrase for it, a great way to put it.

    Ken Cheney:                Yeah. And there’s been a rapid kind of organic team level adoption, but what’s happening is that there’s been a lack of standardization, a lack of scale. When you look at the disrupters who have been wildly successful, and many of those are Chef customers, they have been successful automating at scale. Automating at scale means you do drive consistency across the board.

    The path to get there though, from my view, is you have to be thinking about vertical integration. If you look at our strategy, like take what we announced at AWS, that is where we are actually vertically integrating into their platform in a very real, meaningful way. And we feel the same way about the broader ecosystem of _____ capabilities that we need to integrate in with, to make that job of automating at scale easy.

    Alan Shimel:                Yeah, agreed. I do think you’re onto something with that, also vertically integrating, but it’s also integrating into the IT stack. For instance, for me and maybe it’s my own personal bias and history, if you look at 2017 as a year where we have to see security integrating with DevOps and we can call it DevSecOps or Rugged DevOps – I don’t really care what you want to call it – but there has to be that tighter coupling, that tighter integration into the IT stack of security shifting left as part of this DevOps.

    I know Chef obviously has now some history in trying to bring compliance at scale and compliance in security and speed. But I think that has to accelerate.

    Ken Cheney:                Yeah, it does. We are pouring the investment into that set of capabilities. When we look at what our customers are trying to get done, there are things like driving known vulnerabilities down to nothing, being able to remediate in minutes as opposed to days. To get there, that concept of shift left that you put forward is front and center. You have to start thinking about shifting risk to the left.

    So we’ve invested a lot in, one, our InSpec open source framework that people should go check out, as well as the overall compliance capabilities, where you can have broad compliance profiles that ensure that the state of your infrastructure and applications is conforming to policy.

    Alan Shimel:                Got it. You know what, Ken? You mentioned open source and it triggered something. What’s happening with Chef Habitat, anything new you can share with the audience?

    Ken Cheney:                Yes. Chef Habitat has been in the market now for about nine months. We have been seeing a tremendous amount of traction. We’re going to be launching a Habitat road show in a few weeks, visiting multiple cities. So people should go to the Chef website and see if we’re going to be coming to a location near them.

    We are really now moving into the phase where we have large enterprise customers moving workloads into production with Habitat, and they’re really targeting that job of: how do I easily encapsulate my applications and manage those applications, in a way that I don’t have to spend a lot of calories worrying about extraneous services and getting true portability out of it, meaning I can move that application workload anywhere I want?

    So Habitat is definitely getting legs at the moment. We’ve had some new product news that will hit in a couple weeks, Alan, so you and I will be talking about that.

    Alan Shimel:                Fantastic. All right, a little teaser. Ken, to tell you the truth, we’ve gone way over our usual time that we allot for podcasts. We don’t want to take up too much of people’s time, but let me just do a quick recap. For people who want to find out more about Chef Habitat, what’s the URL?

    Ken Cheney:                They go to the website, which is Habitat.sh or they can go to the Chef site, which is Chef.io.

    Alan Shimel:                Right, and it’ll just run through from there and they can find out about the Habitat road show there. The Amazon stuff you can get right through your Amazon portal or dashboard, if you’re an AWS user.

    Ken Cheney:                That’s right. AWS just did a webinar yesterday that was recorded that does a nice walkthrough on it. So they are actively out there promoting it and pushing it for their customers.

    Alan Shimel:                And the open source compliance and security?

    Ken Cheney:                You can get that through the Chef website.

    Alan Shimel:                Also.

    Ken Cheney:                Yep.

    Alan Shimel:                Okay. I just want to make sure people get it, Ken. Ken, we could probably spend hours talking. We’ll have to have you back on soon, maybe in a few weeks when you have your next bit of news. But until then, keep up the great work. Oh, we should mention ChefConf.

    Ken Cheney:                Oh yeah, ChefConf coming up May 20th to 22nd in Austin, Texas. People can go and register today. We’re pretty excited. We just closed the call for papers last night and had a record number of submissions. So it’s going to be a great event.

    Alan Shimel:                Cool. I went last year. It was a great event. I had a great time. It was some great content. The Alaskan Airlines was still my favorite, Ken. Actually it got me to fly Alaskan Airlines watching that guy present.

    Ken Cheney:                It was amazing, yeah.

    Alan Shimel:                It was the first time I ever flew them, and I was counting to see how quickly the door closed and how quickly they opened it, from his whole presentation. It was a great one.

    Anyway, Ken Cheney, CMO Chef, thanks for being our guest on this episode of DevOps Chat. Continued success at Chef and we’ll be watching and listening to find out what’s new.

    Ken Cheney:                Thank you, Alan.

    Alan Shimel:                All right. Okay, everyone, this is Alan Shimel for devopsy.kinsta.cloud and DevOps Chat. Thanks for listening and we’ll see you on our next DevOps Chat.

    — Alan Shimel

  • Austin & Chef Give ChefConf 2016 That Warm, Fuzzy Feeling

    Austin & Chef Give ChefConf 2016 That Warm, Fuzzy Feeling

    I have written before about the DevOps community in Austin. It is one of the most welcoming and warm groups of people I have ever interacted with. Combine that with the HugOps operation of the friendly folks of Chef and you wind up with a great experience. That pretty much sums up ChefConf 2016 for me. Beyond the warm and fuzzy though, there was also a lot on display on what the future of business is all about. To paraphrase a line from the movie, The Graduate, I have one word for you: AUTOMATION.

    Automation was on full display at ChefConf 2016. And not just your grandpa or dad’s automation either. For those who think Chef is still all about configuration management and automating servers, think again. I am talking about automation extending into every nook and cranny of the business. You only had to see Alaska Airlines CIO, Veresh Sita’s presentation to understand what automation really means. But truthfully that was just one of many talks and sessions that demonstrated the power of automation and how Chef is pushing automation up and down the business.

    There was also the announcements regarding Chef Automate and Habitat. While we have covered Habitat already, a quick word on Automate. Automate which is years in the making, is actually the combination of Habitat, Chef Delivery which was released last year and defines the DevOps workflow, Chef itself and Chef InSpec, the compliance and security project based on the technology from the acquisition Chef made last year of VulcanoSec. All of these are combined in one enterprise class package that Chef CEO Barry Crist called “our flagship product”.

    Beyond that though to really give you a feel if you couldn’t make it to ChefConf this year, there was a palpable buzz around the place. I think the feeling is that so many good things are starting to come together. While many label many of these under the DevOps umbrella, it is automation, DevOps, Agile and more combining to enable businesses to do more, faster and better than they ever have before. In addition, as these trends continue to evolve and accelerate the pace of innovation also continues to accelerate.

    I had a chance to speak briefly with Dr.Nicole Forsgren, who works at Chef but is also one of the key people behind the State of DevOps survey. Forsgren said that is why we are seeing the gap between high performing IT led companies and those who are not continuing to widen. The gulf will continue to grow as companies adopting these practices pull away from those who do not.

    From my point of view it is amazing to see that since I first became involved in IT we have seen IT move from a cost center as viewed by many an executive to the engine that drives the business.  As I remarked to several people at ChefConf, we certainly live in interesting times!

    I was also able to use the Chef Newsroom to do a few interviews there. We have put them up in our DevOpsTV Youtube channel under ChefConf 2016 playlist. There are about 8 or so total but I am embedding two to give you a taste. You should check out the rest if you have a chance, as they are pretty good.

    The first is with George Miranda of Chef and we talk about Habitat and a little about Automate.

    This second interview is with well known DevOps journalist, writer and community member, Ben Kepes, enjoy!

  • Simplify and expedite server management

    Simplify and expedite server management

    Deploying, managing, and maintaining servers can be tedious. Deploying, managing, and maintaining thousands, or tens of thousands of server instances is overwhelming—bordering on impossible—without the right tools. Thankfully, the right tools exist.

    The Pain of Doing Things the Hard Way

    I know I am dating myself, but when I was an IT admin for a dot.com startup, I had to deploy servers the old-fashioned way. UPS would show up with a box from Dell. I’d unpack it, and mount it on the rack in the datacenter. Then, I’d begin the process of configuring the operating system, and installing the application software. Each server was essentially a full-day project, and we seemed to constantly be adding new servers, so my full-time role was basically “racking and stacking” for a while.

    Adam Jacob was experiencing similar obstacles when he developed Chef. Working as an IT infrastructure consultant, Jacob was frustrated by the configuration tools that were available, and the lack of a simple, repeatable process. Rather than starting from scratch and reinventing the wheel for each client, he developed Chef to enable IT admins to create modular, reusable “recipes” that automate the process.

    Server management doesn’t end with deployment and initial configuration, though. You also have to maintain the servers, and resolve issues that might occur. When an issue arises, it could mean hours of troubleshooting to identify the root cause, and another few hours to work out and implement a resolution.

    I used to get into philosophical debates with management and other IT personnel because it was my opinion that we should just skip the troubleshooting and rebuild the server from scratch. My theory was that it might be nice to know what caused the problem, but from an operational perspective I knew I could rebuild the server and restore normal operations in a couple hours. Troubleshooting, on the other hand, was an unknown—both in terms of whether or not it would even be successful, as well as how long it would take.

    The Beauty of Doing Things the Easy Way

    Tools that automate configuration and deployment of server instances automatically solve both of these problems. They expedite deployment and configuration, and simplify management and maintenance—freeing IT admins and IT personnel to focus on more important, less mundane tasks.

    Tools like Chef help IT manage complexity. It is great from a DevOps, automation perspective to have a tool that enables you to quickly deploy and configure servers, but the real value comes from the ability to maintain a stable, dependable network with minimal effort.

    Chef nodes “phone home” periodically to ensure the configuration matches the recipe it has been assigned from the Chef server. Any deviations are automatically overwritten—so any changes that may have been made locally on the node are erased, and the server remains configured as it should be.

    In order to implement lasting configuration updates, the changes have to be made to the recipe at the Chef server level. The bonus is that once the recipe is updated, all Chef nodes working from that recipe will be automatically updated.

    When an issue occurs on a server instance, it should ostensibly be corrected automatically when the node communicates with the Chef server and overwrites the configuration. However, even if that doesn’t happen for some reason, the use of DevOps tools like Chef still makes the task of troubleshooting and/or resolving server issues much more efficient.

    With the push of a button, you can just deploy a new server—properly configured and ready to go. You could choose to kill the server experiencing issues and just replace it with the new one, or you could launch a new server to take over the roles and responsibilities of the affected server while you take the time to troubleshoot if you really need to understand what the root cause of the problem was.

    Embrace simplicity

    Apparently, I was an IT admin at the wrong time, and I got out of that career path right when things started to get interesting. Necessity being the mother of invention and all that, Adam Jacob created a platform with Chef that reduces the complexity and tedium of deploying and managing servers.

    Chef isn’t alone. Tools like this form the foundation of DevOps because a culture that combines development and operations, and iterates new releases on a rapid scale needs to have this sort of automated IT infrastructure to keep up.

  • From Simple Automation to DevOps Like a Boss

    From Simple Automation to DevOps Like a Boss

    Automation as We Know It Today

    If you haven’t been hiding in a cave in the last year or two, you’ve probably heard the terms DevOps and infrastructure automation more than once… But even today, infrastructure automation is mostly focused on setup and deployment of complex systems. For example, if you’d like to deploy your application to the cloud, you would likely automate the steps of provisioning the cloud resources, installing the right components on top of these, and then orchestrating the startup of your components – or better known as – cloud orchestration. Take even the simplest application that has a web server and database. After installing and configuring everything, you’d first need to ensure that the database is started, and only then the web server. You’d also need to propagate specific runtime information from the database to the web server, such as the database’s host and port. This stage, for the most part, is where most automation processes focus on today.

    DevOps: Stage Two

    Automation, however, can do much more for you. Let’s imagine, following your initial automation of the setup and deployment of our your application, that you want to make changes to your infrastructure or application. For example if you have a patch you’d like to install to the database or operating system, or if you want to update a piece of your code on your cluster of web servers. Both of these concerns involve pretty complex processes that you need to automate. For an infrastructure upgrade, you would typically roll out the upgrade server by server, take down one server at a time, update it, then restart it. Even though it doesn’t happen very often, there’s still a lot of benefit in automating this process and minimizing the chances of human errors. Pushing new code is a bit different. For starters, you do it much more often, even a few times a day. Also, (no offence), it is likely that the stability and quality of your code is lower than that of an approved patch to the OS or database. So the risk of something going wrong if you don’t automate is even higher with this process. That’s why there are common automation strategies that help you deal with this risk. Two such strategies are the Canary Instance strategy and the blue/green strategy. The former derives its name from the mining days, when a canary bird was sent out into the mine before the miners, to check if the air was toxic or not. So in this fashion, a “canary instance” is chosen, the new code is pushed to that instance alone, and then a number of sanity tests are performed. If these pass, the code is then pushed to a few more instances. If all goes well again, the code is then sent to the rest of the infrastructure. In the latter strategy (A/B), the code is sent to only a part of the servers. In an even more sophisticated approach, the code is only enabled to a portion of the users, for example, a certain geography or time of day. All of this ties into how to setup and manage an ongoing system. But although this process, as well the infrastructure upgrade, can get quite complex, especially when dealing with large scale, live systems, in most organization code pushes and upgrades are still not automated. But there are more than a few companies that are blazing the trail for the rest of us.

    The Plot Thickens – Monitoring Like a Boss

    So once you finally saw the light, put in your sweat and tears and managed to automate your setup, upgrades and code pushes. What’s next? This where it really becomes interesting, and where you can really take your business to the next level, with the monitoring of the entire system, and eventually, what you do with the aggregated data.

    More to come…

    In my next post I’ll discuss the different levels of monitoring and logging to get the to promised land of proactive DevOps.  And then…take that even further.  So stay tuned.