devopsy.kinsta.cloud

Category: Features

  • Automic v12 Promises to Streamline Business Automation

    Automic v12 Promises to Streamline Business Automation

    Automic is in the business of automation. The company recently launched a unified suite of business automation tools called Automic v12 that promises to streamline operations even more.

    I wrote about Automic about a year ago. At the time, I noted how the name Automic is a sort of double entendre that has “auto” right in the name but sounds like “atomic,” which makes it sound very dynamic and powerful. I also wrote about being confused by what exactly Automic was offering. It was dubbed “continuous everything,” even though it seemed to focus on only three areas, and I wasn’t sure what the actual name of the product or platform was.

    That was then, this is now. Fast forward a year and it seems as though Automic—and its brand messaging—have both matured.

    According to its mission statement, Automic strives to drive powerful outcomes for customers in today’s global economy by automating their critical business processes. In the wake of five successful acquisitions, Automic reports double-digit growth and 2,700 enterprise customers representing a variety of key industry verticals. Apparently, the business of streamlining automation is booming right now.

    “Automic Release Automation (ARA) v12 is all about simplifying the steps necessary to utilize and maintain the solution to enhance the value proposition of the automation platform as a whole. The intelligent ‘Zero Downtime’ deployment and update capabilities are not simply about updating Automic agents with no downtime. They are also about removing infrastructure maintenance bottlenecks with intelligent software that can determine, for example, which agents are best updated at which time,” explains Julie Craig, research director at analyst firm Application Management Enterprise Management Associates in a press release statement. “With an enhanced REST API, customers can leverage ChatOps to use their favorite chat client to interact directly with the ARA platform while processes are in-flight, without going through a dashboard. Both of these new features are impressive differentiators which both current customers and prospects will likely appreciate.”

    In my opinion, one of the most valuable aspects of Automic v12 is that it is designed to leverage the infrastructure you already have. Automic can help you implement and manage a hybrid environment and embrace DevOps principles and practices without ripping out everything you’ve got and starting over.

    Beyond that, it is designed to facilitate automation, and automation is crucial for DevOps success. The new features in Automic v12 claim to remove maintenance windows, simplify management of core business apps and drive intelligent insights across your environment—connecting disparate islands of automation to enable more valuable business intelligence.

    If you are looking for ways to improve automation and embrace the continuous continuousness of DevOps, you should take a look at what Automic has to offer. Automic v12 might help you streamline and manage your DevOps environment more effectively.

    — Tony Bradley

  • DevOps Chat: Cognitive DevOps w/ Hayden Lindsey, Enterprise Systems and DevOps IBM

    DevOps Chat: Cognitive DevOps w/ Hayden Lindsey, Enterprise Systems and DevOps IBM

    I recently had a chance to catch up with Hayden Lindsey, distinguished engineer and VP of Enterprise Systems and DevOps at IBM. I have spoken with Hayden several times as well as seeing him present at various conferences. Hayden truly understands how DevOps and related practices are changing the game for Enterprise Systems. Hayden and I caught up on Cognitive DevOps, as well as new developments in the enterprise systems.

    His leadership of the Enterprise Systems team at IBM has really seen them make tremendous progress in keeping “big iron” relevant and up to date with the latest technologies. This includes acquisitions of new technologies which Hayden speaks to me about that are bringing next generation DevOps to market.

    Have a listen to our conversation and if you like follow along with the transcript below.

    Alan Shimel:                Hey, everyone. Alan Shimel, DevOps.com, here for another DevOps Chat. This DevOps Chat we’re very happy to be joined by an old friend of DevOps.com, Hayden Lindsey, VP and distinguished engineer for DevOps and enterprise systems at IBM. Hayden, welcome.

    Hayden Lindsey:         Thanks a lot, Alan. Good to talk with you again.

    Alan Shimel:                Always a pleasure, Hayden. Hayden, we’ll get right down to it. Lots going on in the world of DevOps, but also lots going on over at enterprise systems in IBM, so let’s jump in. First of all, Hayden, you know – I know, I don’t know if our audience knows, but they will now, you are relatively hands on in going out there and speaking to real live people and hearing what their challenges are in, you know, everything, not just DevOps, but this whole kind of digital reinvention, digital transformation that companies are undergoing. In a bigger picture, how is DevOps helping them with that, and then specifically, how are you and your team at IBM helping them with this?

    Hayden Lindsey:         Yeah, so I do very much enjoy getting out from the lab. I mean, I like visiting the lab, love visiting the lab _____, but like getting out, meeting with clients. And so I do a good bit of travel around the world, meeting with them, either individually or at conferences, such as Edge, that we had in Vegas last week. And in general, I’m talking to them about how to modernize or transform, and of course, a lot of focus these days is on digital transformation. You know, just like some years ago, everything was about service orientation and so and all that.

    Now, I think intellectually, most clients understand that they need to modernize, and of course, the ones I’m talking to are clients that have mainframes, but every single one of those clients have distributed systems and they have – most of them now have some form of mobile app, and of course, they’ve got web apps, so they have all different types of platforms. Now, they may all intellectually understand that they need to do a better job of leveraging the investments they’ve made in their back end systems over the years, and so that we’re talking to them now about is, you know, hybrid cloud topology, API management or the API economy, which is, you know, the latest and greatest way of exposing systems for use by applications on other platforms and the other technologies. And the way DevOps fits, that’s how you go about doing it. If you want to be modifying systems and you need to do it at a rapid pace and yet in a low risk, high quality manner, DevOps is the answer. And now getting people to – you know, getting them from intellectual understanding to action, sometimes that’s the challenge, so that’s why we get out and talk to them.

    Alan Shimel:                Absolutely. So Hayden, big news, or at least in our world, and specifically in your division, was a couple of months ago, you guys announced an exciting acquisition. You want to share that with our audience?

    Hayden Lindsey:         I would love to, yes. Took quite a while to get this announcement out the door, but on June 24th of this year, we announced acquisition of a company called Easy Source. They – and I’ve been aware of them for quite some time, been familiar with the former CEO and all that for quite some time. They have a very strong offering in the area of what we call application discovery and understanding. And basically what it does is it looks at source code, looks at schedulers, looks at JCL, looks at all different sources of information on mainframe systems, but also distributed systems, and then helps you do what if analysis, impact analysis. It helps you understand – the reason we talk about understanding, it helps you understand these very large and complex systems by visualizing linkages, control flow, data flow, many things like that.

    So it’s a very, very powerful tool, supports IBM technologies, but many, many non-IBM technologies as well. So we view this as an essential first sort of step one, if you’re going to go about modernizing for purposes of a digital transformation or any other purpose, you need to understand what you have. And so we’re very excited to be able to offer this to our clients. And just by the way, it’s called Easy Source now, but as of October 4th, we will rebrand it IBM Application Discovery, and we’re combining that with some other solutions that we have around application delivery intelligence, or what we like to call Cognitive DevOps, to give a very comprehensive understanding and improvement offering.

    Alan Shimel:                Got it. So that triggered something with me, Hayden. You know, last year at InterConnect, cognitive was such an overriding, overarching theme of InterConnect, so much so that, you know, you guys are kind of breaking out – I don’t know if it’s called Watson World or whatever, but a conference just dedicated to Watson and I guess things cognitive. And now we’re using the term Cognitive DevOps. Why don’t you share with our audience, if you don’t mind, a little bit about what you mean by Cognitive DevOps?

    Hayden Lindsey:         Yeah, and in fact, from InterConnect last year, we introduced that term associated with a new offering that we had called IBM Application Delivery Intelligence, and I’ll explain in a minute what it’s about. But the more we started thinking about it, the more we realized DevOps is, of course, important, but how do you improve day after day, week after week, month after month, you know, after you have maybe automated all your tests, or you actually do have an automated delivery pipeline? How do you continue to improve? And that’s where we thought we would be able to look at different types of data that is owned by – owned or created by various tools in the overall development and delivery life cycle. And by combining these different sources of information, we believe, and I’ll give you an example, but we believe we can actually make recommendations to our clients about how to be much more efficient in what they’re doing to reduce risk, to improve quality, to improve productivity.

    So our MVP, and it really is an MVP, we’re trying to take that concept seriously, was focusing on testing, because in the mainframe space, but also in a lot of other spaces, there is still a lot of manual testing that goes on. And so that’s obviously – the DevOps parlance, that’s waste and that’s error prone, because it’s being done by humans and they’re gonna make mistakes and it’s slow. And so we combined the information around test coverage, code coverage, tied to test cases, and then mapped that to change sets that we can observe that you change, you know, these three lines of code from when you open it in the editor and from when you save it and check it back in to a source code management system. And then we can tell you of, let’s just say, 10,000 test cases, that you only need to run 12 of them. And, you know, this would save time if the tests were automated, but it saves phenomenal amount of time if you’re doing this manually. So that’s one example of the type of insight, you know, pulling information from a variety of sources, to help people do DevOps better. And so therefore, I claim that equals cognitive.

    Alan Shimel:                Absolutely. So Hayden, you kind of opened a little bit of a genie’s bottle with MVP. You know, one of the – I guess one of the issues, and you probably know this from running around and speaking to them, is this whole how do you get customers comfortable, right? Especially big companies that are doing, you know, once a year updates to their app, and say, hey, guess what, guys? We’re gonna do an MVP release, you know, weekly, or even monthly to start with. You know, how do you – I mean, that’s – you want to talk about a shock to the system. How are you getting – and especially we’re talking enterprise systems with mainframes and Z systems and so forth. How do you get them comfortable around that?

    Hayden Lindsey:         It takes time. [Laughs]

    Alan Shimel:                Has to be more than time, more than time, right?

    Hayden Lindsey:         What you need to do is you need to get small wins, and you need to gain confidence by getting small wins and progressing – if you think about what it takes to get a truly integrated, continuous integration, continuous delivery pipeline, it takes many, many, many different efforts. You have to do test automation, you need to do deployment and configuration automation, you need to have powerful tools, maybe for test case generation, on and on. There’s many things to do. You can’t do it all at once.

    So what we do is we actually offer DevOps assessments for our clients, to go in, talk to their SMEs, figure out where the low hanging fruit is, where are their biggest pain points, and then put together a roadmap to say, all right, we recommend you start here, and don’t – and start with something where there’s gonna be real value if they’re successful. But also, don’t try to get them changing six or eight things in their overall development life cycle at once, because in all likelihood, you change that many things at the same time, you’re gonna fail. So it’s about getting small wins, to help them become comfortable that what people are saying in the DevOps community, what we’re recommending to them, can actually be done even if they’ve been doing things a different way for the last 30 years, because that is, in fact, the reality. I like to tell people my biggest competitor, not to slam any other folks out there, but my biggest competitor is inertia. People are comfortable doing things the way they’ve been doing them, and we have to come up with an argument about why it really makes a lot of sense to change.

    Alan Shimel:                And you know what, Hayden, unfortunately, it’s almost a law of physics. The bigger the organization, the more the inertia.

    Hayden Lindsey:         Absolutely.

    Alan Shimel:                And the more it takes to push it down. And, you know, in doing that, I do think you’re right, it is small wins, it is little things, building up momentum, breaking down these silos, if you will, between DevOps and even QA and security and a bunch of other stuff.

    Hayden Lindsey:         Another thing, Alan, that I’ll just say is that, you know, it’s not just a vendor, like IBM or any other vendors out there saying it. There are many clients, and you hear these folks talk, for example, at Gene Kim’s DevOps enterprise summit, which is coming up here in San Francisco in a few weeks or a month or so. The stories are amazing, and so the DevOps success stories are not by any means limited to the born on the web companies. There are many, many enterprise companies, like Nationwide, like Fidelity Investments, many others that embraced this, even before we were using the term DevOps.

    And they’re very, very mature, but they are – they’re leading the way. But if you are in the insurance industry and you’re not at the forefront, and you say, oh, Nationwide is doing that? Maybe I better pay attention. So it’s good to have some role model companies out there that are also willing to share their story, so I think that’s great.

    Alan Shimel:                Absolutely. You know, speaking of share your story, I just wanted to mention, Hayden, as long as we have you on, that coming up on October 25th, later this month, we’re gonna have a great webinar with our friends at IBM, on breaking down the barriers or the silos between Dev and Ops. And I don’t know if you want to talk about a little bit, but – our friend Rosalind, and –

    Hayden Lindsey:         Rosalind, Rosalind is one of our distinguished engineer, and she and I have been working together for a number of years now. She and Dave Willoughby, who is from the cloud architecture and emerging technologies area, are gonna talk about this – you know, how – the types of things that you can do and we need to be doing. Instead of just using this term DevOps and thinking it’s, you know, more about automating that handoff from DevOps, but what about the Ops back to Dev, and how can we more effectively utilize systems management and monitoring information to then provide – so this is just another example, providing improved insight back to the development community, so that they can do things better.

    And it turns out this is actually an area that we are actively working on for that application discovery intelligence offering that I mentioned previously. You know, there is a wealth of information in the what you think of as the system administrator’s realm or the ops realm, that could be combined with development information to get some really interesting insight. So this is what we’re exploring. It’s also an area where we’re trying to find sponsor users in the client community to help us. You know, let them tell us, okay, I wish I could know or get some insight about this area, and then that can get us thinking about what type of data we would need in order to provide those types of recommendations. But so Rosalind and Dave will discuss some of their thinking around that area.

    Alan Shimel:                Got it. Lastly, you know what, Hayden, we’re probably way over time, I’m just gonna ask you one more question, if it’s okay. And that is we were talking, you know, offline before we got on about both of our schedules, spending way too much time on the road. What events are on your plate coming up, that people might catch you at or might be of interest?

    Hayden Lindsey:         Well, one that I have been to, and I spoke at the inaugural one, was this one I mentioned just a moment ago, Gene Kim’s DevOps summit. It’s here in San Francisco, which is where I live now, and so it’s extraordinarily convenient. [Laughs] And so that’s November 7th through the 9th, and I think it’s just amazing the stories that you hear there, so I intend to be there for the three days. And then I’m sure there will be others that will come up, but our big annual one is InterConnect, which this year, instead of being in February, is gonna be March 19th through the 23rd, yet again in Vegas. And there may be many more, but those are the two that are on the calendar at the moment.

    Alan Shimel:                Got it. I’m looking forward to it. I’m looking forward to both of those. I will be, of course, at Gene Kim’s event filming again and doing interviews, and we’re making some big plans for InterConnect this year, Hayden. I can’t announce them right now, but it’s gonna be a lot of fun. Anyway, Hayden, we’ve gone way over time, I apologize, but I want to thank you for joining us today on DevOps Chat. Continued success with the enterprise and DevOps team, or the DevOps and Enterprise team at IBM, and let’s have you on again soon and let’s catch up.

    Hayden Lindsey:         Sounds good, Alan. Take care, now.

    Alan Shimel:                Always a pleasure. Hayden Lindsey, VP and distinguished engineer, IBM DevOps Enterprise Systems. Thanks for being this episode’s guest on the DevOps Chat. This is Alan Shimel for DevOps.com, and see you again on our next DevOps Chat.

    — Alan Shimel

  • Datadog Monitoring and JIRA Integration

    Datadog Monitoring and JIRA Integration

    Operations uses monitoring to provide early detection of issues with systems/servers/networks. Among those monitoring options is Datadog, a software-as-a-service (SaaS) cloud service/application monitoring tool that allows Operations to be notified of issues with their applications in near-real time. Development teams, meanwhile, use JIRA to track tickets and manage overall agile development.

    But what if your product is the very thing that Operations manages? The sweet spot for DevOps is that location where the app is internally developed, and managed by Operations, because the entire stack is under IT management.

    That’s why Datadog and JIRA are now integrated.

    Datadog can monitor an infrastructure and application on a variety of levels at a variety of locations in the infrastructure stack, from disk availability to response times to memory usage. Datadog has the ability to monitor and emit alerts when things are abnormal or out of specified ranges, or they outright fail.

    JIRA is Atlassian’s Agile project management software, with issue tracking and all of the usual agile tools in one place.

    From Datadog Monitoring to JIRA ticket.
    Configuring Monitoring’s alerting to JIRA.

    Now, Datadog can create JIRA tickets, update them as more information becomes available and populate them with relevant information from the running systems.

    Say, for example, one cloud instance runs low on memory and passes over a threshold. Datadog detects the issue and opens a ticket in JIRA that shows what system and when, as well as whatever other information Operations has defined. That ticket is now in the system like any other, and a developer can grab it and look at memory use for the portions of the overall application that run on the instance in question.

    According to the press release, these are the three automations included in this integration:

    1. Creating JIRA tickets when performance issues are detected within Datadog.
    2. Populating JIRA tickets with pertinent information about identified issues.
    3. Updating existing JIRA tickets when new information surfaces.

    I’m an analytical person, so my first thought was, “How cool,” and my second was, “But what if it spams JIRA with non-development issues?” Since Operations configures what goes to JIRA, and Datadog supports alert rollups (multiple alerts in one or two notifications), it is possible to keep such spamming under control. An overzealous Operations staff initially might configure the reporting to send too much, but as experience grows, this should become a viable way to automate bug reporting. In particular, tagging of Datadog-generated events should show quickly what alerts are creating too many tickets, and give the team opportunity to change the way tickets are created—or, if it turns out they’re created because of a serious application-impacting error, raise the priority of the fix.

    Which brings us to the updating status part. In Datadog, operations can leave a comment on an issue with an “@jira-update” tag and JIRA will get notified of the comment. That means Operations can raise awareness of an issue that may seem mundane but has larger impact than is obvious, or add detail to a JIRA ticket if needed, all from within Datadog.

    It is interesting that Datadog is also integrated with BitBucket, another Atlassian product, so it is possible that opened tickets can be marked as “in test” when an integration build includes reference to the ticket. In short, BitBucket integration means that Datadog’s monitoring is aware of commits and checkouts, and Operations staff will be able to correlate that to issues that occur. Adding in JIRA integration means that Operations sees the commits, gets the alerts and updates JIRA. The wealth of information is such that you’ll want to use their various filtering techniques just to make certain you’re getting the important stuff.

    There are some areas that are intriguing to me, such as the impact of a particularly complex set of queries/reductions on a big data platform. I haven’t tried the Hadoop integration, but it appears that this agent would be able to communicate that a given reduction is the source of the problem to the JIRA interface. Even if these complimentary integrations aren’t there yet, you can feel it coming to where “This reduction caused systemwide issues” is what is generated in JIRA. That’s pretty cool.

    Overall, the integration of Datadog and JIRA is a DevOps dream come true, tying Operations closer into Development, and allowing Development changes to be tracked closer in Operations. After all, the point is to keep the systems running as close to peak efficiency as possible, and automate as much as can be. The issues spawning from Datadog alerts into JIRA tickets also pretty much guarantees increased need for development and operations to talk, as problems that crop up by the automated system are discussed and researched.

    If you’re dedicated to a cooperative environment for Dev and Ops—even if you aren’t calling it DevOps—the Datadog/JIRA integration is worth a look.

    — Don Macvittie

  • TMI: BigPanda Separates the Signal from the Noise

    TMI: BigPanda Separates the Signal from the Noise

    Organizations have struggled for years with the problem of too much information. IT professionals can only address so many issues at one time, but many are faced with an overwhelming volume of alerts to try and review and prioritize. BigPanda thinks it can solve that problem with its event correlation platform.

    If everything is an alert, then nothing is an alert. There has to be some way to separate issues and alerts that really matter so IT teams can prioritize efforts accordingly and address the concerns with the greatest business or security impact. At the same time, though, it’s crucial to minimize or eliminate both false positives and false negatives.

    I spoke with Assaf Resnick, CEO of BigPanda, about the company’s approach to event correlation and why he believes organizations need what his company has to offer. Resnick described the current state and trends in IT infrastructure and IT development, noting DevOps principles and tools have transformed the entire data landscape.

    Customers, he said, are asking themselves, “How do I move away from this monolithic application hierarchy that deploys once a quarter?” The goal, according to Resnick, is not just more reliable uptime, but delivering speed and agility at the same time.

    The BigPanda platform doesn’t try to reinvent the wheel, per se. In fact, it more or less relies on users having other wheels in place. BigPanda sits on top of what a company already has and pulls in data from other platforms such as New Relic or Splunk. It then normalizes and correlates that data so it can spit out results on a scale that mere mortals can comprehend. “Instead of 20,000 alerts from 15 different monitoring tools, I may only have 36 meaningful events,” Resnick said.

    Playtika is one customer taking advantage of the BigPanda solution. Playtika is a digital gaming company that develops social casino games. According to a recent case study, “With BigPanda, Playtika was able to centralize all of its fragmented monitoring tools into a single unified platform that provides instant insight into the health of their IT systems. Moreover, by automatically correlating alerts from multiple systems into high-level incidents, BigPanda enabled Playtika’s IT pros to cut through the noise and quickly detect problematic issues. And as Playtika has acquired companies in recent years, BigPanda has allowed them to consolidate the operations management of these newly acquired companies virtually overnight, with zero migration cost.”

    “We were struggling to work with many different tools in order to monitor the performance of our operations,” said Ella Vidra, director of IT Operations for Playtika. “Investigating alerts across our stack was tedious and time-consuming. We were at risk of missing critical errors that could impact production. BigPanda enables us to centralize our IT monitoring and quickly spot the issues that matter most.”

    The proof is in the proverbial pudding, of course. There have been plenty of solutions that promise to streamline or simplify logging and alerting. It seems as though we now have solutions that are designed to streamline and simplify the results from the solutions that streamline and simplify other logs and alerts. Ultimately, what IT departments need is something that brings attention to major or critical issues so they can be prioritized without ignoring smaller or less crucial alerts at the same time. BigPanda thinks it can be that solution that helps companies pull together input from all sources and boil it down to a prioritized, manageable list.

    — Tony Bradley

  • DevOps Chat: Gene Kim on The DevOps Handbook and DevOps Enterprise Summit

    DevOps Chat: Gene Kim on The DevOps Handbook and DevOps Enterprise Summit

    Today was a red letter day in the DevOps world. Besides DevOps Connect: CD Summit’s being held in Atlanta and Berlin today. It was also the day of the highly anticipated release of the DevOps Handbook. The book was over five years in the making. It contains over forty case studies of real organizations and their DevOps journeys. I had a chance to chat with Gene Kim about the DevOps Handbook today. The audio of which is below.

    The authors of the book are some of the leading lights in the DevOps world: Gene Kim, acclaimed author of the Phoenix Project, Jez Humble, author of Continuous Delivery and Lean Enterprise, Patrick Debois, who coined the term DevOps and the one and only John Willis, @batchagalupe. All four put a lot of time in reviewing each case study and picking from the many different examples the best ones for the book.

    The case studies are organized into examples of each of the three ways that were espoused in the Phoenix Project. So while the DevOps Handbook is a very different style book than the Phoenix Project, which was a novel, it continues common themes. According to Gene, the DevOps Handbook is a prescriptive guide showing patterns that repeat themselves across multiple examples.

    The book itself is available on Amazon (link above), as well as other book sellers and from the IT Revolution site itself.

    Gene also talks a bit about the upcoming DevOps Enterprise Summit in San Fransisco this November. He gives us his top three reasons why he is really excited about this years event.

    Below is the streaming audio of our conversation. We wanted to publish this ASAP, so are putting it up before the written transcript of our conversation is complete. We will append it to this article when it is complete. In the meantime enjoy the conversation and let us know what you think of the book!

    If you read up to here, you deserve a special bonus. Here is a link for a download of 100+ pages from the DevOps Handbook to get you started. Enjoy!

    Alan Shimel:                Hi, everyone. Alan Shimel – DevOps.com here for another DevOps Chat. This is a special edition DevOps Chat with none other than Gene Kim. Gene, welcome.

    Gene Kim:                   Hey, Alan. Always great to hang out with you. Thanks so much for having me on.

    Alan Shimel:                Thank you. Gene, you know a big day for Gene Kim, IT Revolution – John Willis, Jez Humble, Patrick Debois. The official release of the long and anticipated [laughter] or much anticipated book – The DevOps Handbook.

    Gene Kim:                   Yeah. Five and a half years in the making and hopefully people will stop making George R. R. Martin jokes to me. [Laughs]

    Alan Shimel:                Gotcha. Well, you know it’s funny. I’m here in Atlanta recording this at an event we were putting on and John Willis was the keynote and he said the same thing. He said, “Please, if you’re doing DevOps it shouldn’t take you five years to do the book, but it took us – but you’ll do your DevOps faster.” But you know what? Anything worthwhile is worth waiting for, Gene and certainly high, high expectations with this book 

    Gene Kim:                   You know, John and I have talked a lot in the last couple of weeks and one of the things we keep talking about is just how much we learned in the writing process. I mean it’s just a – even looking back I guess when I first met John six years ago and how much – we’re definitely kindred spirits and I’m also including Patrick Debois and Jez in this as well, but this conversation that John and I were having was as much as we thought we knew six years ago just even looking at how much we learned last three years is just amazing and I think we’re both kind of shaking our head at how much – almost the assumptions we had in our head to think that we could have been done by 2013, so I’m just so thrilled that so much that I’m learning made it into the book and I’m just so thrilled with the way the book turned out and I especially love the fact that it has 48 case studies and a lot of it came out of the DevOps Enterprise Summit in terms of how people are putting these principles and practices to work in large, complex organizations like Target, Nordstrom, Raytheon, Nationwide Insurance, Capital One and so forth, so I’m just thrilled to death.

    Alan Shimel:                Absolutely. And Gene, I don’t want to minimize 48 case studies and all that we’ve heard. It’s a tremendous body of work here, right? But what –

    Gene Kim:                   Yeah, yeah. Sorry.

    Alan Shimel:                No, no. I was going to say. It’s a tremendous body of work.

    Gene Kim:                   Yeah. So, [laughs] one of the things that we made as a bonus for people who pre-ordered the book was actually a lot of the dropped case studies – so in order to keep the book below 450 pages, we had to take out 20,000 words.

    Alan Shimel:                Uh-huh.

    Gene Kim:                   And so there was a lot of stuff that I actually was fighting tooth and nail to try to keep in the book, but eventually senior has prevailed and you know I definitely saw that you have to actually be able to carry the book

    [Laughter 

    Alan Shimel:                We might –

     

    [Crosstalk]

                                       

                                        We might see on late night TV sometime outtakes, bloopers and outtakes of The DevOps Handbook.

    Gene Kim:                   Yeah. Exactly. So we’ve actually made that available to some folks and – but then I think it really does represent what we thought were the most important parts and there’s a lot of stuff that we had to cut that I thought was also very useful and valuable. So it’s been an incredible journey putting this book together with my fellow colleagues.

    Alan Shimel:                Absolutely. So Gene, we may have some listeners out here who (a) haven’t read the book obviously. It’s just coming out today. But (b) maybe you’re not familiar in how this book really is a very different book than The Phoenix Project, right? I mean they’re totally different genres, right?

    Gene Kim:                   Yeah, yeah, yeah. That’s a good point. So, The Phoenix Project is of course a novel that was I think really intended win the hearts and minds of the technology community at large and really describe in vivid detail what does the DevOps problem look like? Which DevOps is really the solution for and one of the biggest comments that we got was, “Wow. I see this problem that I didn’t even know I had whether I’m dev, test, or operations or information security, but – and I totally want to replicate the outcomes that that amazing unicorn team did at the end of the book. How did they do that?

    And that’s what really The DevOps Handbook is designed to do is to be that non-fiction prescriptive guide to really inform those people going on the journey with hopefully – with what they need to know – certainly everything that we’ve learned in terms of what are the principles behind DevOps that allow organizations to have this fast flow of work from dev through tests through ops and the infosec, so we’re creating value to the customer while providing world class reliability security and stability whether we’re – regardless of where the value stream we’re in and so the book certainly expands upon the three ways that was first introduced in The Phoenix Project.

    The three ways being the principles of flow, feedback and continuous experimentation of learning, but we also talk more in the beginning about the principles from which you can derive all these patterns from and we have a big section around how to get started, how to organize those teams, what are the various adoption patterns whether – when dev is driven by ops, by chief architects or by developers. 

    Alan Shimel:                Uh-huh. It’s very interesting. I actually just had this conversation yesterday with a bunch of folks who are kind of – you know, there’s always doubters, Gene, right? So I had one guy telling me that DevOps will not truly succeed because it cannot succeed if led by developers. And I said, “Well, who said DevOps can’t be led by ops?” Right? Or –

    [Laughter]

                                        I mean we’re –

    Gene Kim:                   Exactly right. And those were the questions that we set out to answer by doing the DevOps Enterprise Summit, right? Is to really through experience reports really capture what those journeys look like – who it was led by, where they fit in the org charts.

    Alan Shimel:                Yep 

    Gene Kim:                   What the outcomes were and in some of the book we talk about the – where that source of energy can come from, what the outcomes and the transitions look like from those perspectives and what the implications are to how we design our organizations and I just love the fact that we’ve been actually now to document that in writing as opposed to buried in 50 videos.

    Alan Shimel:                Yes. Well, we – I don’t knock the videos. I do a lot of them. [Laughs]

    Gene Kim:                   No, I love the videos.

    Alan Shimel:                [Laughs] I’m only kidding. But Gene –

    Gene Kim:                   No, really. I love this book from The Danish Project. It actually came from Paul Muller, VP at HP.

    Alan Shimel:                Uh-huh.

    Gene Kim:                   And he said, “Messiah’s are great, but scripture is better.”

    Alan Shimel:                Uh-huh.

    Gene Kim:                   There’s still no better, more viral mechanism to spread knowledge than written word, so I love videos as much as you. And I depend upon them a lot in my own work, but in terms of the way that – ideas for the fastest – still through books.

    Alan Shimel:                Agreed. But you know what, Gene? An interesting – and I want to talk about the book, but just a quick aside – when we talk about the body of knowledge when we talk about the lessons that you’ve been able to glean from research, we would be bereft if we didn’t talk about the Puppet – the state of DevOps survey that you’ve done with Puppet Lab or Puppets – no longer Puppet Lab – with Puppet all these years.

    Gene Kim:                   Sure.

    Alan Shimel:                All of the DevOps Enterprise Summit transformation stories now going on into what? Our third year of that, right? The 48 case studies here in the book and frankly – I mean I know you. You’re out talking to people all year around, listening and anytime – your ears perk up when you hear someone talk about their story, so though these are 48 that make it and as you said some wound up on the cutting room floor, the body of knowledge of research that’s gone into where we are today is phenomenal. I mean it’s really some piece of work when you look back at it, no?    

    Gene Kim:                   Oh, yeah. Absolutely, man. I loved the way you framed it. I mean I think kind of the book that we wanted to write was one, we wanted to make sure it was grounded in sound principles, right? And so there we looked to – did the body of knowledge around lean and theory of constraints and plan operations and that goes back 50 years of knowledge – you know, go back 100 years and so that’s I would say the principles and then the primary research around the state of DevOps report. This is the work that I’ve done with Jez Humble, Dr. Nicole Forsgren and Puppet that spans 26,000 respondents over 4 years, so this I think helps us establish validity or confirm validity of these practices

    And then the third part is the experiential part of observation, right? The experience reports often told at conferences, including the DevOps Enterprise new Summit and I think maybe just to show a little bit of the method behind the madness. I mean so many of the talks at DevOps Enterprise are specifically – they were invited just because they were those experienced reports that we were looking for to help fill out The DevOps Handbook and that also attracted other likeminded thinkers, which has just been so great.

    Alan Shimel:                Yeah 

    Gene Kim:                   So, I think it really take all those approaches really to I think kind of the best – especially business books out there are those that kind of have that kind of quality to them.

    Alan Shimel:                Yeah. Gene, one other thing just on the book and then I want to turn over to devs for a moment and that is a lot of people thought we were going – it would be the DevOps Cookbook if you will where there would be a recipe of step-by-step, but that’s not really what the handbook is about, is it 

    Gene Kim:                   Yeah. In fact it will be – yeah. In The Phoenix Project there’s still – in the editions that are out there right now, they still have DevOps Cookbook in them and we renamed it to DevOps Handbook and I think the – I would say this book is definitely a prescriptive guide, but not necessarily saying that you have to do these things in sequential order, but it is still prescriptive in the fact that you have patterns, right? That we have each one of the three ways.

    There’s at least 100 plus pages on each principle on patterns that help fulfill flow, feedback and continuous experimentation learning, so by no means do you have to do them in order. You don’t have to do all of them, but I think what’s so great about patterns, it really does – anyone who has seen the transmission stories, whether they’re told that velocity, whether it’s Google, Amazon, Twitter or Facebook or at DevOps Enterprises, Target, Nordstrom, Nationwide Insurance, Capital One and Disney, right? You can’t help but observe the patterns, right? There is an emergent set of paths that these organizations take and so the goal is really to help to write a prescriptive path so that other organizations can achieve those same outcomes with hopefully less trial and error.

    Alan Shimel:                Sure. Sure. Gene, before we turn to devs, the book is generally available today. People can order it on Amazon. Can they order it directly from IT Revolution?

    Gene Kim:                   They can. In fact there’s a whole great – a bunch of great retailers that you can order the book from including Barnes and Noble, 800-CEO-Read, and you can buy it directly through IT Revolution and I think the reason why people would do that is right now it’s probably the cheapest way they can get the book internationally, but also there’s great retailers like Amazon UK, Book Seller. There’s a fabulous shop in the UK that actually sends books for free internationally and so that’s also a wonderful service 

    Alan Shimel:                Very cool. Very cool. All right Gene. We’re probably going to go over time. I’m sorry. I apologize, but let’s quickly talk – turn over and talk about the upcoming DevOps Enterprise Summit in San Francisco. It’s the third San Francisco summit – fourth DevOps Enterprise Summit if we count London, right? I’m excited. Of course I’ll be there. We’re going to do videos again and looking forward to hearing some perhaps new transformation stories and hearing – catch up some of the people we’ve heard from before, but give our audience a little – what are you looking forward to? What’s getting you jazzed about this event?

    Gene Kim:                   Oh, yeah. Yeah. I mean having coming off two years of DevOps Enterprise in the U.S. and we just did London a couple of months ago with DevOps Enterprise in London. I can still say this is going to be the best DevOps Enterprise yet. What excites me? I love all the conferences, [laughs] but I’m super excited for this one. Let me give you my top three

    One is we get to see a bunch of speakers that we’ve come to know and love and they’re coming back again and telling about their third year and so one of the neatest things that I just feel so honored and privileged is that we can chronicle the journey at Target, Nationwide Insurance, Capital One, Raytheon and their – CSG – all these organizations – we get to hear kind of – they’re going to summarize – here’s what your one and your two were like, which we’ve heard from before and then here’s the challenges that remain. Here’s changes and their responsibilities and here’s what they’re doing about it. And I just love that because yeah, I think most conferences you – I remember going to many conferences and you see the same person and sometimes they’re even giving the same talk and you’re like, “Why am I seeing the same person?” But in this case, right? It’s almost like a documentary, right? 

    Alan Shimel:                Yeah.

    Gene Kim:                   Where you got to see – follow them on their journey. So I think that’s pretty unusual in conferences and I hope everyone appreciates that as much as I do and I’m hoping that people will look forward to that. The second thing I’m really excited about is that we have more companies telling their journey that are – of which are very different than we’ve heard from before. Three come to mind for me. One is the adoption of DevOps inside of the Forge.mil program, so this is supporting the DoD across all of the armed services. And this involves classified systems.

    Alan Shimel:                Right.

    Gene Kim:                   And since this is a really neat story of like – a heroic story of how they were able to get DevOps outcome even though they never actually got to see the classified production systems.

    Alan Shimel:                [Laughs] It seems –

    Gene Kim:                   Astonishing 

    Alan Shimel:                Yeah. It’s kind of working in the dark.          

    Gene Kim:                   Yeah. I know. And still it doesn’t matter how much harder it is when you’re in the dark

    Alan Shimel:                Yep 

    Gene Kim:                   The other one is we have Susanna Brown and Benjamin Chan from American Airlines, so we’ve never had anyone from the aviation industry before, so this is – their area of responsibility include most of the non-ecommerce properties of flight operations, flight safety. All that stuff. I love that stuff. And we have the CTO from Hyatt speaking and so we’ve never had someone from the hospitality industry present before and so those are both keynote talks.

    Alan Shimel:                Fantastic.

    Gene Kim:                   I think that’s – yeah. The third think I’m looking forward to is we actually have two luminaries that I think almost anyone in the DevOps community holds in incredibly high esteem. One is John Allspaw from the famous Allspaw/Hammond presentation in 2009. He’s CTO at Etsy and John Willis, DevOps Handbook co-author. He’s going to be interviewing him about his journey now as a CTO of a publically traded company and one of the talks I’m also looking forward to is from Mark Imbriaco, so he is famous because he was the head of operations at 37 Signals 

    Alan Shimel:                Right.

    Gene Kim:                   So EMG’s Basecamp or Campfire, right? That’s him. Heroku, DigitalOcean, LivingSocial. Oh, he was the head of ops at GitHub for many years.

    Alan Shimel:                Yeah.

    Gene Kim:                   So in my mind these are some of the most storied, famous properties in our –

    Alan Shimel:                Well, no. He’s a great – he’s left a trail of success wherever he’s went with that stuff. It’s pretty interesting. One of the things you – you mentioned American Airlines – we had Dan Glass talk at our regular DevOps at RSA last year actually and they are – they’re doing the whole DevSecOps thing and I’m looking forward to that AA one. That ought to be good and AA I mean American Airlines. Yeah.

    Gene Kim:                   Absolutely.

    [Crosstalk]

    Alan Shimel:                Yeah. Good stuff there. And Gene, just – I think early bird pricing may have just ended maybe today or tomorrow. I don’t remember. I get your emails all the time on it, but people can still sign up, right?

     

    Gene Kim:                   Yes. Registration is still open and anyone who is interested in hearing about how large compass organizations are transforming and the amazing, heroic journey that they’re on. We would love to see you there.

     

    Alan Shimel:                Fantastic. And dates, Gene? Just maybe in case someone is wondering.

     

    Gene Kim:                   Yep. November 7th to 9th at the Union Square Hilton in San Francisco.

     

    Alan Shimel:                Absolutely. I will be there, so I encourage our listeners if possible to come by.

     

    Gene Kim:                   And get in line to be interviewed by the famous Alan Shimel, right?

     

    Alan Shimel:                Exactly. [Laughs] I will be doing video interviews there. So Gene, what are you doing with your free time?

     

    Gene Kim:                   [Laughs] Well, boy. You know, right now my planning horizon is about 15 minutes ahead of me. Everything – right now I’m spending a lot of time around the DevOps Enterprise Summit just getting ready. We have – I’m working with all the speakers in terms of making sure that those talks are as successful as they can be for both the prior community objectives as well as the attendees and we’re doing something new, which is – I think as adult learners we tend to learn less from theory and we learn more from experimental learning and that’s why we do – we share experience reports, but we’re actually – for that reason we’re kind of – we’re having facilitated discussion sessions.

     

    We’re calling them DevOps workshops around the top problem areas that have identified by the community, so that’s organizational design needing change, information security and compliance, minimal competency – essential competencies for our leadership and so those are small group sessions. Each one of them will have an assigned facilitator to really help maximize the learning within that group, so I’m super excited about that.

     

    So yeah, I guess the long and short, I’m working on helping make sure that DevOps Handbook, the people that are looking for it, that they know about it and that DevOps Enterprise is going to be successful and even more successful than prior years.

     

    Alan Shimel:                Fantastic. Well, Gene Kim. We’re way over time. I apologize, but it’s always a pleasure just catching up and just listening to everything you work on energizes me, so continued success with all of this, looking forward to the book. I think we’ll probably be doing something on DevOps.com with maybe a free chapter or something you know, that –

     

    Gene Kim:                   Oh, yeah. Oh, I want to say – you’re right. Thank you so much. Yeah. So, we will make available the first 140 pages of The DevOps Handbook. So if that’s of interest, we’ll have instructions on how to get it, so if you are so –

     

    [Crosstalk]

     

    Alan Shimel:                I’m putting it together. Yep. I’ll try to include it in with this article and audio file for people who want it. I have the background already, Gene. I’ve been working with your team and we’re on it. I’ll do my best.   

     

    Gene Kim:                   Yeah. Great. So that is almost a third of the book if you take out the index and the footnotes and the endnotes, so if you think you’re interested, I think the best way to find out for sure is actually to download the excerpt and see it for yourself and hopefully it will be of value to you.    

     

    Alan Shimel:                Okay. Gene Kim, principle author of The DevOps Handbook.         

     

    Gene Kim:                   No, no. Just one of the co-authors.

     

    Alan Shimel:                Just one of many? Okay. One of the co-authors of – we might as well mention them all – Gene Kim, Jez Humble, Patrick Debois and the one and only John Willis. So great job, Gene. I’m looking forward to the DevOps Enterprise Summit in November in San Francisco and to everything else, I want to thank you for – well, number one for getting me into DevOps as you know, but number two, just for all the work you do and the research and the leg work and running around and cheerleading and you know what? You’re a good man, Gene Kim.

     

    [Laughter]

     

    Gene Kim:                   And Alan, it’s always so fun to be on the spinner with you as well.

     

    Alan Shimel:                All right, then. I will be in touch. Congratulations again on the book.

     

    Gene Kim:                   Thank you so much and see you in a couple of weeks in San Francisco.

     

    Alan Shimel:                Absolutely. Gene Kim, author of DevOps – one of the authors of DevOps Handbook, The Phoenix Project, the man behind DevOps Enterprise Summit. Thanks for being our guest today on DevOps Chat. This is Alan Shimel for DevOps.com.

    — Alan Shimel

  • How Google’s Acquisition of Apigee Impacts Developers

    How Google’s Acquisition of Apigee Impacts Developers

    Google revealed a few weeks ago that it is acquiring Apigee for approximately $625 million. The deal demonstrates both the increasing demand for APIs (application programming interfaces) and Google’s desire to remain on the cutting edge of providing a platform for APIs. The purchase also underscores some important trends for developers.

    If nothing else, Google’s purchase of Apigee validates the importance of APIs for online commerce, as well as for businesses and developers in general. I had an opportunity to get some insight and feedback from Steve Davis, CTO of Four51, a provider of B2B ecommerce solutions, regarding the Apigee news.

    As DevOps and microservices continue to gain momentum, APIs play a very pivotal role. According to Davis, the API is the product for Four51, which makes it the foundation for the company’s DevOps strategies. “Most microservices are API-based as well, so the role is simply that they are the heart of these strategies. The concepts of DevOps relate to continuous delivery and all of the stages from development to deployment and operation. APIs allow developers the tools and visibility they need to work in this environment of delivery.”

    I asked Davis if he believes that it was a smart decision by Google to acquire Apigee at this time. “It was a wise move by Google to jump in, as the API economy is accelerating faster than other areas,” Davis answered. “The corporate customer appetite is increasing for APIs as companies continue their digital transformation, and API management is a critical discipline for security, scalabilty and reliability.”

    Davis pointed out that Google’s primary competitors—Microsoft and Amazon—have been out there with AWS and Azure cloud services for a while, and they’re doing quite well. He suggested that Google has some catching up to do, and that Apigee—as a best-of-breed independent platform—made a lot of sense for Google to target.

    For developers, Google’s acquisition of Apigee elevates Google as a cloud-hosting option—putting it on a more even footing with Microsoft and Amazon. The presence of an API management services enables developers to offload that tedious work and focus on more important things—such as continuing to innovate and bring value to their products.

    Davis also shared some thoughts on what the Apigee acquisition means for Four51. “Though we aren’t directly impacted by this acquisition, it does validate the path we’ve taken with our API-first eCommerce platform,” he said. “Many of the same enterprises looking to tap into the API economy are doing so to improve security, scalability and efficiency in their e-commerce and order management with their customers. We’re a big believer in the benefits of cloud infrastructure, as we recently moved hosting of our OrderCloud.io platform to Azure, and we’re utilizing the services there.”

    DevOps and microservices are all about streamlining productivity and operating as efficiently as possible. APIs play a major role in that effort, because they allow much of the heavy lifting on the back end to be automated. The acquisition of Apigee makes Google more competitive with its primary rivals, and highlights just how important APIs are—and will be for the foreseeable future.

    — Tony Bradley

  • CD Summit/Jenkins Days with John Willis, KK & Mustafa Kapadia

    CD Summit/Jenkins Days with John Willis, KK & Mustafa Kapadia

    JohnWillis1Our fall lineup of DevOps Connect: CD Summit/Jenkins Days are coming on fast. We kick off this fall’s schedule of seven events in Atlanta on Oct. 4 and 6. The speakers and subjects are up on the site, but I wanted to highlight the three keynote speakers: Our three headliners for Atlanta are Mustafa Kapadia of IBM, Kohsuke Kawaguchi (KK), the founder of Jenkins and CTO of CloudBees and, finally, the one and only John Willis (@botchagalupe). Each one of these speakers would be a great listen by themselves, but getting all three in one morning should be great!

    But wait, there’s more. In addition to Mustafa, KK and John, we have a full day of kohsuke-kawaguchiDevOps learning in store. The afternoon features three separate tracks with four or five sessions per track. On track is solely on Jenkins, one is DevOps in general and one is on automation and interoperability. Some of the other speakers include Ben Bomhoff from Security First Insurance and Daryl Bower from Accenture. Check out the website to get more information.

    Mustafa KapadiaAll of the above speakers are on the second day of the event, Oct. 6. The first day, Oct. 5, features several great workshops on Jenkins, Blue Mix and a DevOps Foundation workshop by the DevOps Institute.

    You can check out the full line up of speakers, schedules and topics at the DevOps Connect website. The cost without workshops for the day conference is just $99 and includes lunch, refreshments and access. If you would like to attend the first day of workshops as well it is just another $99. If you are in the Atlanta area, this is a great day of DevOps!

    If you are not in Atlanta, be sure to check out http://www.devopsconnect.com for the full schedule for the fall. Other cities we are visiting include:

    • Amsterdam
    • Berlin
    • Denver
    • Austin
    • Seattle
    • Los Angeles

    If you are in any of these cities, please come spend a day or two with us. Also be sure to stay on the lookout for additional conferences we are planning including DevSecOps events at RSA and InfoSec World 2017.

    — Alan Shimel

  • Q&A: BDO’s Coffman on Change Management, Security and DevOps, Part 2

    Q&A: BDO’s Coffman on Change Management, Security and DevOps, Part 2

    As we covered in part one, DevOps’ best asset and biggest risk factor may be one and the same: swift and precious change. In part two, we continue our discussion of the challenges and solutions with Deena Coffman, managing director of BDO Consulting’s Technology Advisory Services practice.

    David Geer: What specific tools and technologies can help DevOps teams to address change management in these ways?

    Coffman: There are a variety of tools and technologies to address some of the inefficiencies in change management, allowing DevOps teams to automate most of the process. For example, teams can leverage change management software to build pre-authorization processes and workflows to speed up change management approvals. On the security side, infrastructure monitoring tools will alert an organization when a discrepancy is discovered on the network, providing application visibility to the users and monitoring for unauthorized changes. DevOps teams should also consider using a collaboration platform like Slack to improve communication via virtual meeting places.

    Geer: What will these tools accomplish and how should teams be using them?

    Coffman: These tools are only useful if used properly. Deployments that require change management approval can go through the automated workflows. If a change requires approval, developers can use Slack or any other collaboration platform to contact the required party and explain the reasoning behind the modification. DevOps teams must also take the time to identify and automate as many “standard” changes as possible to reduce the reliance on change control meetings to speed up their deployments.

    Additionally, DevOps teams need to consider using an infrastructure monitoring tool to ensure all changes to the production environment are known and authorized. If a change finds its way to production without getting the appropriate authorization, the monitoring tools can detect it and report it to the team so they can rectify the issue.

    Geer: What kinds of things do DevOps teams need to do to address security to the satisfaction of the business?

    Coffman: The development team and security do not need to be at odds. By working in unison, both sides can reach their objectives.

    The InfoSec team would help the development team by making efforts to reduce the latency of security projects. The development team would help not only the InfoSec team—but potentially the entire organization—by preventing an expensive and embarrassing security incident by diligently supporting tools that automate security checks.

    Additionally, the build process needs to run unit testing, integration testing, functional testing, regression testing and information security testing before the code reaches QA. Security testing is often insufficient or not performed at all. DevOps teams need to understand that security testing performs an important function of catching issues early so as to avoid expensive, embarrassing issues downstream.

    Geer: What specific tools and technologies can help DevOps teams to address security in these ways?

    Coffman: Several tools are available to help DevOps teams address security in many different capacities, including setting up continuous integration/continuous delivery processes, automating server configuration, automating UI testing and functional security tests, testing the security requirements of applications and automating the vulnerability scan of applications and infrastructure.

    Geer: What will these tools accomplish and how should teams be using them?

    Coffman: DevOps teams should integrate these tools as part of their continuous delivery process. As soon as a developer checks his code in, the continuous delivery system should run the functional security tests, non-functional security tests and the security scan to ensure that the developer did not introduce a security vulnerability to the application and the infrastructure. These tools are only as powerful as the test scenarios they run. Therefore, teams should write exhaustive test cases to ensure the security of the application.

    Geer: How can teams do all these things and still maintain all the benefits and expectations that people have of DevOps itself?

    Coffman: By deliberately working toward improved collaboration and communication, teams can take advantage of the integration and efficiency benefits of a DevOps structure without sacrificing security or quality. Technical knowledge is important, without a doubt. To be exceptional, though, teams need equally strong teamwork and communication skills to actively support all important functions in the organization.

    — David Geer

  • The DevOps.com Containerization and Microservices Survival Guide

    The DevOps.com Containerization and Microservices Survival Guide

    Containerization and microservices seemed to have leaped out of nowhere and smack into the center of the enterprise IT scene. But neither leaped from nowhere. Both containerization and microservices have been part of the fabric of enterprise tech for some time—within Linux for containers and service-oriented architecture for microservices.

    Largely, thanks to containerization, both now are moving quickly to become central components of enterprise tech. Containers in recent years—and especially in recent months—have moved from tools used in development environments to production. And the ability to manage containers as discrete services is enabling enterprises to be able to respond quickly and be agile to technology demands.

    According to a recent survey from app performance vendor NGINX, of roughly 1,800 IT professionals, while container and microservices software adoption is limited it’s use is accelerating. The survey found that only 20 percent of respondents said they are using containers in production, while about one-third are doing the same for microservices.

    That survey, not surprisingly, indicated that microservices and container use will continue to accelerate, as roughly two-thirds of respondents said that they are currently evaluating containers and 70 percent microservices.

    Microservices are workloads that perform a single task and they are not dependent on the infrastructure or other applications for their ability to complete their designated task or “service.” Microservices are very effective at providing enterprises the ability to rapidly build software that solves business problems. They also make it possible for development teams to scale their efforts and build a more resilient architecture.

    With all of that in mind, and considering the nature of the inflection point most enterprises find themselves in now, we’ve assembled this microservices and containerization survival guide. The guide consists of some of the most important storage on microservices and container management that we’ve ran in the past year.

    DevOps Teams: Containers Will Help You Become Future Ready

    According to The State of Containers 2015 Docker Adoption Survey, more than 70 percent of respondents are either using Docker or evaluating for future use. Given the overwhelming popularity of containers, this post provides three trends that DevOps teams must be prepared for.

    7 Web Interfaces for Container Management and Orchestration

    Managing containers used to be a command-line affair. But a rich and growing ecosystem of web-based interfaces for containers has now emerged. This post provides an overview of the different web UIs available for container management and orchestration.

    ‘Containers-as-a-Service’ Is a Hot Trend

    Containers are still sort of the new kid on the IT block, but already containers-as-a-service is emerging as a thing. Here’s what you need to know.

    A Closer Look at Container Monitoring

    Enterprises have to ensure that all of their microservices perform well and provide their services as designed. Beyond that, how much do they really need to know about what’s going on inside their containers? The answer to this question varies. In this post Gerald Haydtner provides an overview for container-monitoring options.

    Containers: Not One to Rule Them All

    When it comes to the container platform ecosystem, not all container platforms are created equal. This post will help you sort through the options.

    Building Microservices: A Few Considerations

    An experienced practitioner details the top five design considerations when building microservices.

    5 Docker Best Practices You Should Follow

    Support for Docker has been integrated into many products and platforms and a lot of organizations are either already using Docker containers or trying to understand how to do so themselves. Our Tony Bradley provides the Docker best practices one needs to know to succeed.

    How to Choose a Cloud Platform for Containers

    Containerization has broadened from one or two vendors to a broader, more container-agnostic world and a drive to better scalability, redundancy, interoperability and security. Amazon, Google and Microsoft each have their own unique approach to addressing these cloud container concerns as they battle each other for supremacy. Here’s a look at how to evaluate each.

    Container Security: Why Bother?

    Are containers secure? Probably not. But knowing each approach’s strengths and risks—and available options—could help ease adoption.

    Container Security Challenges: Are Container Scanners Enough?

    How secure are containers? Not very, according to open source company Red Hat, which has published a blog post blasting the container ecosystem for lacking good security processes. Here’s a synopsis of what Red Hat has to say about container security, and some additional perspective.

    For Container Security, Automation Is Key

    Regardless of the IT infrastructure element in question, if it connects to a network—even through proxies or human intermediaries—eventually someone will find a security flaw and exploit it. Compatibility issues with new applications or infrastructure elements will emerge. Because of these reasons, all IT infrastructure elements need to be managed, without exception.

    Containers and Security Q&A: Putting a Lid on Risk

    How do containers affect security? And how can IT teams do everything they need to do in order to ensure the containers in use are as secure as they reasonably can be?

    — George V. Hulme

  • Industry Alignment: Supporting Real-World DevOps Workflows

    Industry Alignment: Supporting Real-World DevOps Workflows

    In our ongoing work to advance the interests of customers around adoption of Agile, DevOps and continuous delivery, the theme of evolving workflows to better suit human-scale work cycles often comes to the fore.

    While  it’s easy to say that build, test, release and monitoring capacities should feed into each other to promote continuous collaboration, realizing this vision in the form of integrated tooling is still no trivial feat. And it is further exacerbated by the reality that DevOps teams typically utilize a wide range of products built by a diverse set of providers, from larger players such as CA Technologies to open-source projects.

    If the goal is to drive  application lifecycle processes and supporting technology to better support human work cycles—along with accelerating organizational velocity and improving quality—somewhere in this domain of tooling integration there’s a need for relevant vendors to participate.

    That work officially begins today.

    At the Jenkins World 2016 conference this week, CA, along with a dozen other leading DevOps solutions providers and the DevOps Institute, announced the DevOps Express consortium. Joining us, along with CloudBees and Sonatype, which spearheaded this effort, are Atlassian, BlazeMeter, Chef, GitHub, Infostretch, JFrog, Puppet, Sauce Labs, SOASTA and SonarSource.

    So what’s the big deal? The direction of this initiative—which will focus first on supplying reference architectures that describe field-tested integrations of best-in-breed solutions—strikes directly at the heart of the central DevOps tenet: collaboration.

    The immediate result is that these providers are moving to boost interoperability across a typical production DevOps toolchain. It is truly noteworthy that these solutions providers, some of whom maintain palpable competitive interests, are putting aside those considerations to advance the interests of end users; that is a very DevOps phenomenon.

    In many other areas of IT, this type of cross-pollination either never commences or doesn’t emerge until the sector is so well-established that industry collaboration seems more of an effort to fend off perceptions of commoditization. To anyone familiar with the incredible pace of innovation ongoing in the automation sector, and the potential for competition between many of the companies involved in DevOps Express, it should be immediately obvious that this is a partnership that truly puts customers first.

    Yet, if we’re serious about empowering the kind of wholesale software factory reinvention that we espouse around DevOps, this is something that had to take place. How can an industry talk endlessly about eliminating stovepipes and marrying long disparate processes, and then defer to a walled garden worldview as it relates to the enabling products? That certainly wouldn’t really add up.

    It’s amazing to see what’s going on in the world of technology these days, and ever more exciting to be a part of it. We’re proud to be a member of DevOps Express.

    It’s a great time to be involved in this space.

    — Jeff Scheaffer