devopsy.kinsta.cloud

Category: Features

  • Enthusiasm for DevOps Greater Than Actual Adoption, Survey Shows

    Enthusiasm for DevOps Greater Than Actual Adoption, Survey Shows

    When it comes to DevOps practices, adoption continues to lag intention by quite a bit.

    GitLab recently surveyed some 5,300 software developers and engineers from mostly small and medium businesses around the world. The survey examined overall attitudes and perceptions around the state of DevOps, with a particular focus on uncovering any disparities that might exist between developers and IT management.

    The responses showed that many organizations are sold on the idea of DevOps, especially those who have implemented it. Sixty-five percent of the developers polled felt that using a DevOps workflow would save development time. An even higher 81 percent of managers shared that same sentiment.

    Organizations with an integrated DevOps strategy reported a high correlation between DevOps practices and overall team productivity. Development teams that deploy code on demand and spent more than half their time on new projects, for instance, had twice the rate of DevOps adoption compared to lower-performing teams. Higher-performing teams also generally considered automation of the software development life cycle a top priority and had access to better tools.

    Yet, the number of organizations in the survey that have actually implemented DevOps practices was far smaller than the otherwise enthusiastic support for it would suggest. In fact, only 23 percent of the respondents in the survey identified their team’s methodology as following a pure DevOps model. Thirty-five percent described their DevOps culture as being somewhat established.

    The low adoption rates are somewhat at odds with Forrester Research’s prediction about 2018 being the “Year of Enterprise DevOps” and may reflect the global nature of the survey base and the fact that many respondents were SMBs. Forrester found DevOps adoption to be around 50 percent when it conducted a survey of its own of 237 DevOps pros last year.

    GitLab’s survey found that while developers view DevOps as critical to software development life cycle, many have not implemented a DevOps workflow. Similarly, while over 95 percent of the survey respondents placed a high value on collaboration and 81 percent felt that such collaboration was easy to achieve, 25 percent said they did not have enough visibility into operations, security and production. And 42 percent reported unclear directions as one of their biggest challenges to getting work done.

    Culture continues to be a big roadblock. “It’s hard to get Dev and Ops to work together in a cross-functional way,” said Mark Pundsack, head of product at GitLab. “That challenge can be exacerbated by tooling. Disjointed toolchains lead to disconnected processes and barriers to communication that get in the way of working together.” Many developers in the survey reported using at least five tools to get their job done.

    Nearly 6 in 10 of the survey respondents (58 percent) pointed to ingrained practices as their biggest roadblock to adopting new tools and practices, while 50 percent blamed an inherent resistance to change.

    Interestingly, remote employees tended to have a more favorable outlook on how things were going with DevOps, with 67 percent saying they had good visibility into what others in the organization were doing compared to 57 percent of on-premises teams.

    “DevOps culture is clearly important to achieving high performance,” Pundsack said. “For managers … the survey identified the top challenge in getting work done was unclear direction and managers should be able to play a role in fixing that.”

    The apparent gap that exists between intent and adoption was not the only surprise in the survey, Pundsack said. Also unexpected was the extent of the discrepancy between how executives think their teams are operating, and what developers are actually saying. When asked how often they deployed code, 47 percent of managers were under the impression their organizations were deploying multiple times a day in contrast to the substantially smaller 39 percent of developers who said the same thing.

    That one area aside, though, developers and IT manager seemed better aligned on other issues in the Gitab survey. For example, more than three-quarters of both developers and managers feel they are set up to succeed and about one-third of them felt that they deadlines they have to deal with are fair.

    GitLab’s survey showed that most organizations expect to invest in continuous integration and continuous deployment tools in 2018, with nearly half strongly agreeing that CI can help speed up the development process.

    “CI can help alleviate blockers in the development cycle such as removing the manual process to verify code before deploying,” thereby allowing development teams to have a faster and more effective feedback cycle, Pundsack said.

    — Jai Vijayan

  • DevOps Chat: Concurrent DevOps with GitLab CEO Sid Sijbrandij

    DevOps Chat: Concurrent DevOps with GitLab CEO Sid Sijbrandij

    In this DevOps Chat we speak with GitLab CEO Sid Sijbrandij. Sid has guided GitLab as a co-founder to the heights of the DevOps market. Joining Sid and me is Ashish Kuthiala, who recently joined GitLab. I have always been impressed with the completeness of GitLabs’ vision of the DevOps toolchain and what the company wants to offer. Sid’s vision for this is something he calls Concurrent DevOps. You will hear more about Concurrent DevOps in our chat, as well as in the months to come.

    Another thing about GitLab is is an open source-based company, which allows for a lot of transparency. With hundreds of thousands of organizations using GitLab and thousands of paying customers, GitLab is a great example of a DevOps success story.

    As usual, the audio of our conversation is immediately below, followed by a transcript of our conversation.

    Audio

    Transcript

    [su_pullquote align=”right”] [su_custom_gallery source=”media: 120255″ link=”post” target=”blank” width=”200″ height=”200″ title=”never”]Sid Sijbrandij
    Sid saw the first ruby code in 2007 and loved it so much he learned to program and made this his profession. Before that he was engaged in various businesses, from selling programmable infrared receivers to starting a recreational submarine company. He is always looking to publicly document an answer or to make a process more efficient. He was never good at sports but tries to make it up in quantity by hiking, sailing, skiing, snowboarding, speed skating, dancing Zouk, squashing, and running. In love participating in conversations on Hacker News, and reading Marc Andreessen’s tweets (now that Marc stopped this greatly increased his productivity).[/su_pullquote]    Alan Shimel: Hey, everyone, it’s Alan Shimel, and it’s time for another DevOps Chat. Our DevOps Chat episode today is gonna be all about GitLab, and we’re joined by two GitLab folks. First joining us is the co-founder and CEO of GitLab, Sid Sijbrandij, and I hope I did that justice. Sid, welcome to DevOps Chat.

    Sid Sijbrandij: Thanks for having me.

    Shimel: Thank you. And joining Sid and I is a friend of mine, recently joined the GitLab team, Ashish Kuthiala. Ashish, I didn’t mess that up too bad, did I?

    Ashish Kuthiala: No, you didn’t. No, that was perfect.

    Shimel: Alright! Well, one of the nice things about tech is, I get a lot of experience. Not a lot of Smiths in the group. [Laughter] And as I said, this is Alan Shimel, and we’re here with DevOps Chat.

    So, guys—GitLab. You know, first of all, Sid, I’m sure you’ve probably heard this before, but I’m sure there are people in our audience who still have to ask themselves, “GitHub, GitLab,” right? And as silly as it sounds, they may confuse them sometimes.

    So, let’s set it straight in terms of GitLab. Sid, you co-founded it. What exactly is GitLab and how does it differ, let’s say, from GitHub, and we’ll go from there.

    Sijbrandij: Yeah, so GitLab started off as an open source alternative to GitHub, where you could have Git repositories, do version control, do some issue management, and it’s now grown a lot as a company and as a project. We now do the whole DevOps life cycle. So, anything from planning to rolling out what you’ve made—CI/CD, packaging, monitoring—all in a single application.

    Shimel: Great. And I think you’re being modest, actually. GitHub is certainly—when you say branched out, it’s branched out, I mean, from what I know, and I’m no expert, obviously, the breadth of the vision is pretty dynamic. So, congratulations with that.

    Sijbrandij: Thank you.

    Shimel: Ashish, you were—thank you. Ashish, you recently joined GitLab, and why don’t you give us out, what’s your position there?

    Kuthiala: So, Alan, I think you and I have known each other for a while, and I am very passionate about the promise of DevOps, especially at large scale enterprises, right? Not smaller teams or not those who had the luxury of building out their infrastructure and teams to adopt DevOps and realize the promises of that.

    We still a lot of the larger enterprises struggle with adopting and realizing the promise of DevOps as it is. I mean, a lot of them are achieving results, but a lot of them are still in the journey and trying to figure out how to get there.

    So, and I had heard about GitLab, I knew that they were building out a solution for this, but I think it was in the last DevOps Enterprise Summit, you, I, and Haydn ran into each other. And I started talking to Haydn and I learned more about the company, and I think it’s just the right product at the right time. Because if you look at all the challenges that exist in the big enterprise, there’s massive transformation needed, and there’s several things that prevent this from moving forward at the speed that everybody would like, whether this is different organizations within the large organizations to get them to collaborate together, whether this is stitching together the different toolset chains to kind of make this seamless.

    And I think GitLab has really built something from ground up that addresses this problem. And the good news is that the velocity at which we are adding functionality, we are adding the needs that need to be addressed. Because we are an open source company, we are completely open and transparent, right? We gather input from everybody, a road map is opened. We take the input, we take contributions, and we build the right thing for solving today’s problems. And as needs change, I see this model as something that can really adopt to that, right?

    Because the world of DevOps is changing very rapidly, and often there’s a fear of getting locked in with a vendor, too. With our solution, I think the openness of it and the direction it can go to solve problems as they change is a great promise, and we’re already solving a lot of problems. So, that was a big thing for me, you know, that I considered in joining GitLab.

    And the other thing is the culture and open transparency that Sid and the company have built in the company. It’s just phenomenal for me, right? I mean, that was just an eye opening model of how everything we do is open to the world. It’s open within the employees. Anybody can contribute, anybody can look at everything we do and how we move forward to help our customers.

    So, these two are the main things for me that really inspired me to join GitLab.

    Shimel: Great. So, Sid, I’d like to come back to you. Certainly, we can all have vision, but execution is what kinda separates winners from losers in the market. And I’d like to ask you, as the CEO, what’s the secret, if you want to share it with us, right? How do you translate this vision that attracts someone of Ashish’s quality and character and then translate that into an execution that actually translates to success in the market?

    Sijbrandij: Yeah, I’d love to talk about that. We’re biting off a lot, so I think we’re now—GitLab offers a great solution for 10 different product categories. So, that’s massive. There’s a lot of engineering effort. For example, we have a CI/CD team that is just 10 people, and that they have to compete with people that are 100 times bigger. So, how do they do that? First, they’re just selected. GitLab CI is the leader in this space. There’s GitLab CI on the top right and then whole lot of nothing.

    Shimel: Mm-hmm.

    Sijbrandij: So, how do you do that with 10 people? Well, there’s a couple of secrets. The first one is—you don’t do that with 10 people, you do that with 1,800 people. So, we’ve got over 100,000 organizations using GitLab, and over 1,800 people contributed to the product. So, in the release that will go out in a week from now, there’s not just 1 contribution, there’s not 10, there’s 93 features that people contributed that aren’t paid by GitLab.

    So, we see it as, instead of having a tool chain where you have to make the integrations yourself—and that’s not just a few, we identified over 24 integrations you have to make in a standard DevOps tool chain. Instead of making them all yourself with your DevOps tooling team, you work together with 100,000 organizations and 1,800 people to do that.

    So, that’s the big secret. It’s open source. We work together. Then there’s some small secrets, and one of them is iteration. Iteration is one of our six values, and it’s something that everyone is super excited about when they join, but they all kind of don’t understand how painful it is, because it’s very, very painful to do the minimal thing. We’re now in a call with Ashish that Ashish will probably not like that we’re gonna probably change the header on our home page today, because we thought of something better. He’s probably gonna say no, but give me a month, I can think of something even better. And I will be like, “That’s great, Ashish. Think of something even better and then change it again.”

    Shimel: Yep.

    Sijbrandij: It’s very, very painful, and we do that throughout the company, and in product, it’s a very painful thing. My previous call was with Martin, and they’re working on a cloud native chart for GitLab, and we have a thing with uploads and we probably are gonna ship a chart with an NFS server as a part of that. A very clumsy thing, but it unlocks everybody else, and we can keep shipping. And one of our other subvalues is boring solutions, and I love that, because it’s so obvious. You say “boring solution” and people understand what you mean. We don’t go for the shiny toy. Our mission is, everyone can contribute, and people can only contribute if there’s a stable environment to do that, if they understand what you’re doing.

    So, when I saw the GitLab code base a year after Dmitri created it, I expected to see a hobby project. Someone super passionate, but doing this outside of work, like, putting some new development paradigms in it and having something that is hard to comprehend, especially since 300 people already contributed in that first year.

    And instead, I opened up the code base, and I saw a pristine Ruby on Rails application. Totally idiomatic, like, exactly how it should be, and we kept that. We don’t allow anyone to do anything that isn’t boring, and that enables everyone to contribute. That enables us to make forward progress, and we adopt the best of what’s out there. We embrace Kubernetes, we embrace Prometheus—really great, rock solid open source solutions that we can build for later on, and that’s been a big success.

    Shimel: Absolutely. You know, Sid, that was a mouthful in more ways than one, right? So, let’s hit on a couple things. First of all, harnessing the power of open source, right? Right out of The Cathedral and the Bazaar, you know, the book, and what’s gone on is, I mean, the false multiplier there is amazing. Of course, there’s a certain amount of a herding cats element to it, right, in getting everyone moving the same way. But if you can, harnessing that power is just—you know, it’s amazing.

    No. 2, you know, you spoke about iterations and sort of feedback loops and constant change, right? This is core to DevOps, right? This is DevOps 101 kinda stuff.

    Sijbrandij: It really is. This is exactly what we deliver to customers.

    Shimel: Yeah.

    Sijbrandij: Organizations adopt GitLab because we can make their DevOps life cycle three times faster, but you still have to work between the tools. They had these handles between the tools, they had these integrations that weren’t transparent, that weren’t seamless.

    With GitLab, they can work concurrently. As soon as you push your change, QA is gonna run, security tests run, production is gonna have the first review up out there. There’s not a sequence to it. It is, you paralyze the whole process and you gain tremendous speed and you reduce your time to value, and you allow to get feedback and to improve based on that. That’s the value we bring as a product and of course, we try to practice that ourselves.

    Shimel: Yeah. Well, so, but you more than practice it yourself. I think, you know, Sid, I’m a little different. You’re the CEO of this great company, GitLab. I’m an observer, I’m a fly on the wall of many DevOps companies, of many DevOps transformations, and I will tell you, the DevOps tools providers who do it best are those who eat their own dog food, who are practicing DevOps themselves. If you can’t do DevOps internally, don’t try to tell other people how they should do it or, you know, what they should be doing.

    It’s not a case of “do as I say, not as I do.” You have to live that, you have to talk the talk, walk the walk in order for you to truly understand it well enough to get other people to buy into it and be successful. So, kudos to GitLab for doing that.

    You guys recently announced an acquisition in the securities space. We’re kinda running low on time, but I wanted to just touch on that. Sid, your take on it?

    Sijbrandij: Yeah, we acquired a company called Gemnasium, and I think they’ve done the best job of any company of assessing all the different libraries, all the dependencies that are out there and saying which ones are vulnerable, which ones contain a problem. And that is functionality that’s gonna, that’s part of the DevOps life cycle, so we want them to be right in the product.

    So, we’re really happy to join their team, and I’m super proud of them for embracing our values, because they joined us, and within three weeks, all the Gemnasium functionality ended up in GitLab, so.

    Shimel: That’s cool.

    Sijbrandij: We know we hired the right people, they’re amazing, so congrats to Philippe and his team.

    Shimel: Yep. Congratulations to them. And look, we see it here, you know, this whole DevSecOps thing, if you will, if you want to call it that, is probably, along with testing, two of the hottest areas within the larger DevOps community.

    Ashish, we’re almost out of time. Let me come back to you, though. So, you joined this fantastic organization. We’ve heard from Sid a little bit of a peek into what’s making this engine roar, this rocket lift off. What are you gonna bring to the table? What are we gonna see from you, Ashish?

    Kuthiala: I think we need to take this message out to the marketplace, like, you know, a couple of things you talked about when people hear GitLab, they sometimes confuse it with the Git community and other Git offerings like GitHub, for example. So, we really need to tell the world what we are doing, and we are much more than just a repository engine, right? We do a lot more.

    And now that, you know, we are out there trying to help other customers with their DevOps challenges, it’s actually a little bit more than what Sid mentioned here; he is being humble. So,  not only are we practicing our own DevOps methodology to develop and roll out our application, but we actually use DevOps in every single aspect of running the company, right? Right from the hiring process when I joined to everything that I do in the company adopts a pipeline—you know, you create an issue and you run it through the pipeline and you merge that issue and it goes to production. If something is wrong, you know, it comes back.

    Amazingly, everything, between signing for benefits to updating the webpages to running our tracking of issues of what we do, it’s all run through pipelines. And I think I see that as a future. You know, DevOps is not just to be applied to delivering IT value, but you know, it is a methodology that can be applied across running your company for all functions.

    So, I think the version of GitLab is much bigger than just helping companies deliver efficient software really fast out there with feedback on the DevOps methodology. It’s almost to pivot and change the way we think about delivering value to our customers, in every aspect of running the company that we do. And I want to be part of that. I think it’s really exciting. I can’t wait to kind of start talking about this to everybody and, you know, seeing how we go down this journey.

    Shimel: Great. Sid, we’re over time, but I want to give you the last word, so I’m gonna ask one last question for you. What can we expect to see next from GitLab?

    Sijbrandij: We detail a really ambitious vision, and the vision is called Complete DevOps. And we said we’re gonna realize that this year, and we’re executing on that. We’re really proud, we integrated a complete web IDE in GitLab, so VS code running in GitLab. No need to check out this autocode base, you can just edit right there and then.

    But it consists of a lot of parts. We’re gonna have real time editing of issue and merge request descriptions. We’re gonna add logging to GitLab. You can already do monitoring, but we’re gonna add something for your logs, we’re gonna add application performance monitoring. Lots of other things that we think are needed to have a complete solution.

    So, if you started a startup, the only things you have to do is incorporate it, get a Kubernetes cluster, and install GitLab on it—job solved. Like, you’re up and running, you don’t need any other tools, and the only thing you have to do is, you push your code and already today, GitLab detects what language is it, it builds your code, it tests your code, it tests for all the security issues that might be there, it tests your code quality. It gives you all that. It gives you a review app to work with, kind of a staging app. That already works in GitLab today and we’re expanding on that, and I’m super excited to deliver that to the market this year.

    Shimel: DevOps complete. Thank you. Ashish, Sid—thanks so much for being our guests on this episode of DevOps Chat. Continued success with GitLab. We’ll be checking in—actually, I’ll see some of the GitLab folks at IBM Think in a few weeks, and we’ll get an update there, but Sid, you’re always welcome to be a guest here, don’t be a stranger, and congratulations.

    Sijbrandij: Thanks so much. Thanks for having us. I enjoyed this. Looking forward to coming back.

    Shimel: Alright, Ashish—

    Kuthiala: Thank you, Alan, for having us.

    Shimel: – good luck, my friend. Be well. This is Alan Shimel for DevOps.com. You’ve just listened to another DevOps Chat, and we’ll see you soon on another DevOps Chat, everyone.

    — Alan Shimel

  • Automation Levels the Playing Field for Continuous Compliance

    Automation Levels the Playing Field for Continuous Compliance

    At this point, I doubt there is any company doing business anywhere that doesn’t fall under at least one compliance framework. One universal truth of compliance—in intent, if not in practice—is that it is a process, not an event. Checking boxes and passing an audit is a requirement of most compliance mandates, but the real value of compliance lies in maintaining that level of visibility and security all the time—which is why continuous compliance is important.

    The Need for Continuous Compliance

    Hopefully I don’t need to continue to make the case that compliance is an ongoing requirement. Because it is something that needs to be consistently monitored and maintained, it also makes sense to automate the process as much as possible.

    In a report titled, “How to Avoid Compliance and Audit Concerns When Using DevOps,” published in November, Gartner stated, “Auditors favor the consistency and traceability of automated systems that have strong logging capabilities and transparent auditable controls.”

    A recent survey of more than 1,500 users conducted by Chef found that 74 percent of cross-functional application, infrastructure and security teams do exactly that, though: They assess software for compliance manually prior to production. Once violations and vulnerabilities are discovered, half remediate manually instead of automating the process. Manual processes result in teams’ detecting and remediating security issues in days (31 percent) or weeks (19 percent), instead of hours (18 percent).

    “At scale, manual reviews or out-of-band security tools bolted onto the DevOps pipelines won’t be able to keep pace with new velocity of change,” said Derek Weeks, VP and DevOps Advocate at Sonatype. “We saw this first with paper-based open source governance policies in the application security realm and it’s exciting to see similar approaches applied to infrastructure.”

    Andrew Storms, VP of Product at New Context, agreed. “Infrastructure automation and configuration management are key requirements for securely storing and transmitting data. … In DevSecOps, bringing security into the development process early and often helps ensure more cost-effective data security.

    “Think about this: By 2020 there will be some 20 billion IoT devices connected to the internet,” he continued. “Most organizations can’t even handle automating security and compliance of their existing infrastructure. The growth demand is far outpacing human abilities. Organizations simply cannot afford to rely on the methods of treating our infrastructure as cute and cuddly pets—and spending weeks producing evidence for security audits is not going to happen.”

    “Rather than force cumbersome security processes on top of the DevOps pipeline, security and compliance professionals need to adopt a new mindset of integrating controls as code,” Weeks added. Codifying controls initially improves the performance of the checks, but, over time, serves to expand the breadth of security and compliance practices as more controls are coded.”

    Speaking a Common Compliance Language

    Effective communication is a critical part of the process. “If you were ever in doubt on the importance of testing, then you should ask NASA about how they lost the Mars climate orbiter,” Storms said. “Years of development and likely some millions later, the space probe burned up because two software components weren’t using same units to communicate.”

    Chef just released InSpec 2.0, which advances the capabilities of compliance automation. Mike Vizard wrote, “InSpec provides a mechanism to automate compliance management using a set of declarative tools that don’t require anyone on the IT team to possess programming skills.”

    The leveling of the playing field by removing the need for programming skills is crucial for DevOps organizations, noted Julia Dunn, Director of Marketing for Chef. “One of the main challenges to DevSecOps (getting developers, security and operations to work together) is communication: not only seeing each other’s point-of-view but having a common language to collaborate on requirements,” she said. “InSpec is a framework that allows for those requirements to be expressed unambiguously in something all three groups can understand. This is in contrast to the previous state, where security throws lengthy scan reports over the wall at operations, or developers toss insecure code at security folks for manual inspection.”

    Jon Williams, CTO of niu Solutions and a Chef InSpec user, has seen the benefits of such a technology. “InSpec has helped us unify our compliance, security and DevOps teams and streamlined audits, reducing the thousands of staff hours usually required by as much as 95 percent and eliminating duplication of effort and data throughout the process,” he said. “It has given these teams more control over compliance policies and enabled business units to be more active in maintaining their own environments. Most critically, it allows us to continually monitor for audit compliance, ensuring desired state and eliminating change drift between nodes.”

    Regardless of what technology you choose, it is important you have a process or framework in place that enables the various parties involved to speak the same compliance language. It is also crucial to automate as much as possible to provide continuous compliance rather than just passing a compliance audit and assuming everything will stay that way.

    — Tony Bradley

  • DevOps Chat: Startup Playbook, Founder to Founder Advice w/ Rajat Bhargava

    DevOps Chat: Startup Playbook, Founder to Founder Advice w/ Rajat Bhargava

    Two old, good friends of mine published a book yesterday called Startup Playbook. The book is “founder-to-founder advice on how to create and build your startup.” In this DevOps Chat, I sat down with my friend and co-founder Rajat Bhargava. We talked about why he and Will Herman wrote this book, what he hopes others can learn from it and Raj gives us 3 tips on what every first time founder should know.

    Between them, Will and Raj have started more than a dozen venture backed companies and have had multiple 8 and even 9 figure exists. If you have ever thought about starting your own company, had a great idea but weren’t sure what to do with it or just thought you could do a better job than the people you work for now, this book could be what you are looking for.

    For this week (Feb 19) the book is available on Kindle for just 99 cents! Here is the link for the kindle download.  Also a few times at the end of our conversation I referred to the book as the “Founders Playbook”. I tried to dub in the correct name, the “Startup Playbook” but my feeble interviewing skills are far superior to my sound engineering ability. Apologies.

    As usual the streaming media of our chat is immediately below, followed by the transcript of our conversation.

    Audio

    Transcript

    Alan Shimel:              Hey everyone, it’s Alan Shimel, DevOps.com, DevOps Chat. We’re here with another great DevOps Chat. Today’s chat is a little different for a couple of reasons. First of all, my guest is a very long-time – did I say long-time? – friend of mine, Rajat Bhargava. Raj, welcome to DevOps Chat.

    Rajat Bhargava:         Thanks for having me, Alan.

    Alan Shimel:               Hey man, thanks. Secondly, our chat today is a little different. Rather than talking nuts and bolts of DevOps and that kind of stuff, we’re gonna talk about Raj’s new book, along with his co-author, another friend of mine, Will Herman, called The Startup Playbook.

    Raj, I’ve known about the project for some time, but tell our audience, share with them what is The Startup Playbook. Why did you write it?

    Rajat Bhargava:         Well, The Startup Playbook is really our founder-to-founder conversation with other entrepreneurs. We’ve seen so many books out there that are from DCs, they might be from analysts, they might be from other folks out there, but we felt that founders out there needed a book from us, from other founders. Whether we wrote it, Will and I, or somebody else, we just felt like somebody had to talk about the tips, the secrets, the tribal knowledge, the stuff that we get by getting together and talking to other founders. We just felt like that was missing.

    We have these conversations with so many other founders on a daily basis because we mentor and advise and help support other founders, we just felt like we needed to write it down. It’s like around the campfire, we have so many of these conversations, we felt like we needed to write it down so that there was a place that other founders could go to to learn quickly.

    Alan Shimel:               Just in the way of full disclosure, you know, Raj, you and I have worked together on at least two or three companies, maybe more. But in the way of background, you’ve actually founded, what, about eight companies now, starting when you were still back at university at MIT, correct?

    Rajat Bhargava:         Yeah, that’s right. So, it’s probably now close to 10 that I’ve helped start, or started myself. There’s just a lot of mistakes that I’ve made and a lot of challenges that I’ve faced, and I wanted to share those. On the same, Will has probably started five different businesses and he’s got tremendous experience, tremendous knowledge. We’ve both been through so many different things that we wanted to share those so that other people didn’t have to go through the same pain that we went through.

    Alan Shimel:               As we used to call it, the idiot tax, right?

    Rajat Bhargava:         That’s right.

    Alan Shimel:               Our audience – I don’t know how many “entrepreneurs” we have listening in, though I hope there are some – is made up of a lot of technical people. People, Raj, not that different, frankly, than you, right? Who have a great engineering background, live, breathe and die in today’s technology.

    I’m sure there’s a big subset of them out there who maybe have a great idea, who see a hole in the market, who see a need in the market, have something that they say can really maybe change the world and make a great company. But they just don’t, they’ve never done it. They don’t know where to start. I know that’s who you are aiming for a lot in this book. Talk to us a little bit about that persona, that kind of person. What’s in the book for them?

    Rajat Bhargava:         Well, this book is absolutely targeted directly at them. In general, the book is a founder-to-founder conversation. The founders we’re really trying to target are those first-time founders, the folks that haven’t been through it before. Like you said, they may not know all the steps that they need to go through. They may not even know if founding a business is the right thing for them.

    So, we actually start before you start a business in the book. We start about whether this is the right thing for you to do for you, for your family, for you as a person. Is founding a company the right thing for you? We start there. So, it’s from the ground up and we build up from there.

    I think our belief is – we’re biased because both Will and I were engineers by educational background – we think some of the greatest businesses are started by engineers, by technical folks. We love when technical folks bring ideas forward, and we think they can change the world. The challenge is that the idea is just not good enough. Ideas are, unfortunately or fortunately, either way you look at it, they’re a dime a dozen. It isn’t the idea that’s so differentiating; it’s all of the other work that you do around the idea that makes it differentiating, that makes the successful business.

    That’s a lot of what we talk about in the book is that the idea really just isn’t enough. It’s a seed. It’s a kernel. It’s where you start, but it’s just the beginning. It’s a long, long journey, but a great idea is a wonderful place to start.

    Even an okay idea is a great place to start because you can iterate on it and you can improve it. You can make it much better. Then, a lot of the stuff that you do around that idea is what differentiates you and helps make you successful.

    So, for your audience, for technical folks who have an idea, we think that’s fantastic. That’s a wonderful, wonderful base to start with, to build a business.

    Alan Shimel:               You know, there’s a lot in there, Raj. I want to highlight a few things. First of all, it’s true, whether you have a job or you’re starting your own company, but don’t underestimate that when you go to start, become a founder, that it doesn’t just affect you, right? It affects your family, your loved ones, your significant others and what have you. Because I’ve never met anyone who is successful with a half-hearted attempt to start a company, or to start a business, or to nurture an idea like that, to nurture a business. To me, it’s either you’re all in or you’re out.

    Rajat Bhargava:         Absolutely, yeah.

    Alan Shimel:               That’s one thing I think people need to think. Let me also hit on one other thing and that is when you speak to people, and you’ve spoken to more than me, but I’ve spoken to my share, what’s the biggest thing holding people back? If I took a poll, Raj, and said, “Hey, who would like to start their own company, be their own boss, and make a lot of money and be successful?” Very few people are gonna keep their hands down; everyone wants to do that.

    What stops people from doing that? Well, number one, it’s a little bit of fear of the unknown, having never done it. But number two, the biggest thing I hear from people is, “My god, I don’t have the money to get there. What am I gonna do until my company makes enough money? How do I bridge from having a job with a steady income that keeps the lights on to the time where my startup is enough for me to support myself?”

    That’s probably the single most scariest thing for first-time founders. What do you think?

    Rajat Bhargava:         You know, I think you’re right. I think there’s absolutely the fear of the unknown and fear of failure which, by the way, 9 out of 10 startups, according to the research, fail, which we actually think is low. We probably think it’s closer to 95 out of 100, maybe even 97 out of 100. Very few companies succeed long-term. The ones that are breakout successes that we hear about in the media are few and far between. It’s so few of the ones that start.

    I think the second point is such an important point, which is how do I go start this business when I need money to live on, and there’s no cash coming in at the beginning? Well, that is a very, very hard problem. The good news is there are so many solutions to it, but some of it is a leap of faith, too.

    The thing I would start with is there is so much prep work that needs to happen before you launch a business. That is where we spent a tremendous amount of time in the book talking about how do you vet your idea? That doesn’t have to happen only with you quitting your day job, and going and working on it. Some portions of that at some point you may need to do full time, but you can do a tremendous amount of research, of vetting your idea, of building your plans, of figuring out whether this concept that you have can be really, truly built into a business.

    We encourage people to spend an inordinate amount of time at the beginning going and doing that. Because if you can build the right foundation at the beginning, you’ve got a shot at making it successful. If you start with a very shaky foundation, those cracks are gonna get exposed very fast and the chances of you succeeding go down dramatically. So, even if you need that job, even if you need that income coming in, there’s still a lot you can do to build out your concept and build out your business, even before you take the plunge.

    Alan Shimel:               Fair enough. So, Raj, if we take that as okay, that’s one tip for prospective first-time entrepreneur founders, ’cause we’re running low on time, give me three or four others, if you can. Maybe a top five of what a first-time founder should…really idiot tax saving tips here that they can really rely on.

    Rajat Bhargava:         Sure. I’ll give you three. The first one is the one we just talked about, but I’ll expand it.

    I think completely vetting your idea before you really go all-in is so critical. That includes things like the value prop, the target customer. How can you build a team, or who is the team? Can you actually build the product or service that you’re proposing?

    Can you differentiate it in the market? How are you gonna go get customers? How can you make money? Can you raise money to support yourself? All those things are the art of what we call vetting your idea. The more you can vet the idea from the beginning, the more solid of a foundation you have. That’s tip number one is completely vet your idea.

    Tip number two is build a team around you. People think that starting a company could be just a single founder. Especially as an engineer or a technical person you might say hey, I’ve got this great idea. Why do I need to have another co-founder until later?

    Well, we encourage you to get other co-founders right out of the gate. At least one, if not two. It’s critical that you have some people who are side-by-side with you as you go through this journey. Find people who have different perspectives, but that you can really work with. It’s one of the most important relationships that you’ll have in the business.

    The third thing is find mentors, advisors, board members, people that have done it before that you can learn from. There is nothing like saving yourself time, mistakes, challenges, problems by hearing advice from others. You have to be open to it, but if you’re open to it, you can really accelerate the progress of the business.

    Those are three quick tips that I would say are things a first-time founder ought to think about really hard.

    Alan Shimel:               Yeah, and those are three good ones, too. I’m sure the book is chock full of more of them.

    Raj, let me throw you a little bit of a curveball. In addition to wanting to start businesses because they see needs in the market, I can’t tell you how many engineers and technical folks I speak to that would like to write a book. Again, I don’t know what stops them. Many do cross that chasm, right, and write a book; many don’t.

    For people who maybe don’t want to start a business today, but do want to write a book…and most of them are not novels. They’re much like Founders Playbook, non-fiction kind of books. What advice can you give to budding or prospective authors out there?

    Rajat Bhargava:         You know, It’s different for everybody, but the thing that I think helped Will and I the most was we just broke it down into a little bit of time every single day. When I initially wrote the first draft of the book, I had a target of 1,000 words a day, which isn’t a lot, as you know, Alan. You can write 1,000 words probably in about 30 minutes, so it’s really not that much time. But if you plug away at that for 90 days, for just one quarter [break in audio]. The typical book is really more like 45,000 words. If you did it for 90 days, you’ve got 1,000 words, and you can throw away half of those words because –

    Alan Shimel:               And still have a book.

    Rajat Bhargava:         And still have a book. Or you could have two books, right? I think that’s the biggest thing is just don’t try and do it all in one sitting ’cause you can’t. That’s not the way it happens. You need to just chunk it up and do a little bit at a time. In my case, what I did was I just started writing. I didn’t even really have too much of an outline. I just started writing 1,000 words a day.

    Then, we took that and we started formulating that into a book. So, we threw out tons of it, we rewrote it a couple of times. There’s just a tremendous amount of editing work. But the editing work turns out to be a lot easier – at least it feels easier; it probably isn’t easier – because you already have something down. It’s the blank sheet of paper that’s really daunting. But if you say, “I have a blank sheet of paper today and I only need to write 1,000 words,” that’s not that bad. You can get there.

    Alan Shimel:               Sounds like an agile methodology to authoring a book. Doing it in sprints.

    Rajat Bhargava:         Exactly. Yeah, exactly. Do a sprint a day.

    Alan Shimel:               Yeah, that sounds cool. A little scrum meeting in the morning.

    Well Raj, we’re about out of time here, but for our audience who maybe are interested in taking a look at Founders Playbook, I know you guys are offering sort of an introductory launch as the book goes to launch special. Where can we get some information here?

    Rajat Bhargava:         You can go to startup-playbook.com and you can also go to Amazon and just search The Startup Playbook. It is 99 cents for the Kindle version for the first week or so; that’ll come up on Monday the 19th of February. It’ll be 99 cents for a week. We’ve tried to make it really accessible for everybody. We really just want to get the knowledge and the content out into the founders’ hands.

    Startup-playbook.com, or look on Amazon, The Startup Playbook.

    Alan Shimel:               Raj, beyond the book itself, I know you and Will have plans to try to nurture a community here with that. If people have questions, are there places on the website where maybe they could ask and stuff?

    Rajat Bhargava:         Yeah, absolutely. You can reach us on Facebook, on Twitter, on LinkedIn. If you go to the website, there’s links to all of that. You can reach out to us pretty easily. You can find us, and we’d be happy to engage and chat with you.

    Alan Shimel:               Sounds great. Hey Raj, I know this was a labor of love, but it was nevertheless a labor, and a long time coming for both you and Will. So, congratulations on the launch of the book, The Founders Playbook. You know what? We’ll keep an eye on it and watch it. I hope some of our listeners out here go check it out. The only thing you have to fear is fear itself. If you’ve got a great idea, the world’s waiting and you can make it happen.

    Rajat Bhargava, my friend as well as author of The Founders Playbook, thanks for being this episode’s guest on DevOps Chat.

    Rajat Bhargava:         Thanks for having me, Alan.

    Alan Shimel:               All righty. This is Alan Shimel. We’ll see you soon on the next DevOps Chat.

    — Alan Shimel

  • Testing and Certification in the Age of Automation

    Testing and Certification in the Age of Automation

    If we do not change the way we teach, 30 years from now we will be in trouble. Because the way we teach, the thing we teach our kids are the things from the past 200 years. It is knowledge based. And, we cannot teach out kids to compete with machines, they [machines] are smarter.

    We have to teach something unique so that a machine can never catch up with us.

    —Jack Ma, CEO Alibaba

    tl;dr synopsis

    Given the complexity of modern DevOps, technical certification is becoming more granular. Also, its use is exploding as a requirement for employment. Most technical certification is awarded by passing a test composed of multiple choice questions. Passing a test with multiple choice questions requires the test-taker to absorb a significant volume of knowledge for a short period of time. This acquired knowledge does not need to be retained. Hence, the certification can only attest to the bearer’s knowledge of a technology at the time the test was taken. And, essentially the test measures only the bearer’s ability to take the certification examine. Finally, testing by multiple choice does not ensure the certification bearer’s qualification in terms of required experience or higher-level thinking capabilities.

    Technical certification is valuable in the current commercial landscape. However, at some point, as machine intelligence is able to perform more of the activities presently done by humans, the format and methods by which certification testing is administered will need to change. Future certification testing methods will need to put greater emphasis on analysis and creative problem-solving rather than the current focus on knowledge acquisition and rote application. Also, what and how we teach humans will need to be unique, beyond the capabilities of artificial intelligence.

    Over the last 10 years, DevOps has become a much more complicated landscape. Regardless of the service provider of choice—Amazon Web Service, Microsoft Azure, Google Cloud, OpenStack, IBM, Joe’s Homegrown Cloud Service—you gotta know a lot to play.

    At a high level, service providers offer products that are surprisingly similar. For example, they all provide some sort of machine virtualization, storage services, database services, queuing technology, authorization service, provisioning technology and API management. But the way you manipulate the conceptual knobs and switches to get any of these products to work can vary dramatically between providers. There’s a lot of minutiae involved. Absorbing the amount of detail required to be productive in a provider’s product line takes time. And, ensuring that someone can actually work competently with a service provider’s products is a crapshoot. For example, a deployment engineer may understand the concepts behind dynamic provisioning, but getting a Kubernetes orchestration to work in Google Cloud is where the rubber meets the road. Doing is always much more difficult. As I said, there’s a lot of detail involved.

    So, how do companies ensure that the people they hire to fiddle with their multimillion dollar, mission-critical technology stack will deliver the goods without blowing things up? Most have an extensive technical interview process accompanied by a stringent reference check. Some just hope. Others require certification.

    The Rise of Micro Certification

    The notion of certifying that a person has the skills and experience required to be competent in a profession has been around since 1885, if not before. That was the year Massachusetts implemented the first written bar exam. Before that time, all a person needed to do lawyering was to get a note from a court saying the person could be a lawyer. That’s how it went for Abe Lincoln. Eventually the profession matured. Professional qualification was determined through a test-based approach. Early bar exams used essay questions. In 1972, with the addition of the Multistate Bar Exam, multiple choice questions were included.

    Going to multiple choice questions may seem like a trivial change, but it’s not. Whereas a knowledgeable professional is required to grade an essay question, multiple choice answers can be evaluated by anybody with the answer key. Hence, the popularity of multiple choice questions on the SAT. It’s the most efficient way to grade millions of high school test-takers each year. Evaluation is fast, cheap and a machine can do it.

    As I mentioned at the beginning of this piece, IT in general and DevOps in particular has become a place of increasingly granular complexity. There are a lot of technologies on the landscape. Each of those technologies has a significant learning curve to be overcome to achieve mastery. The conventional wisdom is that the best way to determine mastery is to test a person’s knowledge of the given technology and then measure the results of the test—high score, good; low score, bad. As with early testing practices, the fastest, cheapest way to make an evaluation is to administer a test made up of questions with multiple choice answers. This is sort of like the SATs, except most technical certification is geared to a particular vendor’s product line, not a general body of knowledge. (You can view a sample of test for AWS Certification as a DevOps Engineer Professional here).

    This type of professional testing paradigm is exploding. It’s infiltrating every corner of the technology landscape. And, more employers are requiring certification as conditions of employment, both for prospective employees and those already on staff. This would be OK, except for one thing: After you scrape away all the preconceptions and assumptions about the value of test-based certification, when it comes down to the bare-bones validity of test measurement, the only thing a test really measures is the ability to take the test. For example, a deployment engineer might be sitting for a certification test after having been up all night working to get the company back online after a system failure. The taker, who is exhausted, ends up failing miserably. Yet, some slacker just out of high school who has never seen in inside of a data center can take the test using stolen answers and pass with flying colors. Tell me, please: Who is really the most qualified professional?

    Testing by Regurgitation

    Certification is not the problem. After all, I really do feel more comfortable knowing that the surgeon about to open up my wrist to alleviate my carpal tunnel syndrome is certified by the American Board of Surgery. Rather, the problem is the way that certification testing is conducted. In my opinion, a certification testing process that emphasizes the multiple choice question-and-answer format is flawed. Even when questions are answered correctly, the test format does not really verify the competence of the taker, only that some facts are known. It’s just testing by regurgitation.

    Such testing requires the taker to absorb a lot of knowledge and then dump it out according to the test requirement at hand. There is no guarantee of long-term retention. It’s the age-old process of cramming for the test. Once the “time’s up!” bell rings, the chances are that the test taker will forget most of what she absorbed. Those with photographic memories might remember it all, but for most of us mere mortals, it’s a one-shot deal.

    Few people in tech work using the cram-it-in method. Most of us just use this thing called the internet to get the answers we need, when we need them. It’s what Bill Gates calls “information at your fingertips.” Yet, we still test for the knowledge in your head at the moment.

    There is a better way.

    A Better Way

    Certification is good provided it ensures that the bearer knows what he or she is doing. When certification is about nothing more than checking the right checkbox on an online test, there’s a problem. Put a monkey in front of a multiple choice test for eternity and eventually the animal will score 100 percent.

    So what do we do?  We do the following:

    Support Open Book Testing

    If the purpose of the test is to ensure that a person has a certain body of knowledge at the time he or she takes the test, make the body of knowledge required available at test time. It’s called open book testing. That’s how it works in real life. For example, you need a NodeJS package to provide mock objects. Do you rely upon your existing knowledge about the packages you know? Great, but what if you don’t know? Does work come to a grinding halt? No. You go to NPM to find one that meets your need. That’s how it is in real life. A competent professional is expected to have a baseline, high-level knowledge of his or her professional domain. For the day-to-day details, a pro has the ability to find accurate answers to questions quickly.

    Instead of having a certification testing process based on cramming, maybe we should eat the dog food we make in real life and go open book. There is little downside to researching what you need, when you need it. Besides, the test question created a week ago might be best answered by a new technology created today.

    Put the Emphasis on Essay Questions

    The actual mechanics of testing vary according to the level of cognition being examined. I wrote an article about this a few years back. The best way to test higher-order thinking—analysis, synthesis, evaluation—is by using essay questions. After all, what is a Ph.D. dissertation if not a very big essay question? Essays require a level of thinking that is hard to capture using lower-order test methods such as multiple choice. Essays reveal not only what the test taker thinks, but also how the test taker thinks. There’s a reason why your high school math teacher wanted you to show your work when taking an algebra test. Sometimes how you get the answer is just as valuable as the answer itself.

    A good certification test will provide a question format that allows test takers to illustrate their thinking along with the way they arrived at an answer. Thus, the need for a bias toward essay questions. Again, how you think is just as important as what you think.

    Put a Human Back in the Examination Process

    I’ll share an intimacy with you: I’m not very good at answering multiple choice questions. When confronted with one, I spend a lot of time trying to figure out what the question is really asking. You’d be surprised at the number of test questions out there that are vague and confusing.

    This was a real problem for me back when it was time to go to college and do the SAT dance. Fortunately, my alma mater focused more on my essays than my SAT score. Also, a lot the exams during my collegiate years were administered as essays or conducted as interviews with a professor. Later, when it was my time to be the Guy Awarding the Grade, my process was to evaluate students by interviewing them about one or many of the projects they did. I based my grade on the quality of the project itself and the thinking that went into the work. I tested my students by talking to them and reviewing their work. I like to think that I got a pretty accurate idea of a person’s competence, more than a multiple choice test could ever provide.

    Providing the opportunity for human interaction in the examination process can go a long way to providing the information required for making an accurate evaluation of a person’s competence.

    Require Testimony of a Candidate’s Experience, Character and Competence

    Think about this: It’s possible to be an AWS Certified Solutions Architect Professional without ever having done a day’s worth of paid, real-world work. All you need to have done is pass the certification exam. Yes, of course passing the exam is meaningful. It’s a grueling test. At the least, the test might prove that should be able to work the conceptual dials and switches that go with a given AWS service. But, as far as having the real-world experience, competence and character to play a critical path role in a mission-critical enterprise, some other instrument of certification is required. This instrument is sometimes called a reference—or, as the Wizard said to the Tin Man in “The Wizard of Oz,” a testimonial.

    As you read above, back in Abe Lincoln’s day, a letter from a court attesting the good moral character of the bearer was all that was required to practice law. You can think of good moral character as the ability to show up on time, do reliable work, demonstrate a commitment to excellence and get along with others. Such a description might sound melodramatic, but it counts.

    While it’s true that some certifications require that bearer to have a modern version of “good moral character,”  in addition to good test scores, many do not, particularly in tech. An employer wants to hire people who know what they’re doing and have the wherewithal to work effectively in an organization. Potential candidates want to provide evidence that they have what it takes to be a good employee. Building a testimonial of the bearer’s experience, character and competence into a technical certification will add a great deal of value for all involved. Also, the testimonial needs to affirm the bearer’s ability to adapt to change. In the not too distant future, your commercial value will not be about what you know today, but rather how fast you can work with technologies that will appear tomorrow.

    How to Win the Race

    For better or worse, we’re in a race with machines to keep our jobs. They don’t really want our job. They’re incapable of desire. But, that’s not to say they can’t have our job should the machine prove to be able to do it. Sadly, the current practice of technical certification is focused on passing tests concerned with the acquisition of knowledge.

    Right now, we’re winning the race. Machines still have problems with knowledge-based tasks involving advanced image recognition and reading comprehension. We still know how to differentiate between a dog, cat and an apple tree better than a machine can.

    But, it’s not always going to be this way. Keep in mind that talking machines were very primitive 20 years ago. Their voice had an artificial, staccato rhythm that was distinctly robotic. Today, it’s hard to tell the difference between a human or machine when a robocall disturbs your evening meal. It’s only a matter of time before the machine can do mid- to high-level knowledge-based work better than any human. Jack Ma’s quote at the beginning of this article supports this assertion.

    We need to change. The commercial value we’ll bring to the workplace in the future will be about analyzing, synthesizing and evaluating. Machines will be able to to the lower-level thinking faster and more accurately than we ever can. The realm of human excellence will be about better understanding the world around us and bringing new things into that world. The way we start is by teaching something so unique that the machines can never catch up. This thing is creativity. The paradox is that creativity can always be learned, but rarely be taught. Still, we must try. We have little choice not to.

    — Bob Reselman

  • Manage Cloud Clutter with Better DevOps Hygiene

    Manage Cloud Clutter with Better DevOps Hygiene

    One of the core principles of DevOps is automation. Continuous development, deployment, monitoring, security and every other element of a continuous DevOps life cycle depends on as much of the underlying tasks and actions being automated as possible. Although the provisioning, scaling and decommissioning of most DevOps resources in the cloud are handled by automated processes, inevitably there will be rogue artifacts and other cloud waste left behind.

    When we talk about DevOps, we tend to focus on developers, but the reality is that DevOps straddles development, IT operations and general business processes. It is the cross-section of these three areas and how they mesh that enables DevOps to deliver increased agility, scalability and streamlined app development and deployment when implemented effectively.

    Managing cloud costs and ensuring efficient use of resources should also be a function of DevOps. DevOps teams tend to hyper-focus on app development, integration, speed of delivery and app performance—all of which obviously are very important. It’s also important, though, to reduce or minimize infrastructure costs and ensure that cloud resources are not being wasted or left behind as remnants that are no longer being used.

    Tearing down walls between teams and breaking down silos are hallmarks of DevOps. That same approach should be applied when it comes to managing cost and cloud waste. A white paper from ParkMyCloud notes, “Consider the problem of running resources when they’re not in use and how this contributes to cloud waste. If access is limited and only one person has the ability to turn instances on and off, then one individual is solely responsible for ensuring that cloud resources are used optimally.”

    To efficiently manage cloud resources and reduce unnecessary clutter and costs, you need to pay attention to the right data. DevOps relies on monitoring and feedback to ensure apps are running as expected and performance is optimized. That monitoring and feedback also needs to look at things such as whether cloud instances are properly sized, if they’re continuing to run when they’re no longer being actively used, and whether there are rogue or orphaned resources out there that are unnecessary but still being paid for.

    Just as automation drives DevOps itself, automation can also be an effective tool for managing and minimizing cloud costs and clutter. According to the ParkMyCloud white paper, 44 percent of cloud workloads are non-production—test, development and QA environments, for example—and don’t need to run 24/7.

    “One way to save really significant dollars in dev and test is to switch your resources off when you go home,” suggested Werner Vogels, CTO of Amazon Web Services. An organization can theoretically reduce costs associated with development and test environments by up to 75 percent by simply turning those things off when they’re not needed.

    If you think about it, managing cloud costs and clutter in DevOps is essentially the same as managing security within DevOps, which is essentially the same as effectively managing DevOps. You have to consider cloud infrastructure costs as a part of the holistic environment or project; remove silos to empower everyone involved to be part of the solution; monitor key metrics to get useful feedback; and automate the process as much as possible. As with DevSecOps, the main key is to make cloud hygiene and the reduction of cloud clutter an integral part of the DevOps life cycle—not something that is tacked on after the fact.

    — Tony Bradley

  • The Origins of DevOps: What’s in a Name?

    The Origins of DevOps: What’s in a Name?

    Depending on your frame of reference, DevOps will celebrate it’s ninth or 10th anniversary this year. In 2016, RightScale’s “State of the Cloud Report” estimated that 70 percent of SMBs were adopting DevOps methods. Every indication is that percentage has increased since then. As DevOps prepares for its second decade of existence, it might be worth a stroll down memory lane to revisit the origins of DevOps methods—and even the term itself.

    Pre-2007: A Perfect Storm of Events

    Prior to 2007, a series of circumstances came together that would eventually give birth to what we know today as DevOps.

    “Lean manufacturing” was already well-established as a set of best practices for manufacturing. Often branded as the “Toyota Manufacturing Method,” lean manufacturing strives for process optimization across the manufacturing floor. (Sidebar: Toyota leaders were originally inspired by the ingenious assembly line methods introduced by Ford Motor Company.) “Continuous improvement” is the mantra for lean manufacturing and practitioners continually evaluate ways to:

    • Keep inventory at a minimum. Lean manufacturing means minimal inventory on hand: raw materials (materials used to produce goods) and finished inventory (completed products waiting to be assigned to an order and/or shipped).
    • Minimize the queue of orders. Ideally, orders received should move immediately into fulfillment mode. A key metric of lean manufacturing will always be “order to ship” time.
    • Maximize efficiency in the manufacturing process. Process re-engineering and improved automation will marry into a goal to produce goods as quickly as possible. Each station of operation along the way (cut, weld, assemble, test, etc.) is evaluated for inefficiencies.

    In IT, traditional waterfall methods of application development were already giving way to rapid, iterative methods such as agile. “Speed” was the battle cry, even if quality output was occasionally compromised in the pursuit of rapid development and deployment. Similarly, on the operations and infrastructure side of IT, Cloud Computing and, in particular, infrastructure as a service (IaaS) and platform as a service (PaaS) were coming into their own as mature service offerings.

    Finally, a fledgling set of tools classified as “continuous integration (CI)” tools began to emerge. The notion of CI tools was birthed and branded by Grady Brooch back in 1991 in his Brooch Method.

    Related Content:

    DevOps: The Innovation Power Couple

    A Definition of DevOps for the Masses

    2007-2008: A Frustrated Belgian

    Belgian consultant, project manager and agile practitioner Patrick Debois took on an assignment with a Belgian government ministry to help with data center migrations. In particular, his role was in certification/readiness testing. His duties required him to straddle activities and relationships between the application development teams and the operations teams (server, database network). His experiences—and frustrations over the walls of separation and lack of cohesion between application methods and infrastructure methods—planted seeds of discontent for Debois. His desire for a better way would soon lead him to action.

    In 2008, at the Agile Conference in Toronto, Andrew Schafer posted an offer to moderate an ad hoc “Birds of a Feather” meeting to discuss the topic of “Agile Infrastructure.” Only one person showed up to discuss the topic: Patrick Debois. Their discussions and sharing of ideas with others advanced the concept of “agile systems administration.” In that same year, Debois and Shafer formed an Agile Systems Administrator group on Google, with limited success.

    2009: The Case for Dev and Ops Cooperation

    At the O’Reilly Velocity Conference, two Flickr employees—John Allspaw, senior vice president of technical operations, and Paul Hammond, director of engineering—gave a now-famous presentation titled, “10+ Deploys per Day: Dev and Ops Cooperation at Flickr.” The presentation had a dramatic flair to it, as Allspaw and Hammond would role-play the contentious interplay between representatives of  Development and Operations during a typical software deployment, along with all the finger-pointing/blame that goes on, such as, “It’s not my code, it’s your machines!” Their presentation made the case that the only rational way forward is for application development and operations activities to be seamless, transparent and fully integrated. Over time, this presentation has reached legendary status, and is historically viewed as the seminal moment in time for that called out to the IT industry for methods that we now know as DevOps.

    Unable to attend in person, Debois watched the Allspaw/Hammond presentation by video stream. He was inspired, and—at the prompting of others through social media—formed his own conference called Devopsdays in Ghent, Belgium. By this point in time, the term “DevOps” had officially landed in the history books.

    2010: DevOps in the United States

    With a growing constituency, a Devopsdays conference is held for the first time in the United States in Mountain View, California, on the heels of the Velocity annual conference. Fast forward to 2018: There are more than 30 Devopsdays conferences already scheduled for 2018, including dozens across the United States.

    2013: ‘The Phoenix Project’

    For many of us, another noteworthy moment in the history of DevOps was the publishing of the book, “The Phoenix Project,” written by Gene Kim, Kevin Behr and George Spafford. This fictional novel tells the story of an IT manager thrust into a seemingly hopeless situation, as he’s charged with salvaging a mission-critical ecommerce development project that’s gone off the rails. His mysterious mentor, a board member steeped in the disciplines of lean manufacturing, guides the main character into new ways of thinking about IT and application development, introducing the concept of DevOps along the way.

    (Sidebar: The Phoenix Project helped inspire us to write “Outsource or Else!” a similar business fable in which a VP of Software uses DevOps when outsourcing development of a major new product.)

    DevOps for the Future

    It’s reasonable to describe DevOps as a journey, or perhaps an aspiration, rather than defined destination. DevOps, like lean manufacturing, seeks continuous improvement, seeking higher output, greater efficiencies and even continuous deployment. Automated tools that support DevOps continue to evolve.

    Much has been accomplished since the inception of DevOps in the last decade and we expect to see even more in 2018 and beyond.

    — Steve Mezak

  • Third Annual DevOps Dozen Winners Announced

    Third Annual DevOps Dozen Winners Announced

    We are very pleased to announce the winners of the third annual DevOps Dozen Awards. In many ways this year was a watershed year for the DevOps Dozen, as the process of selecting, voting and choosing the winners was much more refined and mature. In each of the 12 (it is a dozen, after all) categories the winners were absolutely deserving of the award and recognition.

    Many thanks to our all-star panel of judges: Jayne Groll, Robert Stroud, Helen Beal, George Hulme and Ericka Chickowski. I was the tie-breaker judge and had to cast a vote in just one category. I should also say that almost all of the categories were very tight. Every one of the finalists was deserving of the recognition!

    These awards are a reflection of some of the great work being done in the DevOps community. They are not exhaustive; there are many, many worthy companies, people, projects and products out there. But here is our take on the best of the best in DevOps.

    So, without further ado, here are the 2017 DevOps Dozen winners:

    Best DevOps Open Source Project

    best open source solution Jenkins is the winner of the Best DevOps Open Source Project for 2017. Originally created in 2004 as a fork of “Hudson,”  Jenkins is one of jenkinsthe leading open source automation servers available. Jenkins has an extensible, plugin-based architecture, enabling developers to create 1,400+ plugins to adapt it to a multitude of build, test and deployment technology integrations. In 2017, Jenkins surpassed 155,000 active installations (these are active installations that report usage back to the Jenkins project—only a fraction of total installations), and approximately 1.5 million users, making it the most widely deployed continuous delivery automation server in the world. Learn more: https://jenkins.io/

    Best DevOps Commercial Solution

    devops dozen commercial solution ElectricFlow from ElectricCloud is the winner of the Best DevOps Commercial Solution for 2017. ElectricFlow is an enterprise-grade DevOpselectric cloud release automation platform that simplifies provisioning, build and release of multi-tiered applications. A shared, model-driven approach to managing environments and applications allows teams to coordinate multiple pipelines and releases across hybrid infrastructure in an efficient, predictable and auditable way using the tools and stacks teams already know and love. Learn more: http://electric-cloud.com/products/electricflow/

    Best DevOps Solution Provider Executive


    devopsdozen SP exec     The winner in this category was actually a 1/1a entry of the CloudBees team of CEO Sacha Labourey and CTO Kohsuke Kawaguchi. Sacha’s Sacha and KK Cloudbeesbackground prior to CloudBees was as CTO and key person at JBoss. KK was a Sun engineer when he first became involved with Hudson and later founded Jenkins. Together, they have built CloudBees into a powerhouse in the DevOps and CI/CD space. While working closely with the Jenkins community, they also have been instrumental in the continued growth and popularity of the Jenkins project. Together, they are a formidable team worthy of this award!

    Best DevOps Solution Provider


    DD best devops solutions provider     CA Technologies, which now includes Automic, BlazeMeter, Runscope and Veracode, is the winner of the Best DevOps Solution Provider for 2017. From a toolchain perspective, CA Technologies provides the widest array of highly automated solutions necessary to build and optimize integrated workflows across application development, delivery and operations. Spanning the entire application life cycle—including planning, development, testing, release and operations—only CA Technologies offers the full range of capabilities needed to deliver secure, quality applications quickly with an unparalleled customer experience.

    Most Innovative DevOps Solution of the Year

    most innovative devops solutionThe Most Innovative DevOps Solution of the Year 2017 goes to Habitat from Chef, the leader in Continuous Automation. Habitathabitat is a SaaS-based service that provides the fastest way to package apps simply and consistently for deployment and management across flexible cloud-native architectures such as those comprising Docker Swarm, Kubernetes and Cloud Foundry, both on-premises and in-cloud. Habitat gives developers and operations teams complete control over the containerized application life cycle. Learn more: https://www.habitat.sh/

    Best DevOps Evangelist


    devops evangelist    Derek Weeks VP and DevOps Advocate at Sonatype (co-founder of All Day DevOps) is the Top DevOps Evangelist for 2017. This year All Day DevOps offered 100 free onlinederek weeks sessions led by DevOps practitioners to 33,000 professionals around the world. Since its founding, the community has attracted, educated and supported more than 40,000 DevOps professionals. The conference also brought attention to more than 250 DevOps-related communities, meetup groups and organizations that are helping to educate practitioners around the world. In 2017, Derek delivered 18 DevSecOps-related keynote addresses, conference speeches and meetup presentations in the United States and Europe. He also produced more than 75 DevOps-related blogs over the year, covering topics from continuous delivery to DevSecOps, modern infrastructure and cultural transformations. The blogs not only delivered first-hand experiences, but also highlighted the work of more than 40 DevOps practitioners from different companies around the world.  The blogs can be seen at Sonatype, All Day DevOps, DevOps.com and Dzone. In addition, Derek championed a number of DevOps reports and surveys in 2017, including the “State of the Software Supply Chain Report” and “2017 DevSecOps Community Survey,” to shed light on industry best practices and areas in need of attention or evolution.

    Best DevOps Transformation


    devops transformation    CapitalOne was the judges choice for Best DevOps Transformation for 2017. Over the past few years, the 20-year-old financial services giant capital onehas been transforming itself, tapping DevOps to establish a continuous software delivery cycle. Topo Pal, a senior director and senior engineering fellow, told a packed room at the recent DevOps Enterprise Summit in San Francisco the company’s journey suits its entrepreneurial spirit. “We consider ourselves a startup, and like any startup, we have different DNA,” said Pal. “For instance, we build our own software.” All the better for adopting DevOps. The company does just about everything (including back-end processing) in the public cloud now, with a focus on building microservices using open source technologies. Having spent the previous three years building out automation steps, scaling DevOps, adopting open source and cloud and starting to measure its success, Capital One is now turning its attention to continuous delivery. Learn more:

    Best DevOps Transformation Leader


    devops transformation leader    Paula Thrasher is the winner of the Best DevOps Transformation Leader for 2017. Paula is Director of Digital Services at CSRA Inc. Paula paula thrasher Thrasher has 20+ years experience in IT and has spent the last 15 years trying to implement Agile culture in the federal government. Her first Agile project was in 2001; since then, she has led more than 15 programs and projects as an Agile developer, technical lead, Scrum master or Agile coach. Her teams have helped two federal agencies migrate applications to Amazon AWS GovCloud and done some other amazing DevOps ninja work along the way. Paula is also the executive lead for CSRA’s corporate giving group’s search for a STEM charity and joined forces with Girls Who Code to bring summer programs and after-school clubs focused on closing the gender gap in IT. She also participated in the DevOps Forum 2017, co-authored a paper on DevOps learning in organizations,”Continual Learning: Building a DevOps inspired Career Track,” and peer reviewed other paper topics produced by the forum this year. Paula has spoken publicly at multiple industry conferences including AllDayDevOps  2017, Red Hat Government Symposium and DevOps Enterprise Summit 2017.

    Best New DevOps Solution Company

    nClouds is a provider of DevOps and cloud services, products, and solutions, specializing in modern infrastructures on AWS — and winner of Best New DevOps Solutions Company for 2017. They work as an extension of their customers’ teams to build and manage modern infrastructure solutions that deliver innovation faster. nClouds is a certified AWS Advanced Consulting Partner with 250+ DevOps AWS implementations, 500+ deployed pipelines, and 10,000+ managed servers. They call their new SaaS offering, nOps, modern change management at the speed of DevOps. nClouds was named to CIOReview “20 Most Promising DevOps Solution Providers – 2017” and “20 Most Promising IT Service Management Providers – 2017”.

    Best DevOps Book/eBook

    best DevOps“The DevOps Adoption Playbook” by Sanjeev Sharma of IBM won the Best DevOps Book Award for 2017. The book lays out a clear “how-to” devops adoption playbookfor DevOps, including topics such as value stream mapping, how to lead the transformation of culture and scaling DevOps to the enterprise. As Sanjeev himself says, the book started from a Powerpoint deck of best practices he created to take in everything he had learned.

    This book represents the culmination of years of learning by not only Sanjeev, but by IBM in general, and includes some of the leading names in DevOps that he learned from during this time. Sanjeev also stays true to the “playbook” paradigm, making references to cricket, soccer and other sports throughout. It is a good read and well-organized. To learn more, listen to this DevOps Chat: ‘The DevOps Adoption Playbook’ by Sanjeev Sharma.

    Best DevOps Presentation of the Year

    devops presentation“DevOps: The New Reality for Enterprise Transformation,” a 45-minute keynote presentation at IBM InterConnect 2017, is the winner of the Best DevOps Presentation for 2017. It features Gene Kim, John Comas, NBCUniversal and IBM execs and  distinguished engineers including David Lindquist, Angel Diaz, CJ Paul and Michael Elder. Are you dabbling in DevOps? Maybe you’ve run a great pilot project or have a terrific two-pizza team that’s innovating new apps. How do you scale their success to your whole organization? The answer lies with a combination of cultural, personal and technical transformations that blend methodologies and cloud-native approaches with proven automation patterns and existing on-prem development. The presentation includes client stories and demonstrations of the latest IBM solutions to help companies transform their teams and build, deploy and manage their apps for large-scale enterprise IT environments.

    Best DevOps.com Article of the Year

    “Continuous Integration vs. Delivery vs. Deployment: What’s the Difference?” by Chris Tozzi won the Best DevOps.com Article of the Year devopsy.kinsta.cloud articleAward for 2017. Christopher Tozzi has covered technology and business news for nearly a decade, specializing in open source, containers, big data, networking and security. He is currently Senior Editor and DevOps Analyst with Fixate.io and Sweetcode.io.

    Again congratulations to the winners—well-deserved recognition for all of your efforts.

    For those interested in next year’s DevOps Dozen, information will be out in July or August of this year.

    — Alan Shimel

  • The Case Against Human Employment in the Age of Automation

    The Case Against Human Employment in the Age of Automation

    A few months back, the small Japanese restaurant down the street from me was renovated. The owner redesigned the layout of his business to solve a vexing problem: How to stay in business when faced with a continuing pattern of employees not showing up for work.

    The layout of his restaurant went from this:

    Figure 1: The pre-automated local Japanese restaurant

    To this one:

    Figure 2: The neighborhood Japanese restaurant after automation

    The new layout eliminates the need for servers. It turns out, the people he hired to take orders and bring food from the kitchen to a customer’s table just weren’t showing up for work. We’re not talking one or two random no-shows; rather, it was habitual behavior among the workforce he hired, mostly high school grads and college students. Each day was turning into a crapshoot. He’d get a text an hour or two before the beginning of a shift reporting some other excuse: sickness, lost dog, whatever. Usually, the note closed with the inevitable, “Sorry, won’t be in today. See you tomorrow.”

    So he automated.

    Related Content: An Unanticipated Obstacle to Full Automation

    The new design is working out well. You come in. He directs you to a table. There’s a touch-screen monitor at each table diners use to place their order. The orders go directly to the kitchen, which is separated from a row of tables by a glass partition. When a dish is ready, the owner (who also happens to be the chef) or an assistant passes the food out to the diner’s table under an opening at the bottom of the glass (see Figure 2, above). At the end of the meal, the owner meets you at the front station where you pay the bill. No more wait staff. No more cashiers. No more “I can’t make it today” texts.

    Automating to Stay in Business

    Removing the servers and using automation was not an easy decision for the owner. His business is not some large-scale, multi-unit corporate enterprise looking to automate away human labor to increase profit margins. This guy has worked seven days a week for 10 years to build his business into a mainstay of the neighborhood. He prefers to employ humans but he can’t. He’s automating because the available human labor is unreliable. He believed that using such an unreliable workforce would eventually put him out of business. He really needed to make the change.

    So he did.

    No Employees, No Employee Retention

    So far, things are going well. However, now the kitchen staff is beginning to join the No Show Club, despite the fact that he is hiring kitchen workers at a rate of $14 an hour—a good wage for kitchen work. What’s causing people to go MIA? Maybe it’s the product. After all, how many California rolls can you make before going nuts out of boredom? Maybe he’s difficult to work for. And I doubt they’re jumping ship to work at Mickey D’s; the Golden Arches is only paying $10 an hour. In any event, the owner has a employee retention problem that, my guess is, he’s going to solve using automation.

    What Does This Have to Do with DevOps?

    At some point, every human being becomes unreliable. Some have poor work habits, but most simply are being human and humans make mistakes. That’s how we’re built. Some of us make a few mistakes, while others make a lot. But, we all make mistakes. Most of us learn from our mistakes, and we still make others. Our human nature to make mistakes makes us inherently unreliable.

    Machine intelligence doesn’t have this problem. It will get things right, eventually. And then, once operating reliably, it’s rare for that intelligence to revert to a state of unreliability. A human can become unreliable at any point in time—taking a sick day, resigning from the job or all the mistake-making that comes with learning a new skill or technology.

    We in DevOps have a lot in common with the owner of my neighborhood Japanese restaurant. Most of us have worked really hard to keep our customers happy and coming back, well beyond the typical 40-hour work week. For the most part, we’ve been successful.

    Our success increases demand. When demand goes up, the complexity of our work increases, as does the need for more reliable operations. But at some point, our inherent human unreliability kicks in. We make mistakes. To keep things going reliably, we automate. Then we move on to the next thing. Fortunately, for the foreseeable future, there will be a “next thing” to move on to.

    Still, intrinsic to our work is a motivation to eliminate human labor from the technical landscape. We have no evil intention. We just want to do the best job possible. Given the choice between a script and a human doing the work, we’re going to choose the script every time. The script will evolve to 100 percent reliability. The human won’t.

    As I said, for now, it’s not really a problem. There will be a next thing to move on to. And, in terms of my neighborhood sushi joint, maybe the out of work no-shows will get the message about the value of showing up at a job on time and ready to work. Or, maybe they won’t. Maybe as automation becomes the standard of labor fulfillment for repetitive work, immature workers will never be given the opportunity to get the experience they need to enable them to master basic employment skills. After all, they’re human and some point will become unreliable. So why hire them at all? Why not get a few computerized gizmos to do the redundant work and then focus on keeping customers satisfied and staying ahead of the competition?

    Having no employees translates into having no unreliable employees. A business without unreliable employees is an efficient business. Efficiency in business is what it’s all about.

    And this, my friends, is the case against human employment in the age of automation.

    — Bob Reselman

  • Cloud Generation Firewalls are the New ‘Next Generation’

    Cloud Generation Firewalls are the New ‘Next Generation’

    There’s a new kid on the block—or a new firewall, at least. Traditional network and security technologies generally don’t translate well to a cloud environment. Next-generation firewalls might have been “next generation” when they were introduced, but now they’re “previous generation”—legacy technology that isn’t capable of doing the job effectively in the cloud. The new “next generation” is the cloud generation firewall.

    I’ve never been a fan of using the term “next generation” as a label. Where do you go from there? I mean, I love “Star Trek: The Next Generation, but what is supposed to come after that—”Star Trek: The Next Generation After That One”? It’s going to seem silly in 10 years when we’re talking about the archaic, outdated “next generation” technologies.

    Thankfully, we might soon be able to retire one of the “next generation” labels. ZScaler sort of started down this path when it introduced the “world’s first cloud-based next-generation firewall” in 2015. Barracuda recently took it a step farther, dropping the “next-generation” and announcing cloud generation firewalls.

    Cloud Generation Firewalls

    Aside from a catchy new name that’s good for marketing, though, what does that actually mean? Traditional firewalls, including next generation firewalls (NGFW), were engineered to protect a centralized data center or network, or a distributed network spread out over multiple locations. The public cloud is a very different environment.

    At face value, a distributed infrastructure with multiple locations connected across the internet seems similar to a cloud environment. A public cloud environment is much more dynamic, though, and has a more loosely coupled architecture than a dispersed private network. Companies move to the cloud to take advantage of DevOps, automated deployments, elastic scaling and other elements that enable them to develop and deploy applications and services more efficiently, but legacy security tools including NGFWs can’t protect effectively in the cloud.

    As with other network and security solutions, companies need firewalls that understand the cloud. To be effective, a cloud generation firewall should be cloud-native and integrate with cloud management and monitoring tools. It also should be able to automate deployment and scaling, and offer APIs that enable it to work with DevOps development tools. Finally, its licensing and pricing should reflect the dynamic nature of the cloud environment.

    Challenges of NGFW in the Cloud

    I spoke with Tim Jefferson, VP of public cloud for Barracuda, about cloud generation firewalls. He agreed that companies are facing challenges as they embrace the public cloud and try to use tools that were optimized for data center architectures in cloud environments.

    The design principles of an NGFW are “anti-pattern” in the public cloud, he said. The public cloud environment is based on keeping things loosely coupled and easily scalable, while NGFW solutions are designed for tight coupling with ties back to centralized policy enforcement.

    What’s more, he noted the licensing model for a traditional NGFW poses a problem in the cloud. Traditional licensing forces customers to deploy only in production environments, when security best practices suggest automating security architectures in all stages. Customers want to—and should—deploy security in all stages of development to better support DevOps principles, and cloud-generation licensing enables this.

    Cloud-native solutions need cloud-native licensing that takes into account the rapid scalability and dynamic nature of the environment. Barracuda went with an approach that meters based on bandwidth rather than time, so customers are billed only when the cloud generation firewall is used, not when it’s just sitting idle.

    Naming conventions aside, cloud generation firewalls just make sense. Many organizations fall into the trap of moving to the cloud and thinking they can just use the network and security tools they already have—just in the cloud. The reality is that you need cloud-native tools to effectively take advantage of the reasons you’re moving to the cloud in the first place.

    And, if cloud generation firewalls catch on, perhaps we can move beyond “next generation” and stop using that dumb term.

    — Tony Bradley