devopsy.kinsta.cloud

Category: Blogs

  • How to Design DevSecOps Compliance Processes to Free Up Developer Resources

    How to Design DevSecOps Compliance Processes to Free Up Developer Resources

    Anything that gets rolled out quickly in the technological world is a major step forward—as long as it doesn’t compromise security.

    DevSecOps is able to provide compliance processes far more effectively than before, and without jeopardizing security while also freeing up developer resources.

    How do you direct your IT team to keep your applications and customers safe without completely draining your resources in the process?

    The answer: automation.

    Why is Security Vital to DevSecOps?

    To take full advantage of the agility and responsiveness of DevOps, you must bring forward an IT security team through the entire life cycle of your apps.

    In the past, when software could take months or years to complete, it was perfectly acceptable to bring in the security team in the final stage of development. But in the days of DevOps, a security team must get consulted from day one due to rapid and frequent development cycles.

    An effective DevSecOps system can be completed in days, so the outdated approach to handling security during the development phases also must change.

    How Did Security Radically Change DevOps?

    It is important to remind developers that security is a shared responsibility that must be integrated from end to end to guarantee a smooth and functional application. The mindset is so important that many have changed from using the term DevOps to DevSecOps to stress the importance of the need to build a very sound security foundation.

    If you are already making the transition to the cloud with AWS, chances are you have already implemented DevOps strategies into your interface so you can test new virtual machines (VMs) and deploy applications quickly without waiting for infrastructure to get manually configured.

    For those who prefer the old-school method of developing applications, neglecting security until the end leaves their applications exposed. One of the most common security issues with most SMEs is misuse of security during beta tests, which is basically akin to leaving your front door wide open when you go to sleep at night. It’s just a terrible idea.

    Making the transition from DevOps is not a luxury but a necessity for developers who want a secure environment, even during the dev/test period.

    Designing Compliance Processes for IT

    Even if you add “security” to the term DevOps, it means little without properly adding the processes you use into the building of the applications.

    Security must be built into the processes by design, or you are not correctly taking advantage of what DevSecOps means—and your developer resources will get neglected, too.

    Like the name implies, security must be in the center of development and operations. To maintain compliance, you must enforce a shared responsibility between AWS and the customer:

    • AWS is responsible for security of the cloud, including the hypervisor and hardware.
    • Customers are responsible for security in the cloud. They do this by securing your applications and network with services through a third-party vendor.

    The best examples of DevSecOps in the cloud take this philosophy to heart.

    Automation is Critical to Compliance Processes

    A “security first” mentality is good , but are you really taking action with your compliance processes to free up developer resources?

    The answer for many developers is, “Not quite.” Automation is essential to DevSecOps, yet so many systems lack the right tools. This is especially unfortunate for organizations that have the necessary skills and procedures in-house to implement built-in security processes directly into the application by design.

    For other organizations, automation is only achievable by leveraging their services from a security integrator, who can then provide services to help define processes and roles, unveil security tools and set up automation procedures.

    Steps to Establish Automation on DevSecOps

    Now that you know automation is crucial to the success of any system, here are a few basic steps you can take from day one to start building the security processes into your design:

    1. Consider a “shift-left” mentality when adding new processes, ones that incorporate design thinking from customers’ security needs and remain consistent. Steps such as threat-modeling storyboards build cyber resilience into the application before the first code is even written.
    2. Consider a cloud-based vulnerability scanning solution. It will help you define standards for the AMI as well as validate adherence to those standards.
    3. Implement an audit, logging and monitoring solution to track changes to the in-use AMIs. It helps combine account activity and AWS resources from the cloud with data related to security incidents. The in-use AMI changes can do everything from providing ongoing insights to allowing other security controls and having influence over network security.
    4. View the AWS Marketplace catalog. Along with what an internal staff or a systems integrator can help automate, there are a number of other terrific resources out there for automation. The software listings are all from independent software vendors that make it straightforward to locate, test, buy and deploy software that runs effectively on your AWS environment. Like other AWS services, the providers are billed based on consumption, so you pay only for what you need.

    By asking yourself the important question, “What am I trying to accomplish with my security controls?” you can get a variety of answers regarding what measures you need to take for compliance processes.

    Final Thoughts

    Speed, speed, speed. Developers today simply have no time to waste when it is time to launch new applications and get them out the door.

    You have little time to burn on extra features, but you also can’t afford leaving your systems exposed and vulnerable to attacks. That’s why DevSecOps uses superior security processes to free up developer resources by utilizing as much automation as possible.

    — Sam Bocetta

  • DevOps Chat: Chef Goes All-In on Open Source

    DevOps Chat: Chef Goes All-In on Open Source

    In front of next month’s ChefConf in Seattle, Chef has announced a major refinement to its business model. Affirming and clarifying its commitment to open source business models, Chef will now and in the future release all of its software as open source.

    In this DevOps Chat, we speak with Corey Scobie, SVP of Products and Engineering at Chef. He explains that Chef will be adding services, training and support to the open source software, as well as packaging all of this together.

    For the full story, be sure to listen to the podcast and check out the video version of this interview below, along with coverage on DevOps.com.

    [videopress nUXwRMxU]

    Transcript

    Alan Shimel: Hey everyone, It’s Alan Shimel, DevOps.com, and you’re listening to a DevOps Video Chat. Today’s DevOps Video Chat features Corey Scobie, SVP of Engineering, is it?

    Corey Scobie: Product and engineering, yep.

    Shimel: Product and engineering now at Chef. Hey, Corey, welcome.

    Scobie: Thanks, Alan, happy to be here.

    Shimel: I’m happy to have you here. Today is a red letter, big day in the history of Chef. You guys are announcing some pretty significant changes to the model is I guess what I would call them. Though, I don’t think the changes are as big as some may initially feel, but let’s go over them. I’m going to let you take the wheel here. Tell our audience.

    Scobie: It is a big day today for us at Chef. We are announcing some changes to the way we both produce and ultimately distribute our software, which I think are really exciting. There’s a lot going on in the industry right now. As you probably know, Chef has been one of the marquee players in the open source software development community for some time. Part of the announcement that we’re making today that I think is most exciting, is we’re announcing that what we’re going to do is going forward, produce 100 percent of our product-oriented software in that open source model.

    For us, that’s a change because we, over time, and through the addition of additional projects and additional software at Chef, had started following a little bit of an open source, open core path, which means that we had some proprietary software and some open source software, but when we got down to it and we thought about the history of Chef and the roots of what has made Chef such great software in the past, it really came down to collaboration. We realized the best way to make software is to make it in collaboration with the people that actually use the software, at the end of the day.

    For us, that’s the big first talking point about the announcement is that we are committed to 100 percent open source software development going forward. We’re committing all of our software to open source repos and GitHub, and then ultimately attaching it and giving it to the commons with the Apache 2.0 license.

    Shimel: Sure, so Corey, you know – so you’re talking to someone who has a bit of a history in open source, 20 years. I remember when the whole open core movement first came out. Look, Richard Stallman and that crew, they were really against open core, because they felt it was – it was almost heresy to the whole cathedral and bizarre open source kind of crowd, where if it’s open, it’s open. You can’t be a little bit pregnant. But, nevertheless, we’ve seen companies succeed with this open core or premium, where the core is free and there are premium modules on top of it, but it’s a slippery slope. In all honesty, I think Chef got caught in a little bit with that slippery slope with what’s truly open, what’s free, what is our proprietary that we make money from? I don’t know if the market really got it.

    Scobie: Yeah, one of the things that is really tough when you’re following an open core strategy is drawing a very clean line between what is free and what is not free in the world of your technology offerings. For us, that got complicated and then even conflated further as we added more projects to the list. When we opened the InSpec project for security and compliance, when we opened Habitat for application automation, what ended up happening is we had multiple projects that were governed different ways that had different lines between the free and the core part of it. I think ultimately at the end of the day, had I been at Chef during that part of the journey, I probably would have made the same decisions, I’m guessing, but at some point you get back to this reconciliation point where it’s like, “Look. The best thing that you can do for customers if number one, you can collaborate with them, and number two, you can have a really, really clean line between what is free and what is not free in your world.”

    For us, it becomes super simple with this announcement. What is free is all of the intellectual property that we’ve contributed to the commons in the world, and will contribute to the commons going forward. What isn’t free is all of the work we do to make a turnkey, enterprise-class, package that enterprise-class customers can pick up and distribute at hyperscale in their organizations, from that point.

    Shimel: That brings us to the next thing, and of course this is always the issue. So if you’re going to make all the software and the IP open, well how the heck do you make money?

    Scobie: Sure. As it turns out, we’ve got venture capital funding, investors that want to see us turn that into dollars as well. For what it’s worth, we’re feeling pretty good about the shape of the business right now. We just came off the best quarter of revenue we’ve ever had at Chef, prior to this business model, this model change around how we distribute software whatsoever. So we’re feeling really good about the shape of the business, but just the inability to have that simple conversation about what’s free and what isn’t. What we decided was as we’ve matured as a company, and as the space has matured in enterprise automation, what we found is the majority of our customers actually do want to have a very material commercial relationship with us.

    Not only do they want the software, but they want the support. They want the content. They want the expertise. They want the warranties and indemnities that go with all of that. So for us, it was very simple. We realized at the end of the day, that the most — that there was a huge constituency of our customers who what they wanted from us was a very curated commercial offering out the other end. So we’ve decided that’s where we’re going to put our effort and our energy, and we’re going to trust in the open source world to fill in the gaps where that isn’t the offering that they want as well.

    Shimel: We look at this model, Corey. Obviously, Red Hat’s the poster child for much of the open source world in that most of their software, if not all of it, was in fact open source. It was all in the packaging and the support, and that kind of thing. It sounds very much like that’s going to be the model, or am I wrong?

    Scobie: It is – no – there’s a strong analog to what Red Hat has done in the industry as well. What’s interesting is I think that that is the antithesis of what we are seeing from a lot of the more recent moves in the open source industry where there’s a desire to become more proprietary in some cases, or in other cases to try and somehow bend, I think, the definition of what is open source software. For us, the core of it, we get back to the four freedoms, and freedom zero is about the freedom to use the software, but it’s free as in freedom. It’s not free as in dollars.

    Shimel: Free as in freedom, or free as in beer, as we used to say, but Corey, to be fair to these companies that are – that we do see going another way, it’s because they didn’t cut it. They tried the Red Hat model, and for whatever reason, they didn’t cut it and they had too much pressure from their boards and the VCs and investors, and so forth. So they tried to take a path that was quicker to money. I’m not here to judge anybody’s – as someone in the Godfather once said, “Far be it from me to disparage what someone does for a living” –

    Scobie: I hold the same opinion, Alan, by the way. I sit in judgment of nobody on the decisions that they make about what’s right for their business. I just know that for us, what’s right is to try and both honor the open source roots that we came from, and to also serve the commercial customers that were really knocking at the door and saying, “Look, this is what we want from you. We want even more engagement. We want even more value out of your commercial subscription.”

    Shimel: Agreed. Before we jump into the next part of it, which is how it manifests itself within the Chef universe, I wanted to just tie a knot on this one, and mention that when we talk about this model, and you said, “Oh, we’re going to count on the open source community to fill in the gaps,” and one might say, “Well, what are those gaps? I think people have to realize what made Red Hat great was not just distributing Linux. It was that RHEL, and Fedora even before that. It was that package, that distro, that had everything you needed and wanted in there in a really easy to use – you didn’t want for anything else. It was all right there, nicely done for you, instead of running out and having to download different packages and distros from all over the internet, which is what held back Linux on the desktop for so many years. There wasn’t that one package that you could do that.

    I think that’s what you’re referring to is Chef’s strategy going forward is to have that killer distro, with all the right packages in it. If you don’t want to pay Chef for that, and I don’t think at this point, though people like software that is free as in beer, and they get free as in freedom with it, no one denies that a company should be able to make a living, that a company should be able to make a profit, pay its employers, pay its developers, and go forward. So for people who either don’t have the money, or don’t have the inclination to buy the packaged product, they can get the open source components, and the community can work on giving you the glue, in essence, that puts these all together. Correct?

    Scobie: Absolutely, for sure. Santos came out of that exact scenarios. The scientific community needed something that wasn’t going to be necessarily nearly as curated as Red Hat and Linux was, but they needed a platform for scientific computing, so CentOS was born and has a place in that world. I think our vision for that is exactly the same. If there’s a downstream project that is born of Chef InSpec or Habitat, we welcome it into the community, and we look forward to whatever innovations they bring, and we want to collaborate with anybody that wants to come and take that work out of the commons and create the derivative work out of it, that allows them to meet whatever their goals are. That is the freedom in free and open source software.

    Shimel: Absolutely. I think one of the other things though, Corey, is when we look at Chef over the last couple years, we’ve seen the whole InSpec line of security functionality, and we’ve seen Habitat. Look, Habitat was an ambitious piece of – well, it was more than a piece of software. Habitat was an ambitious play, but because there was so many balls in the air and going off in different directions and open/closed, proprietary/not, people never – it wasn’t explained well enough to people. I spent the time at ChefConf so I understood Habitat well, but there’s maybe a thousand people at a Chef conference as opposed to millions of people in the world. I think it was really hard getting InSpec and Habitat, carving out their niche in the world, amidst this whole confusion around the closed/open, supported/paid for, free/not free.

    Scobie: Yeah, that’s fair, and I think this really helps us simplify the messaging around that too. I will openly admit I think InSpec is an easier basic value proposition to parse, which is “Look, I need security and compliance in my organization and the right way to do that is express it as code”, so that should be fairly straightforward, and relatively noncontroversial. I think, with respect, to Habitat, what is interesting about Habitat is I agree with you, super ambitious project right from the get-go to redefine the abstraction of packaging and orchestrating the delivery of software in the world.

    That’s – it’s huge that there’s a huge surface area associated with that. I think what’s interesting is that we’ve learned much – we’ve learned a lot over the last couple of years about how to help people in the mental journey of how to think about Habitat in the world, and how it relates to Chef, and having that foundation of understanding and knowing Chef, which tens of thousands, if not hundreds of thousands of people already have, the ability for them to jump to where Habitat plays becomes much easier in the world where we can talk about them collaterally and not independently of one another.

    Shimel: Fair enough. I think that’s a great segue, Corey, to the next part of our conversation I wanted to have with you, and that is okay, how does this manifest itself down the line of Chef product and services? So we’re going to put all our IP out as open. I guess we should mention also, I think everything is being released under an Apache 2.0 license.

    Scobie: That’s correct. Apache 2.0.

    Shimel: Industry standard, GPL, Apache are probably the two biggest. How does this manifest itself? What changes does this mean for Chef’s products?

    Scobie: On flag day when we release new products with new packaging and new terms and conditions, it’s an evolution that we’ve been going through for some time now, which is really to understand what the interface parts are between the three core engines that we have. So we’ve created an engine for infrastructure automation, which is the Chef project, and going forward, just from a purely – from a branding simplicity perspective, it’s going to be known as Chef Infra, which is I think is fairly straightforward.

    Shimel: I’m sorry. Can you repeat that though? Chef…?

    Scobie: Infra, as in infrastructure.

    Shimel: I-N-F-R-A. Okay.

    Scobie: Yeah, and then we’re going to have Chef InSpec, for security and compliance, and the state relationship between infrastructure and security and compliance are pretty – it’s pretty intuitive, which is that if I deploy an operating system, and I want it to be locked down in a certain way and secure and meet a certain set of compliance standards, then testing that on a continuous basis, managing the drift, and being able to automatically remediate any drift that’s happening there seems like a really obvious relationship. What’s interesting though is the relationship between infrastructure and application automation. This is where things get really murky, even for longtime Chef customers who have been wildly successful in using Chef as an infrastructure automation tool. If you go and talk to those customers, what you’ll find is that they’re doing about – 30 percent of what they’re doing is about infrastructure deployment provisioning and shaping, locking down configuration management.

    70 percent of what they’re doing is deploying software on top of the operating systems in service of some application that’s going to come along at some point in time and need that – those software dependencies to be there. What we discovered, and this is really the genesis of Habitat, which I’m not even sure – we knew that what was happening in the space of Chef was not working at scale for us. That’s the reason that some folks at Chef went off and decided to create Habitat, but what we didn’t realize is why it wasn’t working as it was at the time. The reason is that there is a natural boundary for building systems from the bottom up in a layer cake approach. It doesn’t matter whether it’s Chef or Antival or Puppet or Kubernetes clusters or whatever, that there is a natural boundary layer, if what you’re doing is building systems from the bottom up.

    So you have an operating system, and then you have system libraries, and then you have packages, and then you have middleware, and eventually you have an application. At some point that dependencies graph, the modularity of everything that you lay down, and the dependencies graph gets so complicated that there’s no possibly way you can manage understanding what the implication of changing component X is on all the other components that rely on X. So the beauty of Habitat and where Infrastructure, and where Chef and Habitat come together is that Habitat takes the exact approach. It says, “Look, what you should do is you should define an application, all of the application’s dependencies and all of the orchestration steps for that application to run successfully in any environment regardless of where it is.”

    So by doing that, and by removing that layers – that bottoms up layers dependency, what we’ve really done is created an application stack, an enterprise automation stack is what we’re calling it where infrastructure and application automation marry together at a very, very clean abstraction level. The abstraction level is essentially, if you’re running software on the operating system that runs above the kernel, it should be defined by application packages and orchestration, and not by infrastructure packages and orchestration. Now, it’s architecturally it all makes sense, and it actually works really well in practice. Where it’s complicated is that organizationally, INO – traditional INO operators and other infrastructure groups have been tasked with more and more software deployment on top of their stacks. So this is a very paradigm-shifting idea. I think what is helpful for us is the enterprise automation stack allows us to create a framework and a template for how to do that in a way that helps customers adopt that methodology much, much quicker.

    Shimel: Fair, fair. So we got Chef Infra. We have Chef InSpec.

    Scobie: Yes, and Chef Habitat, Chef Habitat.

    Shimel: Chef Habitat sits – I wouldn’t say it sits on top of them.

    Scobie: Look, the stack is – we have trouble with this with our marketecture diagrams. If, what happens in the podcast now is you’ll throw up the marketecture diagram you’ll have, what you’ll see is a linear stack, but the state relationship between those three things is much more complex than can be defined in a diagram like that.

    Shimel: _____ diagram, sure.

    Scobie: Then on top of that, then there’s two other pieces to the portfolio. So we have Chef Automate, which is the observability layer on the top. Of course, if you have automation on all of these things, you want to be able to see across all of your pipelines, across all of your applications, across all of your systems. What is the state of everything that’s running? How has it been deployed? Is it in compliance or not in compliance, etcetera? So we’ve got this whole enterprise dashboard with automation up top. Then on the bottom end of the stack is we need to make it easier for developers to get started. So all of these engines are representative of either infrastructure compliance or application automation as code. What do you do as a developer? You download a bunch of tools. You get started writing code that are the artifacts that actually drive that automation. So Chef Workstation is the compilation of all of those tools and an easy way to set up your development environment to get going. That’s the entirety of the enterprise automation stack. It’s Chef Workstation, Chef Infrastructure – Chef Infra, Chef InSpec, Chef Habitat, and Chef Automate on top.

    Shimel: And repeat for me, what all of these packaged together is the –

    Scobie: Yeah, the Chef Enterprise Automation Stack.

    Shimel: That’s the whole enchilada, so to speak. This is all packaged up into one coherent type of product, if we can call it that. Fair?

    Scobie: Exactly. Yeah, one coherent distribution, absolutely.

    Shimel: Corey, you know the news broke today around this. Some are saying, “Well, Chef is pivoting. Chef is this.” How do you view this? How do you and the other folks at Chef view this? Is this really a pivot? To me, it’s – I don’t see it as a pivot.

    Scobie: Yeah, it’s not a pivot. I mean, first of all, I don’t think it’s – we’ve – you can question the effectiveness or the efficacy of our open source strategy in the past as we’ve meandered through different phases of that, but this is us staying true to the fact that we are an open source company, and so I think we’re leaning into the business model by which Chef was founded, and by which we’ve been operating for quite a while.

    Shimel: I agree with you.

    Scobie: I think from a technology perspective, we just have more clarity now in how to deliver the components and technology that have become critical to customers who are trying to solve large-scale enterprise automation problems. I don’t know that I would really describe it as a pivot. I think what we’re trying to do is use this as an opportunity to reintroduce people to Chef who maybe haven’t heard of the latest of our innovation in InSpec and Habitat, as an example, who maybe are not familiar with the work that we’ve been doing in the last three or four years. For us, it’s a little bit of a rebirth, but I wouldn’t call it a pivot.

    Shimel: I don’t know if I’d call it a rebirth. So here, look, you’re in the forest.

    Scobie: I am in the forest. There’s no question.

    Shimel: [laughter] I’m outside the forest. To me, obviously a pretty – I’m more than a casual observer. To me, Chef was always an open source company.

    Scobie: Always.

    Shimel: I think in some ways, what we’ve seen with Chef over the last five years, I would characterize as growing pains for an open source company trying to come to grips with remaining true to its open source roots, while at the same time recognizing you took other people’s money, and that’s how – I lived that life for 20 years. You take other people’s money, they want – it’s a funny thing. They want to get a return on their investment. It’s like trying to serve two masters. It’s a hard – it’s a hard –

    Scobie: Not only that, but look, the successful commercialization of any open source project or software company is how you actually create sustainable good in that community for the long-term. Obviously, you need participation from the community, but you also have to inject capital into that community if you want sustained innovation.

    Shimel: You have to extract revenue as well though. That’s where I think it gets dicey. So when I look at InSpec and Habitat, I almost feel at some level, they were somewhat hobbled by having to live in this schizophrenic, open/not open, making money/not making money. Well, if it’s totally open, we’re not making money. How much do we put into this? There was just too much of this all over the place, where to me, this is just a clarification. Look, Chef is and will remain an open source company, and our value to our investors is that No. 1, we have this tremendous community around us who are users of our software as well as contributors to the community, but No. 2, we are now offering a proprietary, a for-profit solution that takes the best of this and packages it up nicely.

    Scobie: Absolutely. Look, Alan, you make a great point. I would love to give a shout out to the community, because the Chef community is absolutely amazing. I know you’ve been to ChefConf. It’s an experience in and of itself. It’s hard to think of another tech conference that is as authentic and as welcoming and as diverse as ChefConf is, and it’s a bunch of people that come together every year, because they are passionate about solving the challenges of infrastructure management at scale, and application deployment at scale, and all of those things that Chef is interested in as well. So the community is a really key factor of it, but to your point is that there’s just this growing demand in the enterprise. We are the experts in figuring out how to do that in a really, really tight curated way. Of course, we’re going to take advantage of that market opportunity as well.

    Shimel: Sure, we got the news out. Is there any part of the news we’re missing, Corey?

    Scobie: No, I don’t think so. That really covers the announcements for today. I think the only other thing is we’ve got ChefConf 2019 coming up in Seattle May 20th – 23rd in Seattle. It’s going to be a great event. We also have a ChefConf for the first time ever, ChefConf coming to London at the end of June. It’s June 19th and 20th, I believe are the dates in London.

    Shimel: Really?

    Scobie: Yeah.

    Shimel: You know I’ll be in London those dates. I’ll have to talk to you about it.

    Scobie: Yeah, absolutely. We’d love to see you there, and we’d love to see others there as well. Yeah, so we’ve got a couple big events coming up where we’re going to showcase the latest of the innovation that we’ve had on all of the fronts, innovation, compliance, application automation, our enterprise consoles, all of that kind of stuff. so those are going to be fantastic events coming up. So other than that, we’re looking – we’re really looking forward to getting back to those community roots and really everybody being there together in the trenches together building the next generation of what this is all going to be.

    Shimel: Cool. So now let me ask you a question. Put on your wizard’s hat. If we’re talking about ChefConf 2020 now, a year from now, this strategy, this road has been – this strategy has been implemented now. We’re going down this road. How does it manifest itself over the next 12 – 24 months you think?

    Scobie: Look, we’ve been talking about that from a product strategy perspective, and I think we’ve got – there’s a lot of opportunity for us to do great work in pulling the pieces of the puzzle together, even more explicitly. We’re doing a lot of work with partners, so I anticipate that the ecosystem partners will be a big player in ChefConf 2020 as well. So we’ve been working with the likes of everybody from Microsoft, Azure DevOps, and the workflows there. We’ve got – we’ve been doing some product work, and continue to do product work with Amazon in the ops work for Chef Automate, managed and hosted to platform there. We’re quite engaged with GitHub because we’re seeing both GitHub and Artifactory and a lot of our customers in the DevOps toolchain and pipeline there. We’re doing some work with that. We’ve been doing a lot of integration with services now. So I actually think that the next phase of this is not just about the Chef parts of the stack, but how those parts of the stack connect with the rest of the ecosystem that is fast emerging in the enterprise automation space as well.

    Shimel: Sounds cool. Well, Corey, we’re about out of time. I want to thank you for taking time on what’s a crazy busy day for you guys, to brief our audience on this. Congratulations to the whole Chef team. Tell Barry and everyone else, looking forward to ChefConf. Unfortunately, I won’t be at ChefConf in Seattle. I’m in Europe, but we’re going to have a full team there doing video interviews there and everything. John Willis, our partner at Digital Anarchist will be there, and Chris Reilly. So we’ll have plenty of Chef coverage from ChefConf. Like I said, I didn’t even realize the London one. We’ll try to get over there. I think that’s Tech Week in London actually, which is a pretty big week.

    Scobie: Yeah, fantastic. Well, thank you so much, Alan. I appreciate you taking the time today as well. Look forward to seeing your colleagues at ChefConf in Seattle, and then hopefully we’ll see you in London in June.

    Shimel: Absolutely. Hey, Corey Scobie, SVP Engineering and Product. I got it right.

    Scobie: You got it right.

    Shimel: You’re talking to us about some news coming out of Chef today, a little bit of a change in how they package stuff, but they’re still an open source company. Corey, thanks very much.

    Scobie: Thank you, Alan.

    Shimel: Take care. This is Alan Shimel for DevOps.com, everyone. Have a great day.

    — Alan Shimel

  • When DevOps and Marketing Collide: Insights From Adobe Summit 2019

    When DevOps and Marketing Collide: Insights From Adobe Summit 2019

    I attended Adobe Summit 2019 to research the intersection of DevOps and marketing. Below are insights from my interviews with leading marketing technology experts. 

    Is DevOps making its way into marketing? In a short answer, a soft “Yes.”

    I recently attended the Adobe Summit, held March 26-28 in Las Vegas. This year emphasized digital customer experiences, and I was curious to take a peek into this world of marketing tech to see if DevOps was having any impact.

    Many of the people I talked to described how marketing technology is adopting a more agile, DevOps-inspired mindset. But is marketing actually driving product change? While Adobe and proponents would like to have us think so, some are not as sure.

    At Adobe Summit, I had some eye-opening meetings with professionals invested in digital marketing, retail and analytics services, most of which are using Adobe products in some form. In this article, we’ll highlight what they had to say on the intersection of DevOps and marketing.

    What is the Role of DevOps in Marketing?

    The need to break down data silos was a common theme among keynotes, sessions and throughout my meetings at Adobe Summit.

    When large digital marketing agencies take on clients, for example, a common disability they find is a disparate and dysfunctional customer data storage system. In our meeting, Alex Kelleher, managing director at Deloitte Digital, noted, “Data needs to be available, and integratable. That is the first thing we typically fix for our clients—we’ll fix the data in silos, get it in the same place, and usually simplify it.”

    Deloitte Digital’s process involves three D’s: first, they consolidate data silos, then build data-driven decisions, then deliver agile products. In doing so, Kelleher is seeing more and more developer practices making their way into marketing and sales.

    “We’re seeing CMOs manage their teams as agile teams, recruiting agile and scrum masters,” he said. “That’s very cool for CMOs to be involved in … those worlds colliding is super exciting.”

    Is DevOps Eroding Traditional Boundaries?

    DevOps principles encourage shared knowledge and open communication between teams. Now, it appears such ideas are affecting how marketing technologists architect their teams and delivery cycles.

    Rachel Lane, solution principal customer experience at Medallia, sees a trifecta of forces working in tandem. “Development, Ops, and Marketing must work together,” she said. “If one of those doesn’t link up, then we see extended lead time to launch, and it can become problematic.”

    Medallia’s goal is to meet customers where they are to gather actionable feedback through push notifications, surveys, embeddable surveys, Net Promoter Score (NPS) systems and other means. Engrossed in this world of customer feedback, Lane reminded us: “You should never ask a question that you already know the answer to.”

    Unfortunately, this still happens often. User data is often locked in silos and if multiple teams (sales, marketing, support engineers, etc.) are using separate user data repositories, customer interaction can suffer. Thus, user conversation data must be de-siloed and unified.

    For Lane, having tight customer experience-driven feedback is critical for reiterating the overall product. However, customers typically hate filling out surveys. To decrease friction, she foresees more passive voice-driven feedback through home AIs.

    Rearchitecting the Digital Footprint for Agility

    By now it’s evident that unclear data initiatives and messy data storage can hinder digital transformation. So, what are organizations doing to de-silo and open up data?

    Anheuser-Busch (AB) is a huge beer corporation, overseeing brands including Budweiser, Busch, Elysian and many others. Each has unique online eCommerce storefronts to sell ancillary swag, such as T-shirts, drinkware and other products.

    With public-facing sites for hundreds of brands, maintaining data standardization across its product lists was a nightmare. Disparate data structures can also inhibit directing change from within, slowing corporate-led growth.

    John Faviano, director of demand and integrated marketing technology at Anheuser-Busch, described how AB solved the issue by creating a consistent data structure from which all online stores can pull content from.

    “We’ve standardized the tech stack in the last three years to enable the business to run more efficient … You have to think from a platform perspective, not a single siloed technology,” he said.

    AB’s web APIs now grant incredible scalability, as the content is now delivered agnostic of client or brand. AB brands retain independent facades yet are fueled by a unified content storage and delivery process. “Our API strategy is a build-once-and-scale mechanism,” Faviano said.

    Digitization with IoT and AI

    Sreedhar Bhagavatheeswaran, SVP and global head of digital business at Mindtree, believes that every part of the enterprise needs to be digitized. He sees a wealth of untapped data that enterprises can leverage to transform user experiences.

    Having built digital presences for big-name consumer brands including Marriott, P&G and Kellogg’s, Mindtree understands the advantages of using a DevOps approach for software cycles. Though it may require a cultural change, “The probability of project success in a DevOps environment is much higher than in a non-DevOps environment,” Bhagavatheeswaran said.

    Release automation is integral to Mindtree’s process. Coding checks and CI tools such as Jenkins bring quick releases, enabling business teams and clients to see the impact on a daily basis. They also privilege API-fueled microservices designs for increased agility.

    “From an agile program management perspective it gets challenging, but that is the nature of the beast,” he noted.

    Bhagavatheeswaran is excited for the potential IoT has to evolve industrial, business and consumer-facing touchpoints. For example, industrial engineers could conduct remote servicing using digital twins of real physical equipment. He also described developing mobile IoT apps for his clients: Push notifications triggered by removing caps from makeup bottles, for example, could empower consumers with advice on how to apply makeup correctly.

    To sum it up, Bhagavatheeswaran described our current paradigm with a simple, powerful sentiment: “The experience is at the confluence of edge, humanity and intelligence.”

    Approaching the Developer Experience

    But digital transformation doesn’t stop with open data, APIs and IoT. Developer experience (DX) has become an important factor in maturing countless digital services.

    Jason Woosley, VP of commerce product and platform at Adobe, founded the Adobe-acquired Magento, which began as an open source project on GitHub, welcoming pull requests to alter product features (with the proper vetting in place, of course).

    By investing in open source and developer communities, Magento over time reached a critical mass, democratizing how developers build online storefronts.

    Woosley estimated that 50 percent of Magento code was built by third-party developers. One such feature is multi-source inventory, which, he said, accounts for 70 percent of the Magento code base. Naturally, he has a deep appreciation for the power of developer communities.

    “You can’t buy a developer community. You can’t steer or create a culture,” Woosley said. “Code custodians are actually participants, and to deserve that role DX needs to be delightful and empowered.”

    To improve the developer experience, Magento has hosted countless events, increased developer support staff, annotated its APIs, adopted GraphQL for extensibility, invested in optimizing developer portals and more. With the inclusion of Magento in Adobe Experience Cloud, more capital is available to further DX enhancements—something Woosley is excited about.

    Wednesday March 27, 2019 in Las Vegas. (Jeff Bottari/AP Images for Adobe)

    DevOps is Kicking Martech to Innovate

    Deloitte Digital’s Kelleher noted that “nowadays, your competitors are any digitally active application,” regardless of whether they’re even in the same industry.

    This means that a consumer will judge a mobile digital experience with an airline with the same scrutiny as calling an Uber or making a PayPal transfer. Users hold the same lofty digital expectations from any business, forcing us all to adapt dramatically.

    Fueled by the relentless pressure from users, marketing technologies are adopting DevOps principles to improve their offerings in a few ways:

    • Continuous releases: Martech uses DevOps strategies to release quickly. User-facing mobile app release cycles, for example, routinely adopt CI/CD.
    • De-siloing: User data and consumer conversation information are being distributed with greater alacrity across enterprises, improving end UX.
    • Open data initiatives: With web APIs, new platform-agnostic content distribution architectures have emerged. Development procedures are standardized across an enterprise, quickening the release cycle.
    • Cloud architecture: By adopting cloud-based management tools, martech is improving the asset management condition of supporting many user-facing touchpoints.
    • Developer experience: By embracing open source and supporting developer community, leading e-commerce groups are improving their offerings.

    While marketing is critical to the whole, it may not be the No. 1 business driver. When I asked Bhagavatheeswaran whether marketing was really driving the creation of new tools, he responded: “Is marketing driving the tools? Are they influencing? Not quite yet. However, it has the ability to influence the developer’s quality. Probably not their job.”

    Even if marketing is not directly influencing product development, we’re still seeing DevOps seep into traditionally non-technical environments, which is exciting.

    It was eye-opening to enter the world of marketing technology to see firsthand how these institutions—such as Ansira, a 100-year-old company—are using agile means to streamline digital transformation initiatives.

    With an emphasis on both creating and managing quality digital customer experiences, Adobe Summit 2019 shed light on many unique strategies being undertaken in the realm of martech.

    Whether it’s leading the development of user-facing apps, overseeing expansive brand catalogs, using a cloud-based single source of truth for content management or building developer communities to reap huge R&D benefits, the intersection of marketing and technology has never required more strategic action than it does now.

    — Bill Doerrfeld

  • 7 Ways to Introduce DevOps in Your Work Culture

    7 Ways to Introduce DevOps in Your Work Culture

    As many industries are going through a wave of digital transformation, a number of key technologies and principles are being deployed to business advantage. One such principle is DevOps.

    This collaborative principle is proving to be extremely helpful for businesses in their pursuit of customer satisfaction and beating their competition.

    The DevOps trend is growing, as cloud providers including AWS, Azure and Google Cloud are moving faster in upgrading their DevOps services and tools to address business needs.

    According to a Google Cloud and Harvard Business Review Analytic Services report, “two-third of companies that use DevOps have seen benefits impacting their bottom line, including productivity (67 percent), increased speed to market (70 percent), innovation (66 percent), customer relevance (67 percent) and product/service quality (64 percent).”

    While most companies are already reaping the benefits of DevOps practices, some organizations are still doing their homework. Here are seven tips to help you understand how your organization can use DevOps to its advantage:

    Begin with a Small Project

    It’s always best to start with a small project before you start engaging DevOps practices on a large scale. A pilot project will enable you to understand the core capabilities of the DevOps process and how you can map them to your business objectives.

    Utilizing Open Source Tools

    Using open source tools will keep you in touch with current trends and practices. Open source tools are easily available and easy to integrate. You can use these to enhance your DevOps experimentation.

    Introducing Security

    The DevOps process faces a threat in terms of security, which can be disruptive to an organization’s reputation. Hence, it is advisable for organizations to introduce security at the early stages of the DevOps process. Additionally, it saves a lot of time and resources, which are very crucial aspects of the DevOps process. The concept of DevSecOps is a wiser approach as it includes security all through the software development lifecycle (SDLC) process.

    Make the Most of DevOps Best Practices

    Looking for a process that drastically improves productivity and reduces time to market? DevOps best practices are designed to increase the pace of software development. With practices spanning site reliability engineering (SRE), end-to-end automation, continuous integration, testing and delivery, proactive monitoring and more, your team will be able to utilize their time effectively and devote their attention to other important business-driving processes.

    Enhance Team Skills

    The key to bringing out effective production results is having a skilled team that is equipped with the necessary training. Therefore, it is essential for organizations to provide their employees with a proper understanding of the organization goals in general and the processes employees are part of. According to the report, three-quarters of the best DevOps teams said they provide hands-on learning experiences with code labs and projects exercises.

    Avoid the Blame Game

    DevOps aims to bring about an increased pace of operations and fast learning, among employees. But most of the time for any new process to fall in place, individuals involved have to experience failure. And with the failure of the first project often comes the blame game. However, organizations need to understand that failure is the initial step to achieving future success. Moreover, firms should adopt the policy of “fail and adjust sooner.” The experience of learning from mistakes will take you farther along in your DevOps journey.

    A DevOps-Friendly Environment

    A DevOps-supportive culture is incomplete without the above exercises. Hence, it is essential that organizations bring about an environment that provides room for shared goals, improved transparency and making growth-driven decisions.

    So, industry change-makers: Are you ready for your DevOps experience?

    — Veritis

  • How DevOps Orchestration and Feedback Loops Can Enhance Your DevOps Pipeline

    How DevOps Orchestration and Feedback Loops Can Enhance Your DevOps Pipeline

    DevOps orchestration and feedback loops can significantly impact and improve your organization’s delivery pipeline. In this post, we’ll define and illuminate the importance of DevOps orchestration and feedback loops, and how together they can make a difference in your pipeline.

    DevOps Orchestration

    Although people widely use the terms “automation” and “orchestration” interchangeably, their scopes vary with significant process differences between them. DevOps automation automates a single function or a task in a DevOps pipeline that is routine and monotonous. Orchestration describes a string of such automated tasks consolidated into a streamlined workflow. In environments where automation would not work because of barriers among the groups, orchestration can be used to help organizations solve issues despite such barriers. Hence, DevOps orchestration simply brings together already existing automation networks into a cohesive, consolidated platform.

    The Need for DevOps Orchestration

    DevOps teams deal with constant navigation between development and operations. It would be highly beneficial if a single platform could pull together the tools teams typically use as various automation pieces from multiple DevOps toolsets. This is where DevOps orchestration services come into the picture, as orchestration enables teams to achieve that.

    The primary goal of DevOps orchestration is to ensure seamless and quick delivery of new builds into production. It also aims to achieve streamlined and error-free software releases. Different organizations use various coding standards, guidelines and policies. DevOps orchestration plays a major role in efficiently managing and ensuring adherence to such policies. DevOps orchestration also is pivotal in properly defining and maintaining third-party software and their multiple versions across different environments.

    DevOps Monitoring

    Effective DevOps monitoring is vital. It is the key to understanding and evaluating various performance metrics in the production environment. When done right, DevOps monitoring can provide visibility into different critical aspects such as deviations in performance, application functionality issues and service availability problems. In any DevOps environment, the ability to identify and pinpoint such issues proactively can be highly beneficial, and DevOps monitoring enables executives to do so.

    Feedback Loop

    Feedback loops are the essence of any process improvement and DevOps is no exception. Typically, such loops initially function with an organization’s policies and progress all the way through implementation, policy adjustments (if any) and usually ending with reimplementation. They are not limited to specific process cycles in a DevOps environment—for example, it’s applicable to CI/CD practices, infrastructure monitoring and application development

    Types of Feedback Loops

    Broadly, there are two categories in feedback loops: reinforcing feedback loop and balancing feedback loop. People refer to the former as “a loop of accelerating change.” In a reinforcing feedback loop, increased value in the first entity results in an increase in the second entity, which in turn further increases the value of the first entity. Conversely, in a balancing feedback loop, value increase in one entity results in a decreased value in the second entity, which consequently decreases the value of the first entity.

    Significance of Feedback Loops

    One of the major capabilities that differentiates and carves a unique niche for DevOps from the rest of its predecessor methodologies is its feasibility to incorporate feedback from customers as well as other systems in the pipeline, including the monitoring system. The feedback obtained helps to bridge the gap between the software’s function and customer expectations. It also provides valuable insights about what you can do to improve the build of the software and ultimately enhance its functionality and reliability.

    Feedback loops are particularly crucial when it comes to continuous testing. To carry out continuous testing effectively, building automated tests is only a job half done; the rest depends on the visibility of the test results and how you can utilize them to enhance the existing process. To do that, you need comprehensive feedback of your software’s performance across various stages of the software development life cycle (SDLC), starting right from development and continuing all the way to post-production. Implementing the right feedback loops will help you gather such detailed feedback.

    Thus, feedback loops serve as the difference between merely speculating how an end-user might consume a product and methodically improving the existing workflow to better suit the end-user requirements through feedback.

    — Kavin Elango

  • Why Unikernels Are Great for DevOps

    Why Unikernels Are Great for DevOps

    DevOps tooling seems to be in a never-ending state of self-transformation. The continuing evolution of infrastructure and the tools that go with it are forever creating new ecosystems. On the infrastructure side, we saw VMWare, Citrix and other companies bring to market massive virtualization in the late ’90s and early ’00s. This, of course, prompted a little bookstore in Seattle to unleash something known as “the cloud,” forever changing the way developers push code to production.

    The cloud, of course, is just an API around virtualization. A lot of the infrastructure changes prompted configuration management software, including Chef and Puppet, to emerge. Puppet is now 15 years old and Chef is around 10; even Ansible is now 7 years old.

    In the past years, we’ve seen tools such as Terraform emerge and other higher-layer infrastructure, such as Docker, which led into Kubernetes. There’s a lot of tooling out there to make applications run on servers. Just as the containers/Kubernetes ecosystems have tried to implement change on top of virtualization infrastructure, we are starting to see developers dig back down into the operating system layer to improve infrastructure there. This leads us to our topic today: unikernels.

    Unikernels are single-purpose virtual machines (VM). They only run one application—which is interesting when you think about it, because that’s precisely how a lot of DevOps practitioners provision their applications today. There is way too much software to provision nowadays—pools of app servers, clusters of databases, etc. Unikernels are fast to boot, fast to run, small in size, way more secure than Linux or containers and, if you have access to your own bare metal—either in your data center or through a provider—you can run thousands of them per box.

    Some stats? Some can boot in less than five milliseconds—that’s slightly slower than calling fork (three milliseconds) but orders of magnitude faster than Linux. Some can run databases including Cassandra and MySQL, 20 percent faster. Some people (yours truly included) have provisioned thousands of them on a single box with no performance degradation. Some are reporting them running as a VM faster than a comparable bare-metal Linux installation of that application. Clearly, this is worth investigating.

    There are many different methods that a given organization can take to provision software, but it’s pretty clear that popping into a server over SSH and sudo apt-get install something is very much of a downtrend. Phrases such as “immutable infrastructure” and “continuous deployment” fly right in the face of this sort of practice. You are either rebuilding your containers, rebuilding your instance or both. Unikernels really embrace this model as there is no apt-get-like functionality in them and you can’t log in to them remotely. They don’t even have the concept of users, mainly because at the end of the day, why do you even need them?

    At scale, it’s getting more difficult by the day to make the argument for this construct. If the app is misbehaving, shoot it in the head and take it down. Debug the reason locally. It’s extremely rare for an end user to reason that there is something wrong with the host operating system.

    In a unikernel, the only reason you’ll see a reference to a user is that most apps will demand to be run as a particular user or do things such as check for root privileges. In our case, if you look at underlying calls such as getuid, they are just stubs because we don’t need or use them. Keep in mind that this isn’t the mid-’90s anymore. We don’t have boxes laying around, where there are half-dozen users all running their code on the same box anymore. That model has long been gone. Unikernels don’t have shells and don’t have the notion of running other programs besides the one that is running.

    If it’s easier to think about, consider that unikernels are much more like processes than they are like operating systems, although they share characteristics of both. While many link, the application in others will load. Unikernels are a concept that are largely misunderstood because everyone assumes it’s somehow like Linux. Just as unikernels are not just “containers 2.0,” unikernels are very different from Linux as well. Linux is a general-purpose operating system, while unikernels only run one application. Linux was made to run on bare metal, while unikernels are made to run as VMs. Keep in mind Linux predates VMWare by a solid eight or nine years and the cloud by many more. Operating system researchers wanted to go this route in the ’90s, but we just weren’t ready yet.

    Another way to think about them is that they implement a sandbox for server-side applications. Much in the same way that the Chrome browser isolates each tab from each other to prevent one evil site from messing with another browser tab, unikernels prevent one application from messing with another application. Although under the hood, the technical implementation is quite different. More importantly, the unikernel single-process model eliminates the capability of running other ad-hoc programs inside the same VM. This is a huge blow to attackers who wish to run other programs on the same machine. It doesn’t prevent things such as remote file inclusion (RFI) attacks for various scripting languages, but for compiled and VM runtimes, it’s a striking blow because it deals with the fundamental problem of attackers using exploits only as a key to enter the server. Once they are on the server, they want to run their programs such as mysqldump or a cryptominer. They could care less about your software. Those organizations practicing shift-left security, or zero-trust infrastructure, should be keeping the development of unikernels on their radar.

    However, unikernels are not without their problems. These are computers we are talking about here. One of the larger problems for unikernel adoption was that to build them, you had to be a lower-level systems engineer doing things such as twiddling CFLAGS in makefiles or patching foreign codebases you’d never patch yourself. This was problematic and something we wanted to fix with a new open source tool. So, we opened up OPS—which can take any existing Linux application and run it as a unikernel, making their use a lot easier.

    If you’ve made it this far, I’m going to reward you: Let’s build and run a unikernel right now. Then, at least, the next time someone asks you if you’ve used them before, you can say yes.

    First thing is, you’ll want to grab OPS. You can paste this into a Linux or Mac shell, and it will download and install itself:


    curl https://ops.city/get.sh -sSfL | sh

    If you’d like to examine the source and build it yourself go to https://github.com/nanovms/ops.

    Next, let’s create a project directory:


    mkdir nodejs && cd nodejs

    Put this into a hi.js file:


    var http = require('http');

http.createServer(function (req, res) {

res.writeHead(200, {'Content-Type': 'text/plain'});

res.end('Hello Worldn');

}).listen(8083, "0.0.0.0");

console.log('Server running at http://127.0.0.1:8083/');

    This is simply a small little web server that listens on port 8083.

    At this point you might be asking yourself, Do I need to install node on my system? The answer is no—not unless you plan on using external packages.

    Now we can run it:

    If it’s the first time you run ops, it will first download a node “package,” which is the interpreter and anything else it needs to run. For some languages, such as Python and Ruby, this can be substantial.

    Then it builds a small disk-bootable disk image that contains your application code and, as you can see, boot straight into it. There is no init manager like Linux and there aren’t 30 different services that get fired off. Keep in mind this is a single process running.

    Now you should be able to hit it up with curl and see it run:

    You’ve just built and run your first unikernel. How cool is that?

    As certain trends such as “serverless,” “functions as a service,” and “no ops” gain steam, these concepts go hand in hand with unikernels. DevOps practitioners at organizations that use microservices and containers will also want to take notice as the amount of work to deploy artifacts such as these decrease considerably. Why is that? One of the many problems that containers had to struggle with was that all the existing underlying infrastructure layers that live at the virtualization layer, such as file systems and networking, had to be shifted up a few layers. Not only did this introduce performance and security challenges, but it also threw out the 20 years of virtualization software and ecosystem development that already existed. What’s interesting about unikernels is that they are always deployed as VMs, so they get to re-use a lot of the underlying ecosystem that has been existent, with very little or no change required.

    From an adoption standpoint, unikernels are flying at ultra-sonic speeds under the radar. A lot of people are not aware of how many projects or organizations are working on unikernels today. Unikernels are starting to attract major attention from many various groups and institutions. Even famous, old stalwart Linux contributors such as Ulrich Drepper, the maintainer of libc, are working on unikernel projects today. Now there are even foundations and consortiums for unikernels. One recently announced was the UNICORE project. IBM even listed its No. 1 cloud patent recently as a hypervisor built for unikernels.

    If you’re are a self-described DevOps person, it’d behoove you to learn more about unikernels and start playing around with them to understand them better. They aren’t going to kill Linux and they’re not going to kill containers, but do expect to see massive adoption in the coming years—not just because of their security and performance benefits, but because of the new forms of compute they unlock.

    — Ian Eyberg

  • 5 Steps for Successful Continuous Testing Implementation

    Continuous integration (CI) calls for code to be always working. Continuous delivery (CD) and continuous deployment mean that we must be able to roll out a working configuration to production at any time. As DevOps becomes the norm, bugs in production are now a developer’s personal problem. Developers (and their managers) want to know immediately if code changes they are working on can break something.

    What is Continuous Testing?

    The only guaranteed way to prove code and configurations are working is to test them. That’s why we need continuous testing. In organizations making this transition, performance engineers and test automation experts shift from minding a backlog of test-building work, to facilitating a democratization of testing. Everybody builds their code and everybody tests their code, continuously.

    Continuous testing is becoming the “new normal.” In this modern and agile testing approach, developers are equipped with low-friction testing capabilities and tools and are expected to unit test any new or updated code. With testing shifted all the way left to developers in pre-production, and all the way right to post production monitoring, organizations are just seconds away from proving code is ready and working to acceptable service levels.

    Although continuous testing is becoming a more standard practice, it certainly presents unique challenges for execution.

    Here are five steps to help any company successfully implement a continuous testing policy:

    Define Tests Early

    Lack of clear requirements can result in wasted time and delays. Behavior-driven development (BDD), acceptance test-driven development (ATDD) and model-based testing using tools should be used so that all requirements are clearly documented and communicated.

    Test cases need to be clearly defined and test scripts created ahead of time, to enable continuous testing of code at all stages of production.

    Optimize Testing Processes and Test Coverage

    Save time, money and resources by testing only what you need to test. Visual models allow various paths to be explored and optimized so that only your tests provide maximum coverage, without unnecessary testing.

    It is also recommended to import test cases, remove duplicates and distribute and allocate optimized test cases in existing tools.

    Shift-Left Your Testing (and Shift It Right as Well)

    What do we mean by shift-left testing? Tests should be run earlier in the development cycle. Developers test as they go, with test automation including functional testing, performance testing, monitoring and security testing. This requires developer-friendly tools for easy adoption and use.

    By shift-right testing, we continue to test through production and post-production, including continuous monitoring. Gartner recently released a report on the shift-right testing, calling it a “must” for adopting DevOps practices.

    Provide Complete Test Environments

    The ability to provide virtualized test environments is critical to achieving continuous testing. Eliminate blocks and reduce wait times by providing complete test environments on demand, with dev-friendly tools (as code, CI/CD integrations, supported open source).  These environments should include test data on demand to ensure that teams can perform comprehensive tests with production-like data. These environments should be ephemeral; ready on demand, decommissioned after use.

    Get the Right Test Data

    The inability to get robust test data causes significant delays in many application release cycles. To accurately test new functionality, test data should be as close as possible to what the application will encounter in production. If the test data lacks certain real-world characteristics (i.e. actual fields, data specifications, negative scenarios, etc.), the tests are unlikely to find many potential issues or break the application where there are weak points. Understanding data models in order to extract the right data is a special skill in and of itself.

    Production data is clearly the best data to use for testing, but data privacy regulations and team silos often restrict its availability. The answer to this dilemma is a robust test data management tool that lets you copy production data and mask it to protect sensitive information, while maintaining the characteristics that make production data desirable for testing.

    Remember, continuous delivery and continuous testing are processes that can take time to implement successfully, but teamwork, continuous feedback based on data and the right tool set will help you on your journey. You can read more about implementing a continuous testing policy here.

    — Sushil Kumar

  • The 5 Core Principles of Lean Management

    The 5 Core Principles of Lean Management

    Lean project management is rooted in cost-saving manufacturing methodologies. The ultimate goal of lean methodology is to reduce waste. Companies in all industries have adopted lean principles into their own processes, but lean is most suited toward physical manufacturing, engineering and related industries.

    Regardless of your company’s size or the activities of your organization, lean project management principles can be extraordinarily helpful in optimizing your everyday operations.

    Below are the five core principles that guide lean thinking for project managers.

    Identify Value

    Defining the value of a given product is crucial. Before beginning any manufacturing project, or any project for that matter, the amount of value a finished product will generate must be assessed accurately. How much customers expect from a given product must also be ironed out.

    Value questions that lean teams will ask:

    • What do customers want?
    • How will they benefit from our product?
    • How much are they willing to pay for the benefits of our product?

    Lean teams can, for example, estimate how much customers are willing to pay based on related data points, then set costs low enough to reach an agreeable price. The goal is to eliminate all waste in every process. Any area where value is not added—or worse, cost is exceeding value—must be addressed early on using lean management principles.

    Map the Value Stream

    The value is the end goal of the project. Once you’ve established value as a definite endpoint, you can begin mapping the way to get there (or value stream). Every single process and every single step involved in getting to the end product is part of the value stream. From gathering the materials needed for production to coordinating efforts with PR, every related process is represented in the value stream.

    Kanban is one of the most popular ways to map value streams. Tasks are broken down and are posted in a “just-in-time” fashion. That way, each task is taken one at a time and only done when it is absolutely necessary to complete.

    Using Kanban flowcharts and other diagrams, managers can set up visual aids for value streams, mapping every single step onto a single page. By mapping the flow of activities, lean project managers can easily identify waste and inefficiencies with the process, allowing them to ameliorate these issues swiftly.

    Create Flow

    By mapping the value stream, project managers can spot waste. Any tasks that are done too early, too late or too often creates waste. By analyzing the map and removing waste, lean project managers can create the optimal flow of activities.

    Lean project managers check for possible bottlenecks and delays. If there are any inefficiencies present, tasks may have to be reordered. Sometimes, project managers may even decide to append value-adding tasks to improve processes.

    Ideally, every step should flow into the other, beginning at the factory (or organization) and arriving smoothly as an end product to the customer.

    Employ a Pull Approach

    Many organizations use a push approach, stockpiling supplies in anticipation of demand. Lean organizations, however, employ a pull approach, wherein production increases as demand increases. Customers can “pull” directly from the manufacturer, getting what they need in an on-demand fashion.

    The pull approach requires extremely accurate market research and exceptionally efficient dissemination of information within the organization. Also, all processes must be streamlined; otherwise, it will take too long to ramp up production to meet demand.

    Seek Perfection

    At its core, lean management is about continuous improvement. Lean management requires a thoughtful approach to every action. Optimization must be part of the corporate culture itself. Every member of the organization must be working toward removing waste, adding value and delivering quality. There is always room for improvement in lean organizations.

    Conclusion

    Lean organizations are removing waste. Lean project managers ensure that activities within the organization flow properly, without interruption, delays or bottlenecks.

    Identifying value means finding out what your product will do and how much customers will pay for it. Value streams are then mapped, following every step in production. Project managers ensure that tasks flow and processes run smoothly. Market data is collected during development and these data points drive production. The most important principle that guides lean thinking is to seek perfection. The lean management philosophy demands a dedication to continual improvement.

    — Ellie Martin

  • F5 Networks Acquires NGINX to Meld NetOps with DevOps

    F5 Networks Acquires NGINX to Meld NetOps with DevOps

    As part of an effort to better align DevOps and network operations (NetOps), F5 Networks plans to acquire NGINX, a provider of widely employed open source loading balancing software.

    The two companies revealed in a call with financial analysts that the primary goal of the $760 million deal is to meld the two companies’ combined expertise to develop application delivery controllers (ADCs) that are optimized for modern applications based on microservices.

    F5 Networks CEO François Locoh-Donou said F5 Network plans to continue operating NGINX as an independent brand in addition to maintaining support for the NGINX open source community. NGINX CEO Gus Robertson will become general manager of the NGINX business unit. NGINX founders Igor Sysoev and Maxim Konovalov also plan to stay on.

    Locoh-Donou also disclosed that while NGINX software is employed today at more than 375 million sites, the company only generated $26 million in revenue in 2018, mostly by providing commercial support to enterprise customers. F5 Networks will move to combine investments both companies have been making in containers to deliver a next-generation ADC optimized for microservices-based applications spanning multi-cloud computing environments, he said.

    As part of that strategy, however, both companies will need to come to terms with Envoy, an open source load balancing project led by the Cloud Native Computing Foundation (CNCF). Envoy is already providing the foundation for multiple service mesh projects that make load balancing a feature of a higher-level mechanism for managing services running on a Kubernetes cluster. Obviously, F5 Networks hopes to fulfill an opportunity to manage multiple instances of service meshes running on multiple Kubernetes clusters that.

    In fact, Locoh-Donou cited industry research that forecasts by 2022 there will be more than 198 million workloads running on some form of a microservices-based architecture. In contrast, 44 million workloads are expected to be running on legacy platforms.

    Locoh-Donou said F5 Networks also expects to leverage NGINX to expand its presence in application programming interface (API) management and associated web application services, which he noted collectively represent multibillion-dollar market opportunities.

    The decision to acquire NGINX reflects the inevitable melding of NetOps and DevOps, he added, noting that inconsistent approaches to managing application service across emerging microservices-based environments are a recipe for creating unstable IT environments.

    While it’s still a little too early to determine to what degree NetOps will be subsumed into DevOps processes, Locoh-Donou said F5 Networks plans to advance adoption of NGINX software by educating its core base of NetOps administrators on how microservices applications are deployed across extended networks and the rigors of the DevOps processes required to manage them.

    Both F5 Network and NGINX made a name for themselves by interconnecting IT environments made up largely of virtual machines. While virtual machines are not going away anytime soon, so-called cloud-native applications built using containers running on Kubernetes clusters require a different approach to delivering infrastructure services. The challenge facing the combined company is finding a way to deliver those services in a way DevOps teams want to consume.

    — Mike Vizard

  • The Linux Foundation Launches Continuous Delivery Foundation

    The Linux Foundation Launches Continuous Delivery Foundation

    The Linux Foundation announced it will provide the home base for a vendor-neutral Continuous Delivery Foundation (CDF) committed to making it easier to build and reuse DevOps pipelines across multiple continuous integration/continuous delivery (CI/CD) platforms.

    The first projects to be hosted under the auspices of CDF, which was launched at the Open Source Leadership Summit conference, includes Jenkins, the open source CI/CD system, and Jenkins X, an open source CI/CD solution on Kubernetes. Both were developed by CloudBees. Netflix and Google, meanwhile, are contributing Spinnaker, an open source multi-cloud CD solution, and Google is also adding Tekton, an open source project and specification for creating CI/CD components.

    Additional projects are expected to join CDF once a Technical Oversight Committee (TOC) is formally named.

    Founding members of the CDF include Alauda, Alibaba, Anchore, Armory, Autodesk, Capital One, CircleCI, CloudBees, DeployHub, GitLab, Google, Huawei, JFrog, Netflix, Puppet, Red Hat, SAP and Snyk.

    Chris Aniszczyk, vice president of developer relations for The Linux Foundation, said that while a lot of progress has been made over the years regarding adoption of CI, the challenges associated with mastering CD have proven more vexing. To address that issue, the CDF will evangelize CI/CD and DevOps methodologies, define and document best practices, provide guidelines and create training materials to better enable organizations to implement CI/CD best practices.

    Kim Lewandowski, a product manager for Google Cloud, said making it possible for organizations to reuse DevOps pipelines is a high priority. Building pipelines is a significant challenge for most DevOps teams; as such, they need to be able to reuse those pipelines across as many CI/CD platforms as possible.

    Kohsuke Kawaguchi, creator of Jenkins and CTO at CloudBees, noted the creation of interoperability standards between CI/CD platforms will go a long way toward achieving that goal.

    It’s not clear yet just how widely the CDF will be supported by the entire DevOps community. While CloudBees provides one of the most widely employed CI/CD platforms, other offerings have started gaining traction, especially among enterprise IT organizations that tend to not have a lot of internal DevOps expertise.

    On the plus side, the CDF should help spur adoption of DevOps best practices even further. Adoption of DevOps processes within most enterprise organizations tends to be uneven at best. One team may be employing DevOps extensively, while the rest of the organization continues to rely on less agile legacy processes.

    The CDF also should play a significant role in enabling organizations to embrace microservices-based applications built using containers. While Jenkins is the most widely relied-on CI/CD platform today, it is built on what is now considered in many quarters to be a legacy Java Virtual Machine (JVM) platform. In contrast, Jenkins X Background an open source CI/CD platform for building and deploying modern cloud-native applications on Kubernetes. The Linux Foundation also plays host the Cloud Native Computing Foundation (CNCF), which oversees the development of Kubernetes.

    It still may take years for IT organizations to fully make the transition to DevOps and cloud-native applications. But in the months ahead, it’s clear the CDF intends to function as a force multiplier in enabling organizations to achieve that goal.

    — Mike Vizard