devopsy.kinsta.cloud

Category: Features

  • Effective Cloud Security Requires a Cloud-Native Mindset

    Effective Cloud Security Requires a Cloud-Native Mindset

    Thousands of developers and cloud security professionals will fill the Moscone Center in San Francisco next week for the sold-out Google Cloud Next ’19 conference. Three days of breakout sessions and keynotes will immerse attendees in deep technical content and opportunities for hands-on learning. It’s all designed to help organizations embrace the cloud. A key element of succeeding in the cloud is cybersecurity. More critically, the cloud demands embracing a cloud-native approach to security challenges.

    Leading the Way

    Since its inception, Google has been pushing the envelope and blazing trails in the cloud. Like Netflix, Google found itself—and often continues to find itself—in uncharted territory, where it was forced to innovate original solutions to address unique challenges.

    To help create standards and tool sets for managing container environments, Google joined the Cloud Native Computing Foundation (CNCF)—a melding of minds between developer communities and platform vendors—as one of the founding members along with CoreOS, Docker, Cisco, IBM and others. Famously, Google also originally developed Kubernetes—an open source container orchestration system for automating application deployment, scaling, and management.

    Google eventually donated Kubernetes to the CNCF. Transitioning it from a Google project to a CNCF project helped it gain broader traction and momentum and emerge as the de facto standard for container orchestration. Kubernetes became the first CNCF project to graduate out of incubation.

    Cloud-Native Mindset

    Kubernetes is a powerful, scalable platform for deploying and managing container environments of all sizes. Still, security needs to be a core focus for organizations, even in a Kubernetes ecosystem. The nature and volatility of container-based applications and cloud environments requires security to be cloud-native as well. Deploying containerized solutions on Kubernetes requires security that is built in to the distributed infrastructure and puts the focus on API security.

    Automation is Key

    The word “dynamic” is a very polite way to describe most cloud environments; volatile or chaotic might be more accurate, in many cases. Cloud infrastructures typically leverage DevOps tools and principles to compress development and deployment time frames and expedite the process of iterating new versions with continuous integration (CI) and continuous deployment (CD).

    The thing is, when development is driven by a CI/CD pipeline, security has to be as well. Manual processes can’t possibly keep up with the volume of assets that need to be monitored and protected, and traditional cybersecurity tools are not equipped for hybrid or multi-cloud environments. What’s more, they lack the scalability necessary for effective cloud security. Implementing and managing security in an environment that is constantly shifting and evolving requires an automated, cloud-native approach.

    Security in this context should be managed via API and also optimized to test API-based applications. Speed is crucial as well, though. Security should not break or impede the CI/CD pipeline due to timeouts or delays.

    Kubernetes clusters need cloud-native security tools to automate application protection. Tools capable of protecting containerized applications, microservices and APIs across public and private cloud environments while minimizing false positives at the same time.

    Conclusion

    There will be a ton of great information shared at Google Cloud Next ’19. While you’re networking, gathering insights on cloud strategies and learning how to solve problems in the cloud, don’t forget the crucial role of security. There are a variety of sessions offered that focus on security—how to manage, deploy, and develop security models for the cloud. Or, take a walk through Google’s dedicated Security Area in Moscone Center’s South Exhibition Hall. It’s a great place to connect with experts and educate yourself on why cloud-native solutions are essential for effective cybersecurity in a cloud ecosystem.

    — Tony Bradley

  • DevOps Chat: Chef Goes All-In on Open Source

    DevOps Chat: Chef Goes All-In on Open Source

    In front of next month’s ChefConf in Seattle, Chef has announced a major refinement to its business model. Affirming and clarifying its commitment to open source business models, Chef will now and in the future release all of its software as open source.

    In this DevOps Chat, we speak with Corey Scobie, SVP of Products and Engineering at Chef. He explains that Chef will be adding services, training and support to the open source software, as well as packaging all of this together.

    For the full story, be sure to listen to the podcast and check out the video version of this interview below, along with coverage on DevOps.com.

    [videopress nUXwRMxU]

    Transcript

    Alan Shimel: Hey everyone, It’s Alan Shimel, DevOps.com, and you’re listening to a DevOps Video Chat. Today’s DevOps Video Chat features Corey Scobie, SVP of Engineering, is it?

    Corey Scobie: Product and engineering, yep.

    Shimel: Product and engineering now at Chef. Hey, Corey, welcome.

    Scobie: Thanks, Alan, happy to be here.

    Shimel: I’m happy to have you here. Today is a red letter, big day in the history of Chef. You guys are announcing some pretty significant changes to the model is I guess what I would call them. Though, I don’t think the changes are as big as some may initially feel, but let’s go over them. I’m going to let you take the wheel here. Tell our audience.

    Scobie: It is a big day today for us at Chef. We are announcing some changes to the way we both produce and ultimately distribute our software, which I think are really exciting. There’s a lot going on in the industry right now. As you probably know, Chef has been one of the marquee players in the open source software development community for some time. Part of the announcement that we’re making today that I think is most exciting, is we’re announcing that what we’re going to do is going forward, produce 100 percent of our product-oriented software in that open source model.

    For us, that’s a change because we, over time, and through the addition of additional projects and additional software at Chef, had started following a little bit of an open source, open core path, which means that we had some proprietary software and some open source software, but when we got down to it and we thought about the history of Chef and the roots of what has made Chef such great software in the past, it really came down to collaboration. We realized the best way to make software is to make it in collaboration with the people that actually use the software, at the end of the day.

    For us, that’s the big first talking point about the announcement is that we are committed to 100 percent open source software development going forward. We’re committing all of our software to open source repos and GitHub, and then ultimately attaching it and giving it to the commons with the Apache 2.0 license.

    Shimel: Sure, so Corey, you know – so you’re talking to someone who has a bit of a history in open source, 20 years. I remember when the whole open core movement first came out. Look, Richard Stallman and that crew, they were really against open core, because they felt it was – it was almost heresy to the whole cathedral and bizarre open source kind of crowd, where if it’s open, it’s open. You can’t be a little bit pregnant. But, nevertheless, we’ve seen companies succeed with this open core or premium, where the core is free and there are premium modules on top of it, but it’s a slippery slope. In all honesty, I think Chef got caught in a little bit with that slippery slope with what’s truly open, what’s free, what is our proprietary that we make money from? I don’t know if the market really got it.

    Scobie: Yeah, one of the things that is really tough when you’re following an open core strategy is drawing a very clean line between what is free and what is not free in the world of your technology offerings. For us, that got complicated and then even conflated further as we added more projects to the list. When we opened the InSpec project for security and compliance, when we opened Habitat for application automation, what ended up happening is we had multiple projects that were governed different ways that had different lines between the free and the core part of it. I think ultimately at the end of the day, had I been at Chef during that part of the journey, I probably would have made the same decisions, I’m guessing, but at some point you get back to this reconciliation point where it’s like, “Look. The best thing that you can do for customers if number one, you can collaborate with them, and number two, you can have a really, really clean line between what is free and what is not free in your world.”

    For us, it becomes super simple with this announcement. What is free is all of the intellectual property that we’ve contributed to the commons in the world, and will contribute to the commons going forward. What isn’t free is all of the work we do to make a turnkey, enterprise-class, package that enterprise-class customers can pick up and distribute at hyperscale in their organizations, from that point.

    Shimel: That brings us to the next thing, and of course this is always the issue. So if you’re going to make all the software and the IP open, well how the heck do you make money?

    Scobie: Sure. As it turns out, we’ve got venture capital funding, investors that want to see us turn that into dollars as well. For what it’s worth, we’re feeling pretty good about the shape of the business right now. We just came off the best quarter of revenue we’ve ever had at Chef, prior to this business model, this model change around how we distribute software whatsoever. So we’re feeling really good about the shape of the business, but just the inability to have that simple conversation about what’s free and what isn’t. What we decided was as we’ve matured as a company, and as the space has matured in enterprise automation, what we found is the majority of our customers actually do want to have a very material commercial relationship with us.

    Not only do they want the software, but they want the support. They want the content. They want the expertise. They want the warranties and indemnities that go with all of that. So for us, it was very simple. We realized at the end of the day, that the most — that there was a huge constituency of our customers who what they wanted from us was a very curated commercial offering out the other end. So we’ve decided that’s where we’re going to put our effort and our energy, and we’re going to trust in the open source world to fill in the gaps where that isn’t the offering that they want as well.

    Shimel: We look at this model, Corey. Obviously, Red Hat’s the poster child for much of the open source world in that most of their software, if not all of it, was in fact open source. It was all in the packaging and the support, and that kind of thing. It sounds very much like that’s going to be the model, or am I wrong?

    Scobie: It is – no – there’s a strong analog to what Red Hat has done in the industry as well. What’s interesting is I think that that is the antithesis of what we are seeing from a lot of the more recent moves in the open source industry where there’s a desire to become more proprietary in some cases, or in other cases to try and somehow bend, I think, the definition of what is open source software. For us, the core of it, we get back to the four freedoms, and freedom zero is about the freedom to use the software, but it’s free as in freedom. It’s not free as in dollars.

    Shimel: Free as in freedom, or free as in beer, as we used to say, but Corey, to be fair to these companies that are – that we do see going another way, it’s because they didn’t cut it. They tried the Red Hat model, and for whatever reason, they didn’t cut it and they had too much pressure from their boards and the VCs and investors, and so forth. So they tried to take a path that was quicker to money. I’m not here to judge anybody’s – as someone in the Godfather once said, “Far be it from me to disparage what someone does for a living” –

    Scobie: I hold the same opinion, Alan, by the way. I sit in judgment of nobody on the decisions that they make about what’s right for their business. I just know that for us, what’s right is to try and both honor the open source roots that we came from, and to also serve the commercial customers that were really knocking at the door and saying, “Look, this is what we want from you. We want even more engagement. We want even more value out of your commercial subscription.”

    Shimel: Agreed. Before we jump into the next part of it, which is how it manifests itself within the Chef universe, I wanted to just tie a knot on this one, and mention that when we talk about this model, and you said, “Oh, we’re going to count on the open source community to fill in the gaps,” and one might say, “Well, what are those gaps? I think people have to realize what made Red Hat great was not just distributing Linux. It was that RHEL, and Fedora even before that. It was that package, that distro, that had everything you needed and wanted in there in a really easy to use – you didn’t want for anything else. It was all right there, nicely done for you, instead of running out and having to download different packages and distros from all over the internet, which is what held back Linux on the desktop for so many years. There wasn’t that one package that you could do that.

    I think that’s what you’re referring to is Chef’s strategy going forward is to have that killer distro, with all the right packages in it. If you don’t want to pay Chef for that, and I don’t think at this point, though people like software that is free as in beer, and they get free as in freedom with it, no one denies that a company should be able to make a living, that a company should be able to make a profit, pay its employers, pay its developers, and go forward. So for people who either don’t have the money, or don’t have the inclination to buy the packaged product, they can get the open source components, and the community can work on giving you the glue, in essence, that puts these all together. Correct?

    Scobie: Absolutely, for sure. Santos came out of that exact scenarios. The scientific community needed something that wasn’t going to be necessarily nearly as curated as Red Hat and Linux was, but they needed a platform for scientific computing, so CentOS was born and has a place in that world. I think our vision for that is exactly the same. If there’s a downstream project that is born of Chef InSpec or Habitat, we welcome it into the community, and we look forward to whatever innovations they bring, and we want to collaborate with anybody that wants to come and take that work out of the commons and create the derivative work out of it, that allows them to meet whatever their goals are. That is the freedom in free and open source software.

    Shimel: Absolutely. I think one of the other things though, Corey, is when we look at Chef over the last couple years, we’ve seen the whole InSpec line of security functionality, and we’ve seen Habitat. Look, Habitat was an ambitious piece of – well, it was more than a piece of software. Habitat was an ambitious play, but because there was so many balls in the air and going off in different directions and open/closed, proprietary/not, people never – it wasn’t explained well enough to people. I spent the time at ChefConf so I understood Habitat well, but there’s maybe a thousand people at a Chef conference as opposed to millions of people in the world. I think it was really hard getting InSpec and Habitat, carving out their niche in the world, amidst this whole confusion around the closed/open, supported/paid for, free/not free.

    Scobie: Yeah, that’s fair, and I think this really helps us simplify the messaging around that too. I will openly admit I think InSpec is an easier basic value proposition to parse, which is “Look, I need security and compliance in my organization and the right way to do that is express it as code”, so that should be fairly straightforward, and relatively noncontroversial. I think, with respect, to Habitat, what is interesting about Habitat is I agree with you, super ambitious project right from the get-go to redefine the abstraction of packaging and orchestrating the delivery of software in the world.

    That’s – it’s huge that there’s a huge surface area associated with that. I think what’s interesting is that we’ve learned much – we’ve learned a lot over the last couple of years about how to help people in the mental journey of how to think about Habitat in the world, and how it relates to Chef, and having that foundation of understanding and knowing Chef, which tens of thousands, if not hundreds of thousands of people already have, the ability for them to jump to where Habitat plays becomes much easier in the world where we can talk about them collaterally and not independently of one another.

    Shimel: Fair enough. I think that’s a great segue, Corey, to the next part of our conversation I wanted to have with you, and that is okay, how does this manifest itself down the line of Chef product and services? So we’re going to put all our IP out as open. I guess we should mention also, I think everything is being released under an Apache 2.0 license.

    Scobie: That’s correct. Apache 2.0.

    Shimel: Industry standard, GPL, Apache are probably the two biggest. How does this manifest itself? What changes does this mean for Chef’s products?

    Scobie: On flag day when we release new products with new packaging and new terms and conditions, it’s an evolution that we’ve been going through for some time now, which is really to understand what the interface parts are between the three core engines that we have. So we’ve created an engine for infrastructure automation, which is the Chef project, and going forward, just from a purely – from a branding simplicity perspective, it’s going to be known as Chef Infra, which is I think is fairly straightforward.

    Shimel: I’m sorry. Can you repeat that though? Chef…?

    Scobie: Infra, as in infrastructure.

    Shimel: I-N-F-R-A. Okay.

    Scobie: Yeah, and then we’re going to have Chef InSpec, for security and compliance, and the state relationship between infrastructure and security and compliance are pretty – it’s pretty intuitive, which is that if I deploy an operating system, and I want it to be locked down in a certain way and secure and meet a certain set of compliance standards, then testing that on a continuous basis, managing the drift, and being able to automatically remediate any drift that’s happening there seems like a really obvious relationship. What’s interesting though is the relationship between infrastructure and application automation. This is where things get really murky, even for longtime Chef customers who have been wildly successful in using Chef as an infrastructure automation tool. If you go and talk to those customers, what you’ll find is that they’re doing about – 30 percent of what they’re doing is about infrastructure deployment provisioning and shaping, locking down configuration management.

    70 percent of what they’re doing is deploying software on top of the operating systems in service of some application that’s going to come along at some point in time and need that – those software dependencies to be there. What we discovered, and this is really the genesis of Habitat, which I’m not even sure – we knew that what was happening in the space of Chef was not working at scale for us. That’s the reason that some folks at Chef went off and decided to create Habitat, but what we didn’t realize is why it wasn’t working as it was at the time. The reason is that there is a natural boundary for building systems from the bottom up in a layer cake approach. It doesn’t matter whether it’s Chef or Antival or Puppet or Kubernetes clusters or whatever, that there is a natural boundary layer, if what you’re doing is building systems from the bottom up.

    So you have an operating system, and then you have system libraries, and then you have packages, and then you have middleware, and eventually you have an application. At some point that dependencies graph, the modularity of everything that you lay down, and the dependencies graph gets so complicated that there’s no possibly way you can manage understanding what the implication of changing component X is on all the other components that rely on X. So the beauty of Habitat and where Infrastructure, and where Chef and Habitat come together is that Habitat takes the exact approach. It says, “Look, what you should do is you should define an application, all of the application’s dependencies and all of the orchestration steps for that application to run successfully in any environment regardless of where it is.”

    So by doing that, and by removing that layers – that bottoms up layers dependency, what we’ve really done is created an application stack, an enterprise automation stack is what we’re calling it where infrastructure and application automation marry together at a very, very clean abstraction level. The abstraction level is essentially, if you’re running software on the operating system that runs above the kernel, it should be defined by application packages and orchestration, and not by infrastructure packages and orchestration. Now, it’s architecturally it all makes sense, and it actually works really well in practice. Where it’s complicated is that organizationally, INO – traditional INO operators and other infrastructure groups have been tasked with more and more software deployment on top of their stacks. So this is a very paradigm-shifting idea. I think what is helpful for us is the enterprise automation stack allows us to create a framework and a template for how to do that in a way that helps customers adopt that methodology much, much quicker.

    Shimel: Fair, fair. So we got Chef Infra. We have Chef InSpec.

    Scobie: Yes, and Chef Habitat, Chef Habitat.

    Shimel: Chef Habitat sits – I wouldn’t say it sits on top of them.

    Scobie: Look, the stack is – we have trouble with this with our marketecture diagrams. If, what happens in the podcast now is you’ll throw up the marketecture diagram you’ll have, what you’ll see is a linear stack, but the state relationship between those three things is much more complex than can be defined in a diagram like that.

    Shimel: _____ diagram, sure.

    Scobie: Then on top of that, then there’s two other pieces to the portfolio. So we have Chef Automate, which is the observability layer on the top. Of course, if you have automation on all of these things, you want to be able to see across all of your pipelines, across all of your applications, across all of your systems. What is the state of everything that’s running? How has it been deployed? Is it in compliance or not in compliance, etcetera? So we’ve got this whole enterprise dashboard with automation up top. Then on the bottom end of the stack is we need to make it easier for developers to get started. So all of these engines are representative of either infrastructure compliance or application automation as code. What do you do as a developer? You download a bunch of tools. You get started writing code that are the artifacts that actually drive that automation. So Chef Workstation is the compilation of all of those tools and an easy way to set up your development environment to get going. That’s the entirety of the enterprise automation stack. It’s Chef Workstation, Chef Infrastructure – Chef Infra, Chef InSpec, Chef Habitat, and Chef Automate on top.

    Shimel: And repeat for me, what all of these packaged together is the –

    Scobie: Yeah, the Chef Enterprise Automation Stack.

    Shimel: That’s the whole enchilada, so to speak. This is all packaged up into one coherent type of product, if we can call it that. Fair?

    Scobie: Exactly. Yeah, one coherent distribution, absolutely.

    Shimel: Corey, you know the news broke today around this. Some are saying, “Well, Chef is pivoting. Chef is this.” How do you view this? How do you and the other folks at Chef view this? Is this really a pivot? To me, it’s – I don’t see it as a pivot.

    Scobie: Yeah, it’s not a pivot. I mean, first of all, I don’t think it’s – we’ve – you can question the effectiveness or the efficacy of our open source strategy in the past as we’ve meandered through different phases of that, but this is us staying true to the fact that we are an open source company, and so I think we’re leaning into the business model by which Chef was founded, and by which we’ve been operating for quite a while.

    Shimel: I agree with you.

    Scobie: I think from a technology perspective, we just have more clarity now in how to deliver the components and technology that have become critical to customers who are trying to solve large-scale enterprise automation problems. I don’t know that I would really describe it as a pivot. I think what we’re trying to do is use this as an opportunity to reintroduce people to Chef who maybe haven’t heard of the latest of our innovation in InSpec and Habitat, as an example, who maybe are not familiar with the work that we’ve been doing in the last three or four years. For us, it’s a little bit of a rebirth, but I wouldn’t call it a pivot.

    Shimel: I don’t know if I’d call it a rebirth. So here, look, you’re in the forest.

    Scobie: I am in the forest. There’s no question.

    Shimel: [laughter] I’m outside the forest. To me, obviously a pretty – I’m more than a casual observer. To me, Chef was always an open source company.

    Scobie: Always.

    Shimel: I think in some ways, what we’ve seen with Chef over the last five years, I would characterize as growing pains for an open source company trying to come to grips with remaining true to its open source roots, while at the same time recognizing you took other people’s money, and that’s how – I lived that life for 20 years. You take other people’s money, they want – it’s a funny thing. They want to get a return on their investment. It’s like trying to serve two masters. It’s a hard – it’s a hard –

    Scobie: Not only that, but look, the successful commercialization of any open source project or software company is how you actually create sustainable good in that community for the long-term. Obviously, you need participation from the community, but you also have to inject capital into that community if you want sustained innovation.

    Shimel: You have to extract revenue as well though. That’s where I think it gets dicey. So when I look at InSpec and Habitat, I almost feel at some level, they were somewhat hobbled by having to live in this schizophrenic, open/not open, making money/not making money. Well, if it’s totally open, we’re not making money. How much do we put into this? There was just too much of this all over the place, where to me, this is just a clarification. Look, Chef is and will remain an open source company, and our value to our investors is that No. 1, we have this tremendous community around us who are users of our software as well as contributors to the community, but No. 2, we are now offering a proprietary, a for-profit solution that takes the best of this and packages it up nicely.

    Scobie: Absolutely. Look, Alan, you make a great point. I would love to give a shout out to the community, because the Chef community is absolutely amazing. I know you’ve been to ChefConf. It’s an experience in and of itself. It’s hard to think of another tech conference that is as authentic and as welcoming and as diverse as ChefConf is, and it’s a bunch of people that come together every year, because they are passionate about solving the challenges of infrastructure management at scale, and application deployment at scale, and all of those things that Chef is interested in as well. So the community is a really key factor of it, but to your point is that there’s just this growing demand in the enterprise. We are the experts in figuring out how to do that in a really, really tight curated way. Of course, we’re going to take advantage of that market opportunity as well.

    Shimel: Sure, we got the news out. Is there any part of the news we’re missing, Corey?

    Scobie: No, I don’t think so. That really covers the announcements for today. I think the only other thing is we’ve got ChefConf 2019 coming up in Seattle May 20th – 23rd in Seattle. It’s going to be a great event. We also have a ChefConf for the first time ever, ChefConf coming to London at the end of June. It’s June 19th and 20th, I believe are the dates in London.

    Shimel: Really?

    Scobie: Yeah.

    Shimel: You know I’ll be in London those dates. I’ll have to talk to you about it.

    Scobie: Yeah, absolutely. We’d love to see you there, and we’d love to see others there as well. Yeah, so we’ve got a couple big events coming up where we’re going to showcase the latest of the innovation that we’ve had on all of the fronts, innovation, compliance, application automation, our enterprise consoles, all of that kind of stuff. so those are going to be fantastic events coming up. So other than that, we’re looking – we’re really looking forward to getting back to those community roots and really everybody being there together in the trenches together building the next generation of what this is all going to be.

    Shimel: Cool. So now let me ask you a question. Put on your wizard’s hat. If we’re talking about ChefConf 2020 now, a year from now, this strategy, this road has been – this strategy has been implemented now. We’re going down this road. How does it manifest itself over the next 12 – 24 months you think?

    Scobie: Look, we’ve been talking about that from a product strategy perspective, and I think we’ve got – there’s a lot of opportunity for us to do great work in pulling the pieces of the puzzle together, even more explicitly. We’re doing a lot of work with partners, so I anticipate that the ecosystem partners will be a big player in ChefConf 2020 as well. So we’ve been working with the likes of everybody from Microsoft, Azure DevOps, and the workflows there. We’ve got – we’ve been doing some product work, and continue to do product work with Amazon in the ops work for Chef Automate, managed and hosted to platform there. We’re quite engaged with GitHub because we’re seeing both GitHub and Artifactory and a lot of our customers in the DevOps toolchain and pipeline there. We’re doing some work with that. We’ve been doing a lot of integration with services now. So I actually think that the next phase of this is not just about the Chef parts of the stack, but how those parts of the stack connect with the rest of the ecosystem that is fast emerging in the enterprise automation space as well.

    Shimel: Sounds cool. Well, Corey, we’re about out of time. I want to thank you for taking time on what’s a crazy busy day for you guys, to brief our audience on this. Congratulations to the whole Chef team. Tell Barry and everyone else, looking forward to ChefConf. Unfortunately, I won’t be at ChefConf in Seattle. I’m in Europe, but we’re going to have a full team there doing video interviews there and everything. John Willis, our partner at Digital Anarchist will be there, and Chris Reilly. So we’ll have plenty of Chef coverage from ChefConf. Like I said, I didn’t even realize the London one. We’ll try to get over there. I think that’s Tech Week in London actually, which is a pretty big week.

    Scobie: Yeah, fantastic. Well, thank you so much, Alan. I appreciate you taking the time today as well. Look forward to seeing your colleagues at ChefConf in Seattle, and then hopefully we’ll see you in London in June.

    Shimel: Absolutely. Hey, Corey Scobie, SVP Engineering and Product. I got it right.

    Scobie: You got it right.

    Shimel: You’re talking to us about some news coming out of Chef today, a little bit of a change in how they package stuff, but they’re still an open source company. Corey, thanks very much.

    Scobie: Thank you, Alan.

    Shimel: Take care. This is Alan Shimel for DevOps.com, everyone. Have a great day.

    — Alan Shimel

  • Agile vs. ITIL: Two Divergent Paths to DevOps?

    Agile vs. ITIL: Two Divergent Paths to DevOps?

    Truth be told, there are dozens of different frameworks in existence and all have their unique perspectives on how a project is managed and executed. Frameworks such as Agile, ITIL, lean, cobit, six sigma and many others were built on an ideology of following a path defined by best practices.

    With DevOps, two of those frameworks, Agile and ITIL, have become the basis for arguments on how to do DevOps correctly. Yet, until recently those frameworks were very different in the paths offered to bring about project success. With the announcement of ITIL V4, things have changed a little: ITIL is becoming a little more like Agile, adding to the confusion of what path makes the most sense in the world of DevOps.

    A Quick Glance Backwards

    The foundations of DevOps were built upon the ruins of non-iterative application release cycles. Or look at it this way: DevOps is all about not taking a plunge over the waterfall. While those observations are little more than metaphors for something that rings true, DevOps has brought a new paradigm to software development and deployment. A new paradigm that comes with its own recommended practices.

    Yet, current DevOps practices—or let’s call them rules—can come up short in the modern enterprise, meaning that other management best practices—such as those illustrated in the Agile and ITIL methodologies—are borrowed to drive DevOps further forward. That said, the common DevOps ideology is built around a few foundational practices that define the framework of DevOps. The primary goal here is to bring application development and operations together as a functional team. Some of the generic best practices include:

    • Implement collaboration and teamwork tools to improve communications as well as improving the ability to share knowledge, ideas and automatically document interactions. Simply put, incorporating teamwork with an audit trail goes a long way toward bringing order to DevOps.
    • Incorporate tools to capture requests. One of the most important capabilities delivered from DevOps is the ability to adapt to change. Traditional application development frameworks are usually based on waterfall processes, where everything is defined at the outset and then an application is eventually delivered. Waterfall ideologies offer no leeway to adapt to change in the middle of a development cycle. DevOps eschews those limitations by incorporating the understanding that change is constant. However, requests for changes must be recorded, tracked and reported upon.
    • Adopt a framework ideology that can match workloads to team capacity. In other words, team leaders and managers need to have a clear view of what work is in progress and the context of that work as it applies to a team’s abilities.
    • Use systems to log metrics on automated and manual processes, which will give insight into productivity, as well as identifying where improvements can made.
    • Implement test automation. The idea here is to automate the testing of code using test data in a sandbox environment. Code that passes the tests can be forwarded further along the DevOps process, while code that fails is automatically returned to developers for correction.
    • Use a continuous feedback system in which a chain of communications can be established to keep developers and operators informed of the status of any issues, as well as change requests and the progress of any code through the framework.

    While the above best practices are simplified representations of a full DevOps framework, they do establish that DevOps relies heavily on collaboration and iterative processes to support the ideology of continuous development and deployment. Now, comparing the DevOps ideology to what constitutes either Agile software development practices or the ITIL Framework brings up some additional questions and establishes that Agile and ITIL may very well have different goals. A more simple way of looking at it is to think of Agile as a set of practices to provide faster software delivery, while ITIL is a framework squarely focused on processes and not practices.

    The Path Becomes Muddy

    Therein lies the real dilemma of establishing a path to DevOps using either Agile or ITIL ideologies. ITIL is designed from the perspective of the governance of IT service management and establishes a framework that supports continuous measurement and should deliver improvement of the quality of IT services. Agile is a set of processes designed for software development that meets the needs of customers and cross functional teams.

    Interestingly, Agile and ITIL prove to be complimentary in the world of DevOps, where ITIL can be used to ensure quality and value while Agile brings forth a continuous stream of improvements in applications. Ultimately, with the arrival of ITIL V4, Agile and ITIL will become more aligned with each other and offer benefits to DevOps.

    — Frank Ohlhorst

  • When DevOps and Marketing Collide: Insights From Adobe Summit 2019

    When DevOps and Marketing Collide: Insights From Adobe Summit 2019

    I attended Adobe Summit 2019 to research the intersection of DevOps and marketing. Below are insights from my interviews with leading marketing technology experts. 

    Is DevOps making its way into marketing? In a short answer, a soft “Yes.”

    I recently attended the Adobe Summit, held March 26-28 in Las Vegas. This year emphasized digital customer experiences, and I was curious to take a peek into this world of marketing tech to see if DevOps was having any impact.

    Many of the people I talked to described how marketing technology is adopting a more agile, DevOps-inspired mindset. But is marketing actually driving product change? While Adobe and proponents would like to have us think so, some are not as sure.

    At Adobe Summit, I had some eye-opening meetings with professionals invested in digital marketing, retail and analytics services, most of which are using Adobe products in some form. In this article, we’ll highlight what they had to say on the intersection of DevOps and marketing.

    What is the Role of DevOps in Marketing?

    The need to break down data silos was a common theme among keynotes, sessions and throughout my meetings at Adobe Summit.

    When large digital marketing agencies take on clients, for example, a common disability they find is a disparate and dysfunctional customer data storage system. In our meeting, Alex Kelleher, managing director at Deloitte Digital, noted, “Data needs to be available, and integratable. That is the first thing we typically fix for our clients—we’ll fix the data in silos, get it in the same place, and usually simplify it.”

    Deloitte Digital’s process involves three D’s: first, they consolidate data silos, then build data-driven decisions, then deliver agile products. In doing so, Kelleher is seeing more and more developer practices making their way into marketing and sales.

    “We’re seeing CMOs manage their teams as agile teams, recruiting agile and scrum masters,” he said. “That’s very cool for CMOs to be involved in … those worlds colliding is super exciting.”

    Is DevOps Eroding Traditional Boundaries?

    DevOps principles encourage shared knowledge and open communication between teams. Now, it appears such ideas are affecting how marketing technologists architect their teams and delivery cycles.

    Rachel Lane, solution principal customer experience at Medallia, sees a trifecta of forces working in tandem. “Development, Ops, and Marketing must work together,” she said. “If one of those doesn’t link up, then we see extended lead time to launch, and it can become problematic.”

    Medallia’s goal is to meet customers where they are to gather actionable feedback through push notifications, surveys, embeddable surveys, Net Promoter Score (NPS) systems and other means. Engrossed in this world of customer feedback, Lane reminded us: “You should never ask a question that you already know the answer to.”

    Unfortunately, this still happens often. User data is often locked in silos and if multiple teams (sales, marketing, support engineers, etc.) are using separate user data repositories, customer interaction can suffer. Thus, user conversation data must be de-siloed and unified.

    For Lane, having tight customer experience-driven feedback is critical for reiterating the overall product. However, customers typically hate filling out surveys. To decrease friction, she foresees more passive voice-driven feedback through home AIs.

    Rearchitecting the Digital Footprint for Agility

    By now it’s evident that unclear data initiatives and messy data storage can hinder digital transformation. So, what are organizations doing to de-silo and open up data?

    Anheuser-Busch (AB) is a huge beer corporation, overseeing brands including Budweiser, Busch, Elysian and many others. Each has unique online eCommerce storefronts to sell ancillary swag, such as T-shirts, drinkware and other products.

    With public-facing sites for hundreds of brands, maintaining data standardization across its product lists was a nightmare. Disparate data structures can also inhibit directing change from within, slowing corporate-led growth.

    John Faviano, director of demand and integrated marketing technology at Anheuser-Busch, described how AB solved the issue by creating a consistent data structure from which all online stores can pull content from.

    “We’ve standardized the tech stack in the last three years to enable the business to run more efficient … You have to think from a platform perspective, not a single siloed technology,” he said.

    AB’s web APIs now grant incredible scalability, as the content is now delivered agnostic of client or brand. AB brands retain independent facades yet are fueled by a unified content storage and delivery process. “Our API strategy is a build-once-and-scale mechanism,” Faviano said.

    Digitization with IoT and AI

    Sreedhar Bhagavatheeswaran, SVP and global head of digital business at Mindtree, believes that every part of the enterprise needs to be digitized. He sees a wealth of untapped data that enterprises can leverage to transform user experiences.

    Having built digital presences for big-name consumer brands including Marriott, P&G and Kellogg’s, Mindtree understands the advantages of using a DevOps approach for software cycles. Though it may require a cultural change, “The probability of project success in a DevOps environment is much higher than in a non-DevOps environment,” Bhagavatheeswaran said.

    Release automation is integral to Mindtree’s process. Coding checks and CI tools such as Jenkins bring quick releases, enabling business teams and clients to see the impact on a daily basis. They also privilege API-fueled microservices designs for increased agility.

    “From an agile program management perspective it gets challenging, but that is the nature of the beast,” he noted.

    Bhagavatheeswaran is excited for the potential IoT has to evolve industrial, business and consumer-facing touchpoints. For example, industrial engineers could conduct remote servicing using digital twins of real physical equipment. He also described developing mobile IoT apps for his clients: Push notifications triggered by removing caps from makeup bottles, for example, could empower consumers with advice on how to apply makeup correctly.

    To sum it up, Bhagavatheeswaran described our current paradigm with a simple, powerful sentiment: “The experience is at the confluence of edge, humanity and intelligence.”

    Approaching the Developer Experience

    But digital transformation doesn’t stop with open data, APIs and IoT. Developer experience (DX) has become an important factor in maturing countless digital services.

    Jason Woosley, VP of commerce product and platform at Adobe, founded the Adobe-acquired Magento, which began as an open source project on GitHub, welcoming pull requests to alter product features (with the proper vetting in place, of course).

    By investing in open source and developer communities, Magento over time reached a critical mass, democratizing how developers build online storefronts.

    Woosley estimated that 50 percent of Magento code was built by third-party developers. One such feature is multi-source inventory, which, he said, accounts for 70 percent of the Magento code base. Naturally, he has a deep appreciation for the power of developer communities.

    “You can’t buy a developer community. You can’t steer or create a culture,” Woosley said. “Code custodians are actually participants, and to deserve that role DX needs to be delightful and empowered.”

    To improve the developer experience, Magento has hosted countless events, increased developer support staff, annotated its APIs, adopted GraphQL for extensibility, invested in optimizing developer portals and more. With the inclusion of Magento in Adobe Experience Cloud, more capital is available to further DX enhancements—something Woosley is excited about.

    Wednesday March 27, 2019 in Las Vegas. (Jeff Bottari/AP Images for Adobe)

    DevOps is Kicking Martech to Innovate

    Deloitte Digital’s Kelleher noted that “nowadays, your competitors are any digitally active application,” regardless of whether they’re even in the same industry.

    This means that a consumer will judge a mobile digital experience with an airline with the same scrutiny as calling an Uber or making a PayPal transfer. Users hold the same lofty digital expectations from any business, forcing us all to adapt dramatically.

    Fueled by the relentless pressure from users, marketing technologies are adopting DevOps principles to improve their offerings in a few ways:

    • Continuous releases: Martech uses DevOps strategies to release quickly. User-facing mobile app release cycles, for example, routinely adopt CI/CD.
    • De-siloing: User data and consumer conversation information are being distributed with greater alacrity across enterprises, improving end UX.
    • Open data initiatives: With web APIs, new platform-agnostic content distribution architectures have emerged. Development procedures are standardized across an enterprise, quickening the release cycle.
    • Cloud architecture: By adopting cloud-based management tools, martech is improving the asset management condition of supporting many user-facing touchpoints.
    • Developer experience: By embracing open source and supporting developer community, leading e-commerce groups are improving their offerings.

    While marketing is critical to the whole, it may not be the No. 1 business driver. When I asked Bhagavatheeswaran whether marketing was really driving the creation of new tools, he responded: “Is marketing driving the tools? Are they influencing? Not quite yet. However, it has the ability to influence the developer’s quality. Probably not their job.”

    Even if marketing is not directly influencing product development, we’re still seeing DevOps seep into traditionally non-technical environments, which is exciting.

    It was eye-opening to enter the world of marketing technology to see firsthand how these institutions—such as Ansira, a 100-year-old company—are using agile means to streamline digital transformation initiatives.

    With an emphasis on both creating and managing quality digital customer experiences, Adobe Summit 2019 shed light on many unique strategies being undertaken in the realm of martech.

    Whether it’s leading the development of user-facing apps, overseeing expansive brand catalogs, using a cloud-based single source of truth for content management or building developer communities to reap huge R&D benefits, the intersection of marketing and technology has never required more strategic action than it does now.

    — Bill Doerrfeld

  • Modern Workers Demand Modern Technology

    Modern Workers Demand Modern Technology

    Companies today must think of their employees as customers and employ technology, including mobile apps, that employees want to use.

    I have always been a bit of a technophile. Many years ago, in another life, when I worked in technology sales, I was one of the first sales engineers on my team to buy a personal laptop to use as a way to manage leads and accounts. The advantages of doing so over pen and paper became clear to me within weeks. I was able to track my activities more effectively and follow-up more precisely than my associates.

    For similar productivity reasons, I also equip my home office with the best tools that make the most sense from a productivity standpoint. This has been true for both the years I worked out of my home office self-employed and as an employee.

    I expected the tech I bought for my work to always be better than the tech that was provided to me by my employers. I grew to dislike the outdated computers, software and support services provided by employers. Why didn’t these employers invest in their employees to make them as productive as possible? There was never a good answer to that question.

    However, in the late 1980s and early 1990s, I was an outlier. Most workers didn’t have personal computers and most appreciated whatever outdated green screen hardware and software was planted in front of them. Today, it’s the other way around: Employees don’t just want state-of-the-art technology that matches their work—they demand it.

    Consider a recent note from research firm Gartner that found outdated technology pushes Australian workers away from their employers. In fact, Gartner’s “Q4 2018 Global Talent Monitor” found that technology now ranks ninth in the top 10 reasons Australian employees will leave their current role.

    “People have become so used to advanced technology in their day-to-day lives, that they expect the same thing from their workplace. However, businesses are having a hard time matching the speed at which technology is adopted at home,” noted Aaron McEwan, human resources advisory leader at Gartner. “It’s not surprising that employees are becoming frustrated when they find themselves wasting valuable time navigating complicated systems and processes that utilize slow and old technology. It’s unproductive and inefficient for everyone involved.”

    According to McEwan, businesses no longer can ignore the needs of their employees and must start thinking of them as they do their customers, making it a priority to offer a personalized, seamless and efficient experience.

    This recent Gartner Australian survey taps into trends that have been underway for some time. A few years ago, Microsoft partnered with SurveyMonkey to conducted a survey of about 1,000 millennials in the United States. That survey found 93 percent of respondents said modern technology was one of the top aspects of a good workplace.

    This isn’t just about new hardware or off-the-shelf software. It’s also about custom-built enterprise apps. For decades it has been accepted practice that enterprise software didn’t have to compete with consumer software, and poor-to-lackluster design was considered acceptable.

    This is no longer the case. Most employees today are technically savvy and their enterprise apps are on their tablets and phones, right next to the consumer apps that they love to use.

    Enterprise IT can try to force them to use leaden software, but this strategy increasingly is unlikely to prove productive. If they don’t like a collaboration platform, they’ll turn to one of the widely available cloud alternatives. If they don’t like the enterprise CRM, they’re likely to turn to Salesforce or a competitor.

    When consumer apps are designed—think Evernote, Uber, Instagram or another favorite consumer mobile app—a considerable amount of thought is invested into the user interface and making the app enticing to use, designing an intuitive workflow and hiding complexity from the user into the background of the app. The same should be true for the enterprise apps designed for the enterprise.

    A recent survey from Adobe found that 61 percent of 1,500 enterprise leaders believe organizations that don’t deploy mobile apps are at a competitive disadvantage. I’d take that a step further and conclude that deploying isn’t enough. But it’s more than simply deploying an app; organizations that don’t deploy effective and compelling apps will find they are at a competitive disadvantage.

    — George V. Hulme

  • The New AI Overseer Role in DevOps

    The New AI Overseer Role in DevOps

    Robot overlords are the stuff of science fiction fantasy. Today, with more teams implementing artificial intelligence (AI), DevOps teams need a hard, continuous look at reality. An AI overseer watching apps at work could make the difference between sadness and euphoria in your efforts.

    We’re seeing AI misses more and more often. Facebook recently admitted its algorithms failed to stop uploads of a particularly heinous live-streamed video. Autonomous vehicles have struck cars and people. Jaywalkers in China have been misidentified and publicly shamed in error. It’s not for lack of effort in development. The adage “If it were easy, anyone could do it” applies.

    A few weeks back, I was in a Twitter chat session on AI led by Kirk Borne, data scientist at Booz Allen Hamilton. The question came up of what organizations should consider when creating guidelines for use of data and bias-free AI. Here’s my response:

    A4: DevOps teams should have someone whose role is AI oversight—actually watching what it’s doing and constantly testing/validating its findings #MITSMRChat

    — Don Dingee (@StratisetDon) February 14, 2019

    Common Problems

    Most fails are due to fire-and-forget, wherein AI systems are trained, tested against training and turned loose. Even a well-designed AI system is subject to several problems:

    Errors: This is where AI makes a factually incorrect decision. False positives and false negatives are in this category, along with miscategorization problems.

    Gaps: There are situations in which AI understands conflicting options and is unable to choose. Then, there are situations in which the AI doesn’t recognize what it sees.

    Bias: Maybe the system prefers green M&Ms or German shepherds because there was some subtle trend in the training data for “candy” or “dog.”

    Ethics: This can be a higher level of bias, where preference is given based on age, gender or race. Sometimes it’s the classic no-win scenario in which both options are harmful.

    An AI overseer guards against these types of problems in near real-time, every day, for as long as it takes to weed them out. Some problems are easier than others. An example of a more difficult problem came when autonomous vehicle researchers realized that not all traffic signs are brand new and well-lit. Shade, branches, wind, scratches, dings, bullet holes, stickers, graffiti and many more types of defects exist in the real world.

    Increased Complexity

    Inevitably, part of the solution the AI overseer leads is better scenario-planning and more training data. Retraining becomes a way of life and at some point, bigger data sets also lead to more complex implementations and another challenge: speed—after retraining to correct issues, the reloaded real-time classification engine runs like it’s stuck in molasses, producing correct results too late to be useful.

    While the A system is running, a B version should be in development with new training and classifier boxes. For very large data sets, teams should consider designing a workload-optimized training server complex instead of relying on elastic cloud resources. This was behind NVIDIA’s recent acquisition of Mellanox—using InfiniBand for more bandwidth between GPU clusters in AI server complexes.

    Where training systems often scale, real-time systems usually don’t. They are optimized for size and power consumption and can only accept so many additions. When a bottleneck emerges, it can kill the entire application. New generations of CPUs, GPUs and FPGAs keep pushing the envelope for designers tackling AI challenges.

    Human-in-the-Loop

    In many use cases, the job of the AI system is not to make every single decision autonomously. Instead, the system should make routine decisions and lead the AI overseer to look at exception cases. Directing humans quickly to areas needing attention is often more valuable than risking bad decisions. Human-in-the-loop is something DevOps teams should architect into systems more often.

    This may all sound obvious, but it points out a big difference in developing AI versus developing code-based algorithms. If there’s a bug in code, you can debug it and find the offending lines of code to fix. But debugging AI is a lot more difficult. Digital twin implementations come into play using playback of real-work inputs to look at what happens inside the classifier black box. There are usually hints of problems developing before catastrophe strikes. Running data both good and bad can help spot issues.

    Ideally, the AI overseer is a hybrid of a data scientist plus a systems architect. Look for someone who understands both data sets and hardware implementations, at least at a high level. They may have to call in other resources on the team for detailed investigations.

    The last thing you want after all the hard work in developing a system is to have it run amok months or years later. It’s just Murphy’s Law. AI systems will do something entirely unexpected at the worst possible time. An AI overseer can help a DevOps team get ahead of AI problems.

  • DevOps: Assessing Behavior Flows Through an Organization

    DevOps: Assessing Behavior Flows Through an Organization

    I’ll admit it, this article is a little strange. It is an experiment. But DevOps is all about experimentation, right? This article explores how information and behaviors in fact “flow” through an organization. It is based upon my observations with literally hundreds of DevOps teams. I have used the Three Ways of DevOps to show how and why many organizations struggle with this flow of information and behaviors, and what you can do to identify and prevent behavioral “defects” in your organization.

    Simple, Really: Mindsets and Behaviors

    IT organizations embarking upon Agile or DevOps ways of working are basically attempting to realize behavior change—simple. Consistent, new “desirable behaviors” that deliver value and reduce risks, wasted effort and costs—simple. This sustainable behavior change demands a change in mindsets and beliefs; it also means unlearning old ways of behaving—not so simple. This requires feedback and correction.

    Start with the ‘Why’

    Changing mindsets relies on the right messaging. Understanding the “why” and the “purpose” is seen as a critical factor in changing mindsets and ultimately behaviors. The right messaging is not just in terms of words, but also in terms of behaviors, such as people “walking the talk” and “leading by example.”

    Desirable behaviors are about people doing the right things. Doing the right thing implies making the right decisions; for example, prioritization of work. To be able to make the right decisions requires having the right information, such as mentioned above—the why. Knowing and understanding the why helps underpin decision-making and prioritization. However, we see many teams that do not fully understand the why behind new ways of working. Somehow the message did not arrive. Somewhere in the flow of information the “why as a justification” ended up as the “why as a question.”

    This made me realize that information and behavior flows through an organization. I decided to look at many of the challenges organizations face when adopting new ways of working, by using the Three Ways of DevOps.

    The First Way: Flow

    The First Way of DevOps is all about the flow from left to right, understanding the flow of work, seeking to increase the flow and never passing a defect downstream.

    Understanding the Flow

    The messaging I mentioned above flows through the organization traditionally top-down and bottom-up; with the shift to Agile ways of working, this flow is through the end-to-end value stream. This flow is upstream and downstream: Downstream is the flow and hand-offs; upstream is feedback.

    I mentioned above that desirable behavior is related to making decisions. If we consider the traditional decision-making authorities in an organization, we have from left to right: strategic, tactical and operational decision-making. This can be seen, for example, as the CIO and business management team (strategic decisions), e.g., product portfolio and key goals; product owners and IT delivery managers (tactical decisions), e.g., product backlog, IT changes; teams and employees (operational decisions), e.g., sprint planning, incident prioritization.

    The importance of decision-making was also confirmed in a recent Forbes article, “The Netflix Decision Making Model Is Why They Are So Successful.” Netflix’s desirable behaviors are driven by a set of corporate values. The first value is judgment, which “outlines the company’s decision-making ways and accompanying expected behaviors.  Notably, Netflix encourages its employees to ‘make wise decisions despite ambiguity’.”

    Every so often, it is decided to embark upon an Agile transformation at a strategic level. This decision is communicated throughout the decision-making levels as an intent—usually the why is vague and left open to interpretation. A set of values or principles are also communicated, such as customer-centric, end-to-end responsibilities and continuous improvement. However, these usually are not translated into desired behaviors. An implied set of behaviors are linked to these values and principles, and can end up being inconsistent with the original intention. This miscommunication of the why and the desired behaviors downstream is a form of a defect—often causing assumptions, driving poor decisions and resulting in undesired behaviors.

    For example, as a frustrated senior level manager complains that teams won’t take ownership and frustrated teams protest about lack of empowerment, desired behaviors were not defined and agreed. As a result, they were not displayed, waste occurred, frustration, delays, lack of return-on-value from the effort and behaviors spent.

    Another concept of flow is constraints—constraints stopping or slowing down the flow. In terms of behaviors, a constraint could be attitudes and beliefs (not understanding or believing in the new behavior), lack of skills or ability to perform the behavior and allocation of time to learn and experiment or improve behaviors. So, not only must we ensure that we do not pass defects downstream, we also must ensure feedback can identify constraints, impediments and barriers that are stopping the right behaviors from flowing smoothly through the end-to-end stream.

    Other types of constraints could be ignorance, indifference, confusion, fear, uncertainty, doubt—or, as Mike Orzen said in a discussion, focus, attention and trust.

    So, on one side we have:

    • Intent, implied, assumptions and inconsistent behaviors flowing from left to right (cause), meeting….
    • Indifference, confusion, fear, uncertainty, doubt, lack of trust etc. (effect)—you get the picture.
    • A management solution to this is then to tell people to collaborate, or to tell people they are now empowered—again, without defining the associated behaviors.

    “… Application delivery is a set of very human-oriented tasks, a key factor that determines the speed to value is trust,” wrote Carmen DeArdo in his latest book, “A Leader’s guide to Digital Transformation,” co-authored by Jack Maher. “As teams lose trust in the validity of the work as it flows through the lifecycle, a large amount of rework and waste is introduced.” he wrote. Trust is probably one of the biggest enablers or barriers to the smooth flow of behaviors through the system.

    The Second Way: Feedback

    The Second Way of DevOps talks about shortening and amplifying feedback right to left so that corrections can be made. One form of this is giving feedback when poor information is passed downstream or undesirable behaviors are displayed, enabling us to correct the messaging and the behaviors. Inconsistent messaging, in the form of managers giving conflicting signals or failing to “walk the talk,” are forms of defects. A commonly chosen ABC card (Attitude, Behavior, Culture) in global workshops is, “Don’t do as I do … do as I say.” However, if there is a culture of blame or a lack of safety associated with giving feedback to correct defects in behavior, then feedback isn’t given. Undesirable behaviors and inconsistent messaging goes unchecked.

    Failing to stimulate and act on feedback is another form of defect. This means undesirable behaviors continue to occur. All these behavioral defects create cultural and behavioral debt. As this debt accumulates, the ability to deliver value diminishes. In response to this, managers often revert to old management and control styles of behavior (cause), creating more resistance and reducing the willingness to give feedback or experiment (effect). Again, examples of passing defects downstream and creating more barriers and impediments to the flow of desired behaviors.

    As suggested in DeArdo’s book, “Feedback should be treated as though it were the most precious of our resources, because it is.”

    The Third Way: Experimentation and Learning

    The Third Way of DevOps is about continuous experimentation and learning—recognizing that repetition and practice are the prerequisites to mastery.

    Agile ways of working are still relatively new. DevOps is still evolving. Old management styles and models are no longer fit for purpose in this new world. Leadership in this new world is also characterized by experimentation. As one CIO stated, “Leaders are expected to walk the talk, but we don’t know what that looks like in this new model.” Leaders often fall back to an S1 leadership style—which is in conflict with the culture associated with Agile ways of working. Poor leadership styles and behaviors, as mentioned above, are also a form of defect—more behavioral debt.

    Feedback becomes even more important to validate experimental approaches and to learn and improve. However, the Third Way also stresses the need to allocate time for the improvement of daily work. We sometimes see impediments on team backlogs, but we don’t see behavioral or cultural debt as a backlog item. We don’t see this on the backlog of management teams’ behavioral debt, either. Teams often are frustrated with a lack of time reserved for technical debt, impediments or improvement work—let alone behavioral debt. As Orzen stated, “Even more important than daily work is the improvement of daily work.” Continual learning and improvement should be core skills and behaviors.

    As stated in the Third Way, repetition and practice are prerequisites to mastery. There is nothing new about this. It isn’t that the DevOps movement came up with this “new” philosophy: Aristotle, around 300 B.C., stated, “We are what we repeatedly do. Excellence, then, is not an act but a habit.”

    We need to practice and repeat behaviors until they become habits, or culture—like “the way we do things here.” Practice also requires feedback to validate improvement gains as a result of practice.

    Top athletes practice with the guidance and feedback from coaches. Scrum and Agile coaches are often assigned to Agile and DevOps teams as they practice and grow. These coaches are skilled in training on Agile and DevOps tools, such as standups, retrospectives, value stream mapping, Kanban and Kaizan. But what we see globally is that not all coaches have behavioral management skills.

    Another form of feedback from practice is measurement. In terms of DevOps, teams measure metrics such as number of releases—successful releases. We don’t often measure the behavioral change. We can measure the frequency and decisions from standups, but can we measure giving feedback, confronting undesirable behavior? One small example I use is measuring the amount of “yehbuts”—yes, but!—in a dialogue, or worse still, “yehbuts” before somebody has even finished talking. This is an example of a communication defect. Another example is assuming that we understand what somebody has told us without confirming if what we believe we heard was the actual intention.

    Communication defects are a major barrier to successful flow of information. Another measurement needs to be related to the impact of behavior change, such as improved customer and employee satisfaction and less rework because of assumptions.  Although measuring the actual impact is difficult, having the dialogue itself is valuable: How is what we are doing adding or reducing value to the team and to the organization?

    Conclusion

    If we want desirable, consistent behaviors focused on creating high-performing teams, we need to look at how messaging and behaviors flow through the system and recognize defects we are passing downstream. Our vague intentions need to be quantified in terms of desirable behaviors. We need to build in feedback mechanisms to signal defects and undesirable behaviors. We need to identify the constraints and impediments that may be causing undesirable behaviors such as fear, uncertainty, doubt—behaviors that may be slowing down or stopping our transformation to new ways of working. We need to recognize that experimentation and practice is needed—knowing that we will struggle and fail, but ensuring we learn and improve as an end-to-end capability. Leadership skills, styles and behaviors need to be aligned to facilitating the three ways of behavior flow. Finally, remember there is really nothing new about this.

    Case

    In a Phoenix Project simulation, after recognizing the above defects, a team posted the strategic portfolio and goals on the visual management board. The CFO explained these goals and why each backlog item was prioritized against value creation (benefits and value to be realized)—an example of clarity of the why. At the same time, the CFO displayed the right behaviors—being at the standup to explain to the team and ask for direct feedback and understanding.

    At the same standup, the CISO and IT team explained the potential “value leakage” backlog items in terms of debts, risks and defects, so that together—business and IT—could make the right decisions on prioritization.

    “This made me realize,” said the product owner, “I was now able to make a better-informed decision, because of the team’s input on value leakage—giving feedback from the usage of the applications and the negative impact. It also gave me more trust in the team as they demonstrated an understanding of the ‘why’ and had a clear understanding of value. If, as a team, we did not fully know or understand the value, this created a great dialogue to explore it and ensure we didn’t pass a ‘defect’ (wrong decision on prioritization downstream).”

    The team in the simulation learned that as well as visualization of impediments on their backlog, they also needed to visualize the behavioral defects that needed work on and allocated time.

    — Paul Wilkinson

  • SRE: How Do You Get to Blameless?

    SRE: How Do You Get to Blameless?

    Site reliability engineering (SRE) is the companion to DevOps. If we take my normal divisions of DEVops versus devOPS, traditional DevOps is mostly the developer side, and SRE is mostly the Ops side. This isn’t a perfect comparison, but it is a good starting point, and for this blog, I’ll run with it.

    While DevOps focuses on getting releases out consistently, quickly and with minimum disruption, SRE focuses on putting those releases into a robust infrastructure. Tracking application issues, regardless of whether you’re using DevOps, always requires information about the software and hardware infrastructure. SRE takes the proactive steps to try and make sure the environment is stable and designed to host the applications being thrown onto it. When that process fails, SRE attempts to identify the code/architectural problem and resolve it without blaming staff members—hence, the idea of “blameless.” It’s about the environment and improving it, not about the individual and punishing them.

    There are a lot of interesting parts of SRE that most IT groups could look at and see as a good place to start taking advantage of. Closely tracking errors in a cross-section of IT interest areas—by application, by infrastructure, by toolset—means problem areas can be definitively identified and fixed. But there is more behind SRE. In this example, once tracking of where errors occur more frequently is in place, a selective reduction in deployments using that piece of the IT puzzle can go into effect until the issues are resolved. This is not unlike what we are doing today, but it is formalized and has measurements around it that generally are not in place.

    Which brings us to Blameless, a company that just came out of stealth to champion SRE. While some large organizations with large application footprints are using SRE already, Blameless is offering a platform to make SRE more accessible to companies without large staff and application footprint.

    We took a bit of time to talk with Blameless last week, and their vision is a strong one: Address directly what DevOps addresses more indirectly. Build and environment that can measure and improve the entire application ecosystem.

    If nothing else, the idea of error budgets is intriguing. Counterintuitive at first glance, the idea of saying, “Piece X gets Y amount of issues before we start backing off of it or raise the priority to fix the underlying issues,” is a solid one.

    If your organization is looking for a mechanism to improve reliability and stability while reducing time to resolve issues, SRE in general and Blameless specifically are worth a closer look.

    — Don Macvittie

  • 5 Practices That Will Help Drive Successful Robotic Process Automation Deployments

    5 Practices That Will Help Drive Successful Robotic Process Automation Deployments

    Enterprises have begun to invest considerably in robotic process automation (RPA) in recent years. RPA is one of the critical technologies that enterprises now are leveraging to automate their business processes to make their businesses more agile and efficient. When properly deployed and managed, RPAs can automate everything from mundane tasks (such as organizing files) to more significant functions, such as user provisioning to resources and more.

    While RPAs do promise to help businesses move more quickly and with more agility, a company’s RPA strategy needs to be executed effectively across the organization. In my recent discussions with CIOs and others implementing RPA, the following practices repeatedly were mentioned as essential to RPA success.

    Understand the Process Being Automated and How RPA Impacts Other Operations

    One of the most common mistakes made is not thoroughly understanding the process being automated and the potential implications of automating the process. An example would include a business process that is time-sensitive and automated, which throws off the workflow timing. It’s possible the automation moves things along too quickly.

    Another example would be not taking into consideration the nature of the data being accessed. If it’s regulated or sensitive data, there are security implications that must be managed. If the bot potentially will process sensitive or regulated information, the bot should be controlled appropriately. RPA technology is an IT function, and at the very least IT and security should be made aware of and potentially involved in the oversight of the bot. It’s much better to inform them and bring them in early than them finding out later about an army of bots managing and manipulating information.

    Consider Refining the Process Before Automating

    After the process being automated is understood, evaluate the process for potential optimization. Sometimes processes that have been underway in organizations for years are more than just inefficient; they’re actually antiquated. Look for ways the process can be made more effective; consult all stakeholders when necessary or warranted; and consider refining the process if possible. There’s no sense in automating an inefficient process.

    Document, Test and Catalog Your RPAs

    After the RPA is built, make sure to document its functionality and how its code works and examine its functionality. The RPA is now ready to be used. It’s important to continually monitor the RPA for potential abuse or something gone wonky. Send all the documentation relating to the bot to the IT department.

    As the Bot Owner, Know You’re Responsible for Your Bot

    Should the bot do something unexpected or unwanted, prepare to answer for it as the bot, or business process, owner. Make sure the bot is managing risk within its acceptable scope and that it’s not given too high a level of access. While you as process owner may have administrative rights to an application, does it make sense for those rights to be transferred to the bot? Perhaps it does, and perhaps it doesn’t. It’s a risk management call and will vary from situation to situation. Just be aware that, as the owner, you also own the risk.

    Remember Your Bot May Live Beyond Your Tenure

    In one, two or five years from now you may not be in your current job position. It’s essential that the bot is constructed for this eventuality. Beyond the proper documentation and cataloging of the bot mentioned previously, put into place the appropriate identity and entitlement management of the bot. Down the road, someone may want to enhance the bot with machine learning or AI capabilities. For when this happens, it’ll be essential to have the bot well-documented as well as its entitlements well-defined.

    Consider Centralizing the RPA Deployment Process

    While some enterprises have chosen to undergo organizationwide automation efforts, others are dipping their toes in tepidly with small RPA pilots. Others still haven’t decided on how to incorporate bots within their organization, so it’s happening organically and haphazardly as individuals and departments decide to make a go of it on their own.

    If an organization is large enough, it may make good sense to put into place an RPA center of excellence. It will provide a repository of policies and strategies for RPA deployment, help catalog bot capabilities and dependencies and help drive overall RPA efficiency as the organization becomes more and more automated.

    These bots are here to stay—in fact, they are going to grow in numbers. Technology research and advisory firm Gartner expects global spending for RPA to reach $2.4 billion in 2022, up from $680 million last year. That’s an increase of 57 percent annually. RPAs help drive efficiencies and remove mundane, low-risk tasks and workflows so that humans can focus on higher value and higher cognitive tasks.

    — George V. Hulme

  • Deeper Technical Information with SweetCode and Fixate

    Deeper Technical Information with SweetCode and Fixate

    I know many of you go right to the article you want to read on DevOps.com without ever visiting the front page. But if you go to the home page and look down in the bottom right corner of the page, you will see a link to Sweetcode.io. Sweetcode is a site run by Chris Riley and the folks over at Fixate.io, offering great deep, technical articles on all things DevOps and more. Check it out.

    More importantly, it is the visual evidence of a deeper strategic partnership that we here at MediaOps have formed with Fixate. I have known Chris Riley, the founder of Fixate, for about five years now. Chris has done a great job of developing a deep bench of technical writers who can really deliver the technical viewpoint without the marketing fluff. I know this is exactly the kind of content our audience loves. Not just our reading audience, either—our sponsors are also challenged to create good technical articles for their audiences. Now, we can deliver this either to our sponsors or on behalf of sponsors.

    On top of this, we are also going to add Chris’s technical chops and talent to our roster. You will see him hosting webinars, doing interviews at conferences for Digital Anarchist and otherwise engaged with our audience. Chris brings a practitioner/analyst viewpoint to his work that I think our audience will appreciate.

    I should emphasize this is not an announcement of an acquisition or merger with Fixate. Fixate will continue to operate as an independent company (which it is). We will just be working really closely with Chris and team there to help bring you deep technical articles and other forms of content that we believe you will find interesting. It is all part of our plan to keep creating and distributing the kind of material that our audience is looking for.

    So, go check out Sweetcode. We also will be promoting Sweetcode through our social media and newsletter channels. Stay tuned as we turn up the gas, and we hope you enjoy.

    — Alan Shimel