devopsy.kinsta.cloud

Tag: devops

  • DevOps Chat: DevSecOps with Signal Sciences’ James Wickett

    DevOps Chat: DevSecOps with Signal Sciences’ James Wickett

    James Wickett is the man to go to for DevSecOps. The founder of the Rugged DevOps movement, which has merged into the DevSecOps group, James is one of the most knowledgeable people on the subject of DevSecOps.

    In this DevOps Chat, James shares his views on what’s on the horizon for DevSecOps and the most important things we can do to make our teams more secure. Have a listen as James gives us his take.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.


    Transcript

    Alan Shimel: Hey everyone, it’s Alan Shimel, DevOps.com, Security Boulevard and you’re listening to another DevOps Chat. I’m really happy to be joined on this DevOps chat by one of my friends from the Austin boys, James Wickett. James, welcome.

    James Wickett: Hey, thanks Alan. Really glad to be here.

    Shimel: It’s my pleasure to have you here. So for those of you who may not be familiar with James, you know, James is really one of the lynchpins in the, for my money, the best DevOps community in the world and it’s the folks over in Austin with Ernest Mueller and James and Karthik and Boyd and just a whole bunch of great people coming out of Austin. But when James isn’t busy building the best DevOps community around, he also works at Signal Sciences and of course James was also kind of the founder, prime mover behind Rugged DevOps and now DevSecOps, as well as with Shannon and those folks. So, I hope I didn’t make you blush, but –

    Wickett: That’s all right. We loved having you come down to Austin and we’re pretty proud of the – you know, it feels like the InfoTech community here in Austin is really close knit and there’s a real family dynamic to it, to a certain sense, the good kind of family. I know that can be different for different folks, but you know, it’s very nice that we’ve been able to do that for so many years and we’re excited for bringing DevOps days Austin, continuing to come back year after year, and your partnership with that has always been great, Alan, so we –

    Shimel: Yeah, no, it’s my pleasure. I don’t say it lightly, especially this past year and a half, two years, I have probably been to more DevOps days and DevOps events around the world, not just here in the US. And Austin’s a special, special place for it. James, as we were talking off camera a little bit, you know, over here at DevOps.com and the rest of the media op sites, we’re doing an event called Predict 2019 and it’s a virtual conference. It’s more than just kind of like a video like this where it’s actually a virtual conference where the sponsors have booths and there’s a whole session auditorium and everything, but the idea behind it is really to put on our crystal ball hats and take a look at, hey, what does the near term future look like? You know, if we’ve been in tech long enough, we know better than to make a long-term future guess.

    Wickett: That’s right, that’s right.

    Shimel: Right? Who knows? But, near term we’ll try. So, I wanted to spend a couple minutes with you and talk about what do you see 2019, the next 12 to 18 months. What’s factored in on the DevSecOps scene, on the DevOps scene, on AI Ops, on you know, just what we live in day to day, James?

    Wickett: Yeah, yeah, that’s right. Yeah. Well, that’s a good question because when you think about DevSecOps and where it is today, I think, I always hope that we’re further along than we are, and the data still shows that like we have some ways to go. There is a SANS survey on secure DevOps and kind of the joining of just security and DevOps and where it is in the current state of the world and Signal Sciences, along with several other sponsors, are happy to kind of be a part of that survey and that research, but one of the things that is pretty astounding out of that whole piece is that 95 percent of securities responsibilities or 95 percent of security people reported that their main kind of thing they were doing was supporting legacy applications.

    So, and it’s real telling that we see this – we’re seeing really in security and DevOps kind of the same problem we had with Dev and Ops, you know, ten years ago, right, this whole throw it over the wool mentality, you know, different groups value different things. Developers valued feature speed and deployment ability, I guess. Operations valued stability, reliability, things like that, and they always felt like they were at odds. Well, security comes in as sort of this third wheel here and what does that do to the mix?

    And I always – I’m really happy like to be spending time with some of our customers at some events and I get to hear stories of people doing things that are radically different than securities done over the last decade, but to say that that’s not – to say that that’s ubiquitous would be false. Like that is just not where we’re at right now and I think it goes back to that quote, right, and I think this helps us with this 12 to 18 month prediction piece. Like the future is here, it’s just not widely distributed. Right?

    So, I can see what leading high performing companies are doing and I can just say, next 12 to 18 months is like more people doing some of that stuff, and I’m excited about that.

    Shimel: I agree with you. You know, James, this is something that I, you know, I’ve always kind of felt is that, you know, we certainly suffer from shiny trinket syndrome and we always want to know what the next great thing is or what’s the next big thing. But what we often forget about is the whole world isn’t a green field. It would be so great if we only had green fields and new apps to work on.

    Wickett: That’s right.

    Shimel: But we live in a world of legacy apps, right? We’re going – no one’s throwing out their mainframes. No one’s going to just throw away all of these applications that have been built. Excuse me.

    Wickett: Yeah, we live in the brown field, Alan.

    Shimel: Right, we live in a very, very brown field and so to me, should we be focusing near term at modernization, right, and how are we – because at the same time we have these legacy apps, James, we have such new infrastructure possibilities with, you know, so how do you containerize and Cubernettisize and make servo lists, all of this brown field application and at the same time, I really think that’s an opportunity for security.

    Wickett: That’s right, yeah.

    Shimel: To come in and let’s do it right. Right?

    Wickett: Right.

    Shimel: Let’s fix something that really wasn’t set up right the first time.

    Wickett: Well, and the inverse or the – there’s another layer to that problem is that we’re seeing people in the security space that are almost cutting edge for like what we would call like DevSecOps and people that are doing great, but they’re all coming from like large enterprises, which is a really weird thing because innovation from the DevOps space came from like small, medium sized companies that were able to be nimble and be fast, but when you’re at a certain size, you don’t have security talent on hand. You don’t have anybody on staff and you’re also – you have a different set of concerns. Right? Your concerns are staying in business for the next several quarters.

    Shimel: Right, keeping the lights on.

    Wickett: Right, when we’re seeing people like Shannon Lietz over at Intuit or we’re seeing Topo over at Capital One, right, or Chitra Elango over at Fannie Mae, right, we’re seeing these stories and the way that people are piecing together pipelines and they’re tying all these security things into these DevOps movements. They’re all coming from big companies and I don’t think there’s – I think there’s definitely a connection there. We’re going to see some of that security wisdom and kind of learning of how to do DevSecOps come from the other end of the spectrum, which I think is a really cool thing.

    If you’d have asked me three years ago, I would not have predicted that at all.

    Shimel: No. So I’m going to tell you, along those same lines, James, to me, this may wind up being the hidden gem or the hidden legacy of DevOps. When I first launched DevOps.com and you were around, you remember, you know, besides everyone yelling about making a commercial, there was this huge argument raging between people about startup DevOps versus enterprise DevOps, you know, could enterprises do DevOps, was it a unicorn thing? And you know, some of the – I always felt that the right answer was, there’s only one DevOps. Right? There’s not a startup DevOps and an enterprise DevOps.

    But, different organizations have different things. I think what we’ve seen over the course of evolution is that yes, DevOps really kind of took hold in the small midsized businesses out of necessity, but enterprises shaped and molded it and made it work at these large enterprises, and in doing so, created opportunity for the security folks, the QA folks. Right? All of these other silos, right, that maybe you didn’t have in a small company, right, because everyone wears like seven hats.

    Wickett: That’s right, that’s right.

    Shimel: Right? But, at the enterprise level, it’s giving it a chance to flourish and to – it’s like a giant petri dish, right, and so it’s great and it’s giving security people a chance to get in here. So –

    Wickett: Well, and you look at, you know, you have a vision of like, hey, let’s put this for, this is not just for startups and west coast companies. This is something that can hit across – and I love that. I love going to middle of America, DevOps Days events like the Kansas City one was quite _____.

    Shimel: Kansas City was pretty cool, yeah. Right.

    Wickett: And I liked it because it’s people at larger companies wrestling with these problems, trying to make this happen and I’ve been lucky enough over the last several years of my career to do a lot of green field type stuff, but that’s not the yeoman’s work. The real tough work that we’ve got to do is DevOps transformations in these large enterprises. And you and Jean Kim through the DevOps Enterprise Summit we see stories of these types of transformations happening and seeing security being brought into that. I mean, it’s just–it’s really pretty cool.

    Shimel: Absolutely. So, now not to prod you along here, James, but –

    Wickett: Yeah, prod me. Prod away.

    Shimel: I see it, so we’ve had a chance to build a little bit of a foundation. Right? We see these things happening at the enterprise level. Kansas City, Chicago’s another great example. Big companies, right, talking about their DevOps and their DevSecOps. How do we build on this momentum, James? What shape does that take? As security people, what do we need to do to move that needle?

    Wickett: Yeah, that’s a good question. Let me – can I take two answer on that and –

    Shimel: Sure, take three if you want.

    Wickett: Sure, all right, good. Well, I think there’s – let’s start with the first answer. I think we’re still developing a mental model or some sort of platform of like how the delivery pipelines work. I mean, of course you can go to any number of talks, you can see how our people have assembled them, the tooling that they’ve put in place, but security has a lot of success whenever they can look across the – whatever the software delivery value chain and say, “All right, how are we delivering the software and where is our build system or our development process and then how do we deploy and deliver all the way to the customer?” And look across the broadly and find discreet tools that they can add in those places, and not just tools for like software, but like practices or ways to transform. Take the ISC squared, CISSP manual and rip out a good – a lot of the waterfall bits, but figure out like were does this change, like in the modern world? And I think that’s – I’d say that’s like the first approach that I would like to say.

    I think philosophically, I would – I’ll kind of answer you with a situation that I was posed with recently. A friend of mine, he works at a large enterprise company. They are not cloud, just now thinking, “Hey, maybe we should do this cloud thing.” And which I – some people that are listening to this are like, “What?” But that’s just the way it is. People are –

    They have so much investment in the data center that like now it’s like they finally kind of said, “All right, we have some new stuff that we’re building. We think the cloud is where we should do it and we’re hearing these things like Cubernettis and servals and containers and whatever.” Right? “So we’re trying to figure out what direction we take with all this.”

    So now some of these companies, legacy type companies, are putting resources towards it. Well, this guy is a security guy and he had them kind of come in and say, “You’re going to DevOps the thing and DevOps is going to be the way.” Right? Not really – this is a company where maybe they’re not even really sure what DevOps really means. Right? Which is okay. They’ve heard it. They’ve got some of the vision and they’re interested.

    He’s very concerned. He says, “James, what do I do about this? I know you like the DevOps and stuff like that, but you know, what do I do from a security perspective.” I give him an answer. I didn’t feel great about it. I gave him a, you know, I don’t know, find ways to inject stuff, I don’t know. I went home, thought about it, woke up the next morning and I was like, “I got to call him back.”

    So I gave him a call and I was like, “Hey, the real answer to your question is never say no ever again. Never say no ever again. Security has lived the world of no for so long and you have to throw that away. Like your new answer has to be yes, you can have yes with qualifiers, yes with whatever, but you’ve got to find ways, creative ways to say yes and kind of the cloak and dagger, blame it on compliance, slowing down things for whatever, you can’t do any of that stuff anymore and you’ve just got to throw out all the security gets to be a blocker.”

    And I said, “Because every organization that I see high performing, what we would now call like DevSecOps teams, but security and DevOps working together, they sort of rip tear up the security playbook and they say, if you are a blocker, everyone in the organization is going to route around you. But if you play ball, like you get to be a part of the solution. And security has to find a way to add value inside of that.”

    And so, and I said, “Every single one, you talk to any person who’s undergone the transformation and working in any of these companies to Hulu to Netflix to Inuit or whatever, you know, or all the ones that we see at DevOps Enterprise Summit, they’re always finding ways to say yes and be part of the solution and not to see themselves as a blocker. So I think that’s a fundamental like philosophical shift that security has to undergo.

    Shimel: Absolutely, and you know, just to turn it around on you a little bit, I’m going to be the Jewish guy spreading the good news too, right, the gospel. We started off this conversation early on and you said, “Well, we still have so much more to do and I wish we would be further along.”

    Wickett: Yeah.

    Shimel: You’re right, but someone much smarter than me once told me, James, change, especially in technology, is 99 percent of the time evolutionary, not revolutionary.

    Wickett: That’s right.

    Shimel: But, when we look back over a period of time, we’re often astonished what that evolution – how that evolutionary change adds up. So the fact, that little piece that you espouse right there about not saying no anymore, and yes, we can and you know, whether you qualify it, caveat it whatever, but yes, we can.

    Wickett: Yeah.

    Shimel: It’s so powerful. It really is. It’s so powerful. Look, I spent 2001, I’ve spent 18, 19 years hearing now from people, and – or being part of the no too, to be in all honesty.

    Wickett: Right, yeah. I mean, that’s –

    Shimel: You don’t want to wind up in jail, do you?

    Wickett: I got my ISC squared books up there. They look at me with an evil eye.

    Shimel: Yeah, so you know, it’s refreshing. It’s like liberating to take that attitude. How do we spread this gospel though? What more – what better way can we do it? I know you’re involved in DevSecOps.org and DevSecOps Days and I’m obviously too and at RSA this year, we’ll be doing our thing again. But what more could we do in 2019, James?

    Wickett: Oh man, that, I don’t know the answer. I don’t know the answer to that question yet. I mean, I think there’s some pragmatic things. I think we continue to say the story side. I relate to what’s going on with security right now to what happened with DevOps in the beginning and if you rewind to like 2009, 2010, or like let’s say the first three years, right, DevOps equaled chef or puppet, like straight up, like that’s what it was. And so what was that? That was operations was finally able to be developers, to act like developers, to shift left. Right?

    They participated in agile type practices with infrastructure, revolutionary, really cool, really great. But then there was also another movement that happened at the same time and that was the idea that like we needed the instrument a lot better. We need to do monitoring. Now, people talk about observability. There is a big push of understanding like what your systems are actually doing. So, and that was, then there’s kind of the whole like monitoring conference, there’s monitorama, there’s monitor movement and the monitorama in response to it.

    There’s some great products have come out of that space. We see things like Data Dog and before them Cloud Kick and other folks. Well, I relate all that to security because if you look in the first several years of this security and DevOps kind of joining forces, I know people have been talking about it for many years, but over the last couple years, what are the things that everybody has kind of gone around? It’s the whole idea of shift left. So you see great companies, _____ types, Nick, other folks like that, that are focusing on, you know, putting all of your development testing, you know, figuring out what binaries, talking about software supply chain, bill of materials, all that sort of stuff. We’re still missing the push right.

    Now, I mean, I work at a – or a shift right type thing and so like, Signal Sciences, we sit at that end of the spectrum, so this is a bit of a self-serving type statement, but I mean, you need to have sort of insight into sort of what your application is doing and when we started the company, we were singularly focused on one question and that question is, do you know right now if you’re under an attack with any degree of certainty, we’ll say. You don’t have to know 100 percent, but with any degree.

    I ask people that question like in conferences and whatever and almost zero, like I’d say just one or two people per, y, every six months or whatever, says yeah, they know. Most people have no idea, like what’s actually going on in their site. And we see this like when was the last time you read the headline that said, like, you know, we lost a million account credentials, but we stopped them. We’ve been following them through our system, we saw that they were starting to leak data and so we turned it off, but we didn’t get it off soon enough, but they didn’t get 100 million, but they got a million. Right? You never see that news story. Right? You see the news story of like _____ we lost it all and we lost it all three months ago. We had no idea until it started being traded on the market. Right?

    So, like we lack a critical insight and you know, operations had the same problem, right, didn’t understand what was going on with the systems, performance-wise, everything. Security is quite similar to that. So, being able to instrument and see and do monitoring and have that sort of security observability is like an important aspect to where I think we need to go. So, we’ve done a lot of the shifting left similar to ops in the early days, and I think we’re going to see some more movement on the shift right part.

    Shimel: It also sounds though like opportunity there for some better tooling.

    Wickett: Yeah.

    Shimel: Right? I mean, a lot of our security tools, you know, from vendors really aren’t made for that, right? I mean, you know, we have so much – so you know, the fighting point in the security vendor space is, should we put more focus on security response versus prevention? Right? Threat intelligence, all of these things, and you’re right. We don’t have that analog of a Data Dog, a Signal FX or that, for the most part, in securities.

    Wickett: Yeah, ’cause like I see that across like – so for Signal Sciences, like we provide that for web applications, but that’s also stretching across other boundaries that you need to look at, and a more fuller picture, and I remember whenever we were launching – when Ernest and I were doing cloud stuff at our first just kind of thing over at National Instruments, we were trying to find vendors and just straight up like operational monitoring vendors and I talked to like – that was my job. That was one of my first parts of the things.

    It’s like, okay we built the thing that – we have this whole thing we called Pie that spun up our system, much like Chef and Puppet, but then we said, all right, what are we going to monitor this stuff with? And I started calling and monitoring vendors and getting demos and stuff and it’s like, at the end of it, I was like, I just need one that has an API. Like just a good working API that can like admit _____  _____ and connect, you know, ’cause this is early days. Right? And we ended up finding a Cloud Kick and they ended up getting bought and sold and we moved later on to Data Dog, but it was so hard at the time to find that. And I feel like security vendors are similar in that space, that they struggle from where like when – and we’re not alone in this. There’s other people in our space that are doing thing that are helpful for people, but emitting data back to all the systems that you work with, like the Slack and JIRA and whatever, but also just being part of like the whole tool chain approach. And I think that’s a story we’re going to see continue to be developed and companies that are adopting that and embracing the idea that they’re a piece in the overall tool chain, like I think those companies are going to continue to do well.

    Shimel: I know that you can have like just standalone security monitoring and so I think security monitoring has to be part of your larger monitoring.

    Wickett: That’s right.

    Shimel: Two or three other points I wanted to hit on, get your opinion on. So, just like DevOps, same analogy, tools are important. Culture trumps tools in a lot of places and we can have the best tools in the world, but until the security people’s mindset is changed and we’ve touched on it in this talk, you know, not saying no anymore and all of that, so how do we go about changing that mindset? Well, one piece of it, James, is at the organizational level, right, we embrace security.

    We make them part of the team. Another part of it though is at the community and industry level and we mentioned it now a few times and a lot of my friends mention it, and this is the whole what are security people taught. And I’m not knocking ISC Squared or SANS or any – we’ve got some –

    Wickett: Let’s knock them. No, yeah.

    Shimel: I’ve got friends on the board there. You know, I got to be careful. But, I mean in all honesty, and again, I’m not singling any one of them out, but when security people are being taught to say no, it’s very hard for us to come in and say, “Don’t say no.”

    Wickett: Yeah.

    Shimel: Right? So, do we, you know, you know I’m part – I’m one of the cofounders of DevOps Institute and we have a DevSecOps class there that you know, some of my more enlightened friends have helped us with. You and Ernest have done the DevSecOps thing. You guys do _____ and stuff like that. But those messages, I don’t know if they’re finding their way into, let’s call is mainstream security training, ISC Squared, CISSB exam, the SANS certifications and not just those two. There’s a half a dozen other very well-known ones. How do we, you know, is 2019 the year we make a beachhead there?

    Wickett: I don’t know. I don’t know. I’m an optimist, Alan, so I would love to say yes to that question. I would love to say yes, but when you think about it, like it’s – security has a very tough job. They’ve worked themselves into a scenario that they’re beholden to compliance and standards and outside bodies of influence and then they also are inside of an organization trying to affect change, but they find themselves at a very like inequitable distribution of labor. So like you have a hundred developers. You have ten operations folks. You may have one security person. Right? That’s a recipe for a problem. They’re all going to have different values.

    So, I don’t know if we’re just going to automatically fix the culture in 2019. I will say tools influence – tools have an influence on culture and that you know, in the CANS model or CLAIMS model for DevOps—or CALMS, I guess people like to call it as well, but for the M, for measurement and sharing, I think those are the two pieces that can really help influence the culture in a more positive light.

    And so, you know, security has, you know, we have stocks and we have like we – we remit all the data and have people taking concern of this, but we’re not – if you’re not creating – this is something else that I try to influence among my security friends. It’s like, if you have any tooling that you’re buying and implementing whatever, but it’s not creating a feedback loop to developers, then it’s like worthless. You should _____. Right? If it’s just going to a sock and someone’s doing it like, that’s not good. Right? There’s no DevOp telemetry out of what’s actually happening.

    And so I think finding products that can help or whether it’s like open source testing tools or you know, instrumentation on the right or detection on the left or whatever, but like creating those like, I don’t know, able – we’re able to have a sort of virtuous cycle of feedback loops to development. I think that’s going to be key. But yeah, I’m going to say like no, 2019 is not going to be the year just because of – just because of the way humans are. I would like – but I think that it’s going to get better.

    Shimel: I think it will get better. Here’s what I think we need to do, James, and you have got to do in this.

    Wickett: All right.

    Shimel: You and I are both wired into this community. We know people on the ISC Squared board. We know SANS instructors. We know the folks at SANS. We know all of these places. If one of the missions of the DevSecOps.org group, team, community, DevSecOps Days, what I do with my stuff, is to make sure we reach those people and say, “Hey, we represent a very large piece of this community and you’ve got to listen to us. It’s like a voter registration drive then. And – but you’ve got to lead it, right? And so, I’m going to put the onus back on you, James, and say you can make this a reality. You –

    Wickett: All right. Well, Alan, let me tell you how I am working on that. So I do have somewhat of a plan here or an idea here. So, I’m working with several others, Shannon Lietz, Ernest Mueller, John Willis and we’re working on the DevSecOps handbook, which is going with IT Revolution, and we’re trying to tell those stories of like how DevSecOps is transforming at large companies, small companies, helping people see the new path forward. So hopefully yeah, that’s –

    Shimel: In 2019?

    Wickett: That’s – the – what is it called, the like the draft galley copy, like we’re hoping to have some of those available in 2019 at the DevOps Enterprise Summit _____ either in London or in Vegas, so yeah, it depends on – you know, it’s a – they run a real great shop over there, so it depends on the schedule for editing and all that stuff.

    Shimel: Well, that’s a revelation. Good stuff, man.

    Wickett: So hopefully 2020, 2021, like we’ll start to see the industry really shift. You know, I do hope that – I always hope that things happen sooner than they do, but –

    Shimel: It _____ before it’s time either, my friend, right?

    Wickett: That’s true.

    Shimel: So James, I told you this was only going to be 15 or 20 minutes. I’ve got a feeling we’re at like twice that already.

    Wickett: Yeah.

    Shimel: But, it’s okay. I think it was a great conversation. People will enjoy it. Predict 2019 folks, you can go to predict2019.com. James is, of course, with Signal Sciences and you should definitely check out their site and what they’ve got going on. And also just a quick reminder, RSA conference is here March 4 this year. And we are doing our DevOps Connect DevSecOps Days at RSA Monday March 4, so if you’re going to RSA, come a day early, come in at the Moscone Center, Shannon the whole crew _____.

    Wickett: Yeah, if people are listening to this and they haven’t been to that, like, you’ve got to go. Like, I think that’s a very valuable single day track that’s really helpful. And also, some of the sponsors and vendors back there, like some of the really forward-thinking folks, I mean, it’s the – it’s really like the part of RSA that I feel like has really kind of anchored that for me in my experience at RSA. It’s like “Oh, okay. Here’s the crew. Here’s what we’re trying to do.” I can see the message going out, and I love the fact that kind of the vision, did you decide to start that or is that –

    Shimel: That was me.

    Wickett: That’s – I think that was good because –

    Shimel: DevSecOps Days, I just – you know, I – so I have a relationship with RSA for 15, 20 years and I originally went to them with it. This is the fifth year we’re doing it. I’m really pumped. I mean –

    Wickett: Yeah, well, and thank you for that because basically like we – I’d say one of the ways we make the transition is by going to security events and kind of helping make this happen.

    Shimel: Yeah, raising the flag.

    Wickett: Yeah, and you – that is an outpost for DevOps over at RSA, RSA of all conferences. So –

    Shimel: Yeah, if we can do it there, we can do it anywhere.

    Wickett: I agree. I agree.

    Shimel: All right. Hey, James Wickett, Signal Sciences, thanks for being our guest on this DevOps Chat. James, thanks for all you do too for the community and everything else. Keep up the great work and we’ll speak to you soon.

    Wickett: All right, thanks Alan.

    Shimel: All right. This is Alan Shimel for DevOps.com Security Boulevard. You’ve just listened to another chat. Have a great day everyone.

    — Alan Shimel

  • Reigniting Business Culture: Is Marketing Crippling DevOps?

    Reigniting Business Culture: Is Marketing Crippling DevOps?

    DevOps, as we understand it, is a set of protocols and value-systems that enable development and operations teams to work in perfect harmony within an organization. Surveys of DevOps response teams reveal that the strategy of building bridges between developers and marketers has helped organizations speed up software release cycles, improve product quality and augment platform security. We know for sure that DevOps improves performance, strategic outcomes, company productivity and, more importantly, the product’s market share. But there is a glitch; apparently, everything isn’t as rosy as we imagined it to be.

    There is the elephant in the room, a problem that lurks in the background, an issue that refuses to go away. It would appear that there is a growing chasm between Marketing and DevOps that businesses are struggling to bridge. Some voices would say that developers are opposed to the traditional ways of marketing and the “My way or the highway” attitude that marketers sometimes adopt, which is not conducive to collaboration. Far from helping organizations work harmoniously, is Marketing killing the golden goose of DevOps? Are we in desperate need of reorienting our business culture?

    Miromind’s SEO strategist Agnieszka Podemska believes the growing impact of technological change has made it obligatory for developers and marketing professionals to meet on a common platform. As the DevOps methodology takes control, marketers are increasingly joining the conversation and influencing the outcome, and this need not be the friction-ridden, nerve-jangling process it is made out to be. Let’s come to terms with the role-playing we see in the DevOps arena.

    DevOps needs marketers to collaborate and provide access to policies for determining the thrust of business growth. If marketing aims to improve the user experience and customer satisfaction levels, marketing needs DevOps to create a strong and reliably efficient IT backbone to facilitate that experience.

    The developer working in backroom isolation would be far removed from the battlefront where the customers are changing preferences and deciding product choices at a fast pace. For developers, it’s easy to lose track of what fulfills customer intent most effectively. Without regular feedback from marketing and sales teams, developers wouldn’t know how, when, and where functional gaps need to be plugged to improve the user experience. At the same time, marketers must become aware of what is engaging the developer’s attention so that a coordinated effort can be railroaded to smooth out product promotions and software rollouts.

    Whatever misgivings marketers and DevOps teams may harbor regarding each other, both are locked in a mutually interdependent relationship in which cooperation and coordination are necessary to achieve business goals. Can we prevent marketers and DevOps teams from working at cross purposes? Yes, it’s possible if we’re clear regarding the principles and the culture promoting successful business growth.

    Marketing and DevOps Can’t Function in Isolation

    Marketing and DevOps are like two sides of the same coin that rings the cash registers in business growth. Their aims are the same. If Marketing aims to bring in new customers and plan strategies to target specific audiences, developers will focus on improving the product and speeding up delivery so that customers enjoy a friction-free user experience. This is a symbiotic relationship where the guy promoting products and reaching out to customers is as critical as the dude deploying customized code.

    You need Marketing to promote the software and an efficient sales team to improve the customer’s interaction with the product. The paying customer doesn’t stop at engaging the product; they’ll want to talk about it. The marketer will go social, they’ll go mobile, hold webinars, and host trade shows to forge a deeper connection with the paying customer because it’s not just about acquiring customers but of retaining customer loyalty to the brand. The point is that Marketing can’t do this in isolation; Marketing needs IT support and services that only DevOps can guarantee.

    Software Development and Product Marketing are Symbiotically Wedded

    In every aspect of business development, the software release cycle and sales funnel are mutually interdependent. When Marketing promotes products across social media and blog posts, and through channels such as emails and press releases, every software update launches simultaneously to enhance the user experience across all platforms.

    Strategic partnerships lubricate business growth, and partner programs and reseller programs are automatically updated by DevOps to ensure that the product is shareable with prospective customers. Each software update can be programmatically controlled as Marketing gathers and shares customer feedback. DevOps can leverage these insights to improve services.

    Marketing feedback is the driving engine propelling software development, and every feature request or design change improves customer satisfaction. It’s possible to make a significant improvement in customer fulfillment levels only if Marketing and DevOps work in tandem.

    CRM Optimization Peaks When Marketing and DevOps Collectively Analyze Data

    Predictive data analytics is a hugely collaborative endeavor where Marketing and DevOps teams need to get their act together. With the help from IT-driven automation, it should become possible to organize and view data regarding the customer’s browsing history and how frequently the customer interacts with the product. The consumer database mutually analyzed lays the foundation for formulating policies that promote customer retention and sales growth.

    The DevOps team should make it possible for customer relationship managers to view consumer usage and responses in real-time so that the feedback enables more satisfactory support solutions that enhance upselling options.

    Marketers essentially breathe content and life into digital ads aiming to multiply the number of paid users over time. DevOps, having built the product from scratch, is more aware of the product’s USP and can give a positive direction to marketers regarding the message that is conveyed to the consumer.

    What DevOps Ought to Be Telling Marketing: ‘We Believe in You’

    A common perception among DevOps members is that Marketing really doesn’t comprehend what the software is capable of doing. What DevOps needs to understand is the consumer is not as focused on the thrills and spills of the cutting-edge software that is packaged and presented to them. The consumer is more concerned about how the software can benefit their business.

    If consumers are not personally invested in the product and do not see something that is customized to solve their business issues, they’re least likely to buy the product. This is the reason why Marketing focuses a lot on tailoring the message to fit each demographic to fulfill a specific need. The message may not match the expectations of a developer, but it might indeed be what a customer wants to hear.

    For DevOps, testing or updating any kind of software involves strict adherence to a protocol of rigorous standards. The same approach can’t be expected of Marketing. Marketing is more invested in imagery and the message, which may differ drastically when different types of consumers are approached. In Marketing, strategies may change by the hour, if only to understand how different audiences resonate better with the product.

    This is where DevOps and Marketing ought to be communicating with each other through all stages of the sales funnel. If, for example, marketing is leaning heavily on mobile software but that happens to be the weakest link in the chain, it is for DevOps to step in and suggest features that deliver more value for the consumer.

    What Marketing Expects From DevOps: ‘Plug the Weaknesses and Play to Our Strengths’

    Improving Marketing-DevOps relations is vital to any growing businesses. Marketing can’t stop at tabulating requirements and then leave it for DevOps to come up with the perfect solution. The Eureka moment in software development comes only when marketing hand-holds DevOps through each stage of the development process. For example, marketing could become more involved in user model testing.

    DevOps knows the strengths and weaknesses of the software; Marketing is better aware of the audiences’ needs. Both teams have access to predictive data analytics that tracks consumer behavior concerning the product that is being improved. If Marketing can pinpoint user models that are not likely to work in a real-time situation, that model can be rejected at the drawing board. Proper communications and effective collaboration mean that the consumer is less likely to be presented with a product that promises a lot but delivers far below expectations.

    Conclusion

    If the universe of product acceptance and consumer satisfaction is like a gigantic jigsaw puzzle, Marketing and DevOps are the pieces that have to lock in place harmoniously to complete the picture. If parts go missing, it will show by way of an inferior product.

    Even if their priorities differ, both Marketing and DevOps have to work in harmony to fulfill business objectives. Mutual trust and communication are two vital cogs that turn the wheel of progress and business growth.

    The future will be molded by teams of DevOps, Marketing, and Sales that work together, aggressively analyzing consumer feedback to deliver user experiences of high quality consistently.

    What marketers and developers should be asking themselves is, “Am I a Lone Ranger riding off into the sunset?” or, “Am I a team player guiding the company through wild terrain, conscious that there’s a destination to be reached together?” The answer determines your business trajectory.

    — Josh Harisson

  • The Third Way of DevOps: From Knowing to Being

    The Third Way of DevOps: From Knowing to Being

    Why is the Third Way of DevOps so difficult to master?

    We all know the feeling. You’ve seen an opportunity to improve things and genuinely want to do something about is. But you soon feel the clarity and anticipation draining out of you. First there’s the pressure of the day job. Then there’s the inertia of your co-workers who are oblivious of, or indifferent or hostile to, your ideas. And finally, as if these concerns weren’t enough, you’re faced with the difficulty of unlearning old habits and developing new ones. You wonder why you bother. In a way, it’s a bit like a lottery: You buy a ticket fully knowing that the odds are stacked against you, but it’s the worth it for the uplifting short and illusory dream that it might just happen.

    This is, of course, all about the Third Way of DevOps—creating a culture that fosters two things: continual experimentation, which requires taking risks and learning from success and failure, and the understanding that repetition and practice are the prerequisites of mastery. It all sounds great in theory, but why is it so difficult to put into practice?

    Information Abundance  

    It’s not due to lack of knowledge and skills. We have more information available to us and more easily accessible than ever in the history of our planet. We can thank technology for that. But has it really made us more effective at what we do? If I know something and don’t apply it, how is that so different than not knowing it at all? If I know how to read but choose not to, how is that so different that not knowing how to read? If I don’t use what I know, how long will I retain it and how will I integrate it with the other things I know? Most importantly, can and will I apply what I have learned to make things better?

    Lifelong Learning 

    We have been learning and studying all our lives in an attempt to make things better: our ability to understand IT, frameworks, methodologies, capabilities, practices, work processes and, of course, tools. Consider how many thousands of hours you invested in school, self-study, read out of curiosity, listen to podcasts and do work-related training and certification. Most of my friends proudly describe themselves as “lifelong learners” and one claims the day they stop learning shall be the day they die!

    Training in Skills

    We’ve also invested considerable time and effort into improving ourselves so that we may work more effectively with others: communication skills, teamwork, shared rituals, structured problem-solving, leadership and coaching all fit into this category. Who (in IT) has not been a part of some Service Management, Lean, Agile, Scrum, Kanban or DevOps training? After all of our training, you’d think we’d be further along than where we find ourselves.

    Knowing and Understanding 

    Knowing and understanding are not the same thing—I can know the Four P’s of IT Service Management (People, Process, Products and Partners), yet not view it as a lifecycle model and understand that to obtain the benefits of this knowledge, my team must determine the roles of people and objective of work of processes and then implement tools to automate the processes enabling people’s roles and tasks.

    Understanding and Doing

    Nor are understanding and doing the same thing—I can know that structured problem solving is based on the scientific method and can be broken down into four stages (Plan, Do, Check, Adjust). I may also understand that structured problem-solving is preferable to the reactive educated guesswork my team engages in whenever it encounters a problem. But if we do not change our behavior when it comes to problem-solving, our understanding will not manifest as action.

    Doing and Being 

    Neither are doing and being the same thing—I can be doing something and still not have internalized it so that it becomes who I am. I can know the core principles DevOps (engage in systems thinking, amplify feedback loops, foster a culture of experimentation and learning). I may understand that all three principles must be applied to foster a sustaining DevOps environment. I may be even be holding initial planning sessions with my team during which we map out the DevOps value stream, identify bottlenecks, create feedback loops and introduce changes to create a create a culture of experimentation and learning.

    Moment of Truth 

    But what happens when we meet with our next P1 incident? Does the knowing, new understanding and behaviors get tossed aside while we fix the problem, or do we hold fast to the new way of doing things as we grapple to not only remediate the incident but to view it within the context of DevOps?

    So, What’s a Person To Do?

    The velocity and degree to which we can move from knowing to being determines how effectively we can apply what we know. Here are a few things you can do to smooth the transition from knowing to being and help put the Three Ways of DevOps into practice:

    1. See and feel the potential impact from moving beyond knowing to being
    2. Use the uplift to get motivated
    3. Take action by changing something within you and your team’s circle of control
    4. Reflect on the fact that you have realized potential (this is very powerful when you share it with your team)
    5. Check and adjust based on the outcomes of step #3
    6. Allow the feedback from the previous step to motivate you to carry on!

    Curiosity and Humility

    This approach requires a certain attitude. One of curiosity and humility. You need to be inquisitive so you keep striving to understand. And you also need to be unassuming and you see yourself as having plenty of room to learn and grow; being deeply (almost obsessively) interested in what’s going on and how things could be improved; trying to be aware of and to set aside any preconceived ideas—adopting a beginner’s mind.

    To Be Continued …

    In our next post, we’ll explore specific steps you can take to leverage The Third Way of DevOps and make tangible progress based on what you know and don’t know, understand and don’t understand, and do and don’t do.

    This article was co-authored by Mark Smalley.

    — Mike Orzen

  • The State of Web and Mobile DevSecOps

    The State of Web and Mobile DevSecOps

    As the leaders in coverage of the DevOps world, the reporters here at DevOps.com are constantly pressed upon to convey the true state of DevSecOps at any given moment in time. We try to dig up the state of the union by asking important questions of IT executives, DevOps champions, security leaders and other pundits. The answers we get offer illuminating narratives of the DevSecOps journeys in global organizations across a wide range of maturity levels. But, at the end of the day, these are anecdotal at best.

    We know that nothing beats hard data, which is why we’re partnering with NowSecure to field a new survey that we hope will offer solid numbers around secure DevOps maturity and adoption practices.

    With this survey, we hope to do a couple of things: First, we obviously want to provide insight into how well organizations are automating their software delivery and application security (appsec) testing practices across their entire software portfolio. We’d like to understand the perceived benefits organizations have for folding security into DevOps and the inhibitors.

    Second, though, we want to take a deeper dive into where DevSecOps is specifically when it comes to delivering mobile applications. There’s not a lot of visibility into whether there’s a different set of standards, practices or benchmarks for mobile software, but we have a hunch there might well be a gap. We can’t be sure until we’ve got some data to show where the industry stands when it comes to mobile DevSecOps.

    Which is where our esteemed readers come into the picture.

    We’re asking executives, managers and practitioners to take our new survey and let us know how things stand in their organizations. We’re looking for respondents from a range of backgrounds—be they DevOps engineers, developers, architects, security pros, operations gurus or QA professionals. Help us out so we can provide important data back to the industry.

    — Ericka Chickowski

  • How to Become a Great Engineering Manager

    How to Become a Great Engineering Manager

    I wrote recently about the attributes of a 10x software engineer and a 20x one. Consequently, I received a few comments suggesting I create a similar list for engineering managers. I checked online and found a few articles, but none conveyed a full list in my mind. If you consider that software development efficiency has become the No. 1 priority for executives, ranking higher than access to engineering talent or capital, then I guess that’s a topic worth discussing! Engineering managers just might have the biggest impact on the productivity of their engineering teams.

    I’ll first go over the attributes for a good engineering manager, and then list what makes a great one.

    The 10 Attributes of a Good Engineering Manager

    I tried to prioritize this list, from essential to bonus. Feel free to give me your opinion on this in the comments.

    1. Be trusting

    If you refer to what destroys developer productivity most, trust is at the heart of it. Managers need to trust their team and give them some space. Micromanaging means undermining the team’s skills and ability to get things done, and therefore their motivation and self-confidence (which, by the way, they need to address difficult problems). Trust between the managers and their teams should be the foundation of their relationship. And it’s the manager’s role and job to create it, not the other way around. So first, trust your team.

    1. Emotional intelligence and ability to establish trust

    This is linked to the first point. You need mutual trust within a team for it to work well. And again, it’s the manager’s job to ensure that their team trusts them. How does the manager do this? By showing emotional intelligence.

    Engineering managers must be sure to make themselves available for questions, even if they’d rather not be interrupted. This can run counter to high efficiency, but it’s important that managers spend the time to form personal connections, so honest and open conversations happen naturally within the team and with the manager.

    Another important point is to listen to anyone and everyone. When the team gets together for a meeting to sort out a project they plan to do, everyone in the room needs to be heard and considered. The manager needs to respect any experience or insight their team can bring to the table. Any member of the team may be the most knowledgeable on a particular subject.

    Now, this is not Disneyland for sure: Managers might need to be tough and unwavering sometimes. But when that’s the case, it’s important to be transparent about the reasons to keep the hard-earned trust intact.

    1. Ability to shield the team from interruptions

    Interruptions and meetings (consider them planned interruptions) are greatly loathed by developers, and for good reason. They literally destroy their productivity. Developers love projects that move fast; they’re invigorating! Managers would do them a great service by removing distractions, ensuring that they have large chunks of uninterrupted “maker time.”

    A classic example is a really long email exchange on some technical topic with no resolution. The manager could simply put everyone on the thread in one room together for 10 minutes to get the issue resolved promptly.

    1. Ability to manage the team’s motivation

    The team’s motivation is directly linked to their output and the success of their releases. There are many ways to keep the motivation high. Here are just a few:

    • Help the team see their work through the lens of their impact on the company.
    • Don’t worry too much about failure. Things will always fail: builds, deploys, schedules missed, etc. If things fail, take responsibility and focus on the learning. The point is not to have to fix the same issue twice. This will give your team a mindset toward action and growth.
    • Celebrate every achievement, whatever its size. And credit the participants profusely.
    • Be transparent. Default to overcommunicating. While some leaders believe they need to shield their teams from things going on at the executive level to prevent distraction, there’s a difference between productive shielding and unintentionally denying engineers important company context. Managers want their teams to feel included and invested in the direction of the company.
    1. Strong work ethic and cohesion with company culture

    Work ethic means being exemplary in all the ways you expect your team to behave. Lead by example. Managers must embrace the company culture, or it will create confusion sending team members mixed signals of whether or not they should embrace the company’s culture.

    This also means shielding your team from people who don’t act accordingly. For instance, hiring someone with a huge list of qualifications and experience can still be a disaster if they have a different set of values than those held by the company. Or, unfortunately, having to let people go who are not team players or aligned with the company and team culture, whatever their skill level.

    Good managers keep in mind that the team’s productivity as a group is important, not individual performance.

    1. Advocate for change to seek positive evolution in a constant manner

    Good managers install a mindset of positive evolution within the team.  They adopt a state of mind that continuously seeks to improve a certain situation. When a process is slow or cumbersome, they try to turn that around and make it better. One way todo this is using OODA loops: Observe, Orient, Decide, Act.

    1. Ability to work with departments outside of engineering

    The language used by non-technical stakeholders might be very different than that of the development team. However, good managers don’t want to be the Single point of contact, as they would introduce a potential liability into the project—a strong dependency on them. They tend to include their team in relevant discussions, but make sure they protect the team members from continuous interruptions. Good managers try to be the First point of contact, which is a big difference from being the single point of contact.

    1. Mentoring abilities

    Effective managers are measured by the effectiveness of their software teams. Anything that they can do to make each person on their team better makes their overall teams exponentially better. And most of the mentoring is actually about matchmaking team members with high-impact projects that align with their interests, the value they want to create and how they want to grow. That’s when motivation becomes organic.

    There are already so many articles on one-on-one meetings, I won’t spend much time on them. But mentoring also happens every day at every occasion. For instance, if a manager happens to review code or explain certain conventions, they make sure to clearly explain the reasoning why the team is doing something in a particular manner. Good managers are great teachers.

    1. Technical literacy and problem-solving skills

    On the one hand, technically competent managers will have a superior handle on the work that they are managing, and will therefore be better able to make recommendations and evaluate necessary trade-offs. On the other hand, technically talented managers are often better able to garner respect from their teams. Non-technical managers may struggle to manage high-performing individual contributors, who tend to have a tough time taking orders from managers who don’t fully understand their craft.

    But, if you don’t have any engineering background, it doesn’t mean you cannot be a good—or even a great—manager. I would advise you to learn about the technical dimension of the project (with humility) through every interaction you have with your team, and possibly in your off hours. Having enough technical experience to understand the projects and the decisions, mixed with the ability to trust others with technical depth, works really well.

    1. Ability to make hard decisions and manage conflicts

    Good engineering managers will make the courageous decision, even if it isn’t popular. They don’t make the wrong decision just because it is socially right and their team suggests it. Good engineering managers focus on the road and the values they believe in, and not only on the immediate issues they are facing. They don’t just choose best industry practices because they will attract the “right” talent. For instance, managers will avoid  automatically using the latest technologies: they will weigh the pros and cons and make the best choices based on the resources and requirements of their team and company.

    How Can You Become a Great Engineering Manager?

    1. Embody the vision

    Great managers have a natural ability to visualize the future of the company and motivate others to work toward those lofty yet realistic goals. They communicate the company vision at every opportunity by illustrating it contextually in discussions. They challenge the decisions made with, “What if we had 20x more users? Would that be sustainable?” or, “Will this technology decision still be valid in 24 or 36 months?”

    1. Support your team in the short and long term

    At the end of the day, as a technical manager, your job is mostly about humans. Great managers think about supporting their team in two ways: day to day and year to year.

    Great managers help their people devise a long-term plan to be happy and productive at their company. Sometimes, engineers don’t know how to articulate how they want to evolve. This is not an issue, as great managers have built strong relationships and trust with their team members to help them uncover this part and build a plan for:

    • the skills to improve.
    • the technical/non-technical experiences they want to have.
    • how they want to increase the scope of their impact at the company.

    And, naturally, the great managers will help their people to achieve their ambitions (within or maybe even outside the company).

    1. Challenge the team to grow faster

    Great managers try to super-charge learning within their team. They know to push their people out of their comfort zone and challenge them to accomplish things they didn’t even know they could do. But in doing so, they set up strong guardrails to help their team course-correct along the way.

    For instance, one way for engineers to increase their impact is to architect and own increasingly complex technical problems requiring novel solutions. Great managers will throw their people  into the deep end of the pool, with the right mentorship and guardrails, and have them start solving increasingly complex problems immediately. They will also try to maintain a balance between more experienced and less experienced engineers, having them work together and regularly share knowledge. They want to build in opportunities to make sure these conversations are truly bi-directional.

    Another instance is estimates. Do you know this law?

    Hofstadter’s Law: It always takes longer than you expect, even when you take into account Hofstadter’s Law.

    Estimating is hard. When you do it more often, you will get better at it … but you will still get it wrong from time to time. Challenging your team to make estimates is a way to get them used to getting out of their comfort zone.

    1. Embrace the differences

    Diversity is invaluable. All people are different and live different lives. This is incredibly valuable, because your users will also be different. Surround yourself with passionate people of different backgrounds. By background, I don’t only mean gender or race, but also education and past experiences. The more people in the team think differently, the more approaches your team will consider when addressing a problem, which results in better decisions. A great manager understands this and will strive to get input on the same problems from a variety of people with different backgrounds.

    1. Level up the product and engineering partnership

    Great engineering managers know to extract the essence from the user story; they focus first on the core capabilities, providing valuable input and challenging the product as early as possible. They never blame the product on complex features, or make the excuse that some things were not explained in the first place or changed 10 times. They never complain about the lack of visibility on the ideation process. They build such visibility themselves. They challenge the product team with several alternatives for the same idea. They know to explain pros and cons without deep diving into tech level and buzzwords. They help the product team make better decisions and educate them about the technical layers of features.

    1. Know themselves and educate others about themselves

    Great engineering managers have made the effort to understand how they themselves work,  behave,  think and make decisions. Being transparent with their team about these things, builds more trustful and productive relationships. They work on visibility and commitment for their own goals and tasks in front of their team members.

    1. Push their team for ownership as much as possible

    Great engineering managers tend to give more ownership of user stories to their developers. Transferring ownership to developers gives them more motivation to do a great job. Great engineering managers don’t try to own all of the responsibilities or give them all to a few select  individuals (tech leads).

    Great managers know when to give input, when to make decisions, and when to step back and allow the team to take more ownership. They promote a culture of accountability, where they trust each team member to do his or her job and never  insist on full visibility into what their team is doing.

    1. Understand the system’s bigger limitations

    Great managers understand the inefficiencies within the processes and organizations, as well as their own limitations. They share their thoughts to upper management to help the organization grow (and not only their team).

    This concludes my list of the attributes of good to great engineering managers. If you think of any I’ve overlooked or have any ideas or suggestions to add, I would love to hear your thoughts! Please leave your comments and opinions below!

    — John Lafleur

  • Software Delivery: Shaking Loose of a Local Optimum

    Software Delivery: Shaking Loose of a Local Optimum

    This is an article about software delivery, that magical or onerous field between pushing code to version control and using it in production. Why does software delivery matter? Because software matters, and because software developers matter. I don’t want my software delivery to get out of the way; I want it to make me a stronger developer.

    The path from code to production is crucial because it determines the length of our feedback loop and, therefore, our rate of improvement and that all-important lead time.

    Our current tools think of delivery as generic—they take control, and we fill in the blanks. They let us build, test, deploy. But these words mean something different for every application, on every team and within every business. And there’s a lot more that goes on, or can go on, between the writing and the running of code. There’s often linting, approving and coordinating. There’s data to think about and security policies, business team structure and auditing. There are other steps we don’t bother with, because there’s no slot for them: updating libraries, fixing the code to meet standards, taking a different set of steps for a documentation-only change.

    What else have we not thought of, because it doesn’t fit inside the current set of tools? I want to think about delivery from the top. In doing so, I can gain a greater understanding of my work.

    We’ve seen major shifts in Operations technology: containers, log aggregation, observability, programmable infrastructure and Kubernetes. In my opinion, the killer feature of Kubernetes is its API—it is controllable by automation. It accepts instructions and sends out events.

    The potential here is even bigger than we have realized, for operations and for delivery.

    Modern options for operations have achieved new levels of abstraction and potential. Yet progress in delivery has only been incremental.

    Our tools are very configurable, with plugins and scriptable bits, and we can always fall back to running shell scripts of unlimited length. But are we stuck in a local optimum? Are we thinking big enough?

    What We Have

    Today’s popular software delivery tools are fill-in-the-blank. This is great for getting started, when you don’t know what you’re doing. What about when your business grows, when your delivery is no longer cookie-cutter? Let’s look at two examples.

    Travis CI is a favorite of mine because it’s so easy to get going. It will happily build any public GitHub repository, on its infrastructure, for free. If my repository fits a few simple profiles, such as a Rails app that deploys to Heroku, poof! Some very simple YAML in your repository:

    I like that YAML. It is declarative. It contains data. It fills in the blanks that Travis provides. But the real measure of a tool comes when you get off the happy path. You can do just about anything in Travis; just call out to a shell script. Here’s a small piece of one of my .travis.yml files:

    It has shell commands. Shell syntax for an if-condition. And that infinity of flexibility, a call to a shell script. That travis-build.bash script happens to contain four functions, eight conditionals and (if I’m reading it correctly) up to seven program calls.

    This is programming, OK? As soon as your YAML calls out to a shell, it is not declarative. It is imperative. Or a mix. 😕

    This is programming, but my YAML and Bash do not qualify as software. Bash is a scripting language. It is designed to glue programs together, not to construct programs; it lacks composable modules with encapsulation. I can live without types, but in Bash you don’t even declare function parameters! And it’s full of pitfalls. Use Bash for what it’s good at: calling other programs. As a fallback for flexibility that we lack at a higher level, Bash does not shine.

    YAML is not a programming language, although it does support variables and has its share of pitfalls. Until the YAML contains commands that get executed; then it becomes a program, with even fewer options for error-handling or clear naming than in Bash. Please use YAML for data, not for code.

    There is a gulf between programming and software. We have standards for software: it has design—domain-driven design that we’ve worked through with experts. It has automated tests. It has modularity. It uses libraries and abstractions. I want to use a language designed for all this.

    Skip ahead to Jenkins. This is more widely used than Travis for more extensive delivery flows. It can run on-prem. There are hundreds of plugins available. So many blanks to fill! So many options! And now, there’s pipeline as code. For one thing, you can write a piece of Groovy to define which repositories and branches get a pipeline. That makes Jenkins more responsive to new repositories and branches. It’s a step in the direction of responding to new projects.

    The people building Jenkins recognize that delivery needs to be controlled by the people who control the code. Instead of constructing each repository’s build in a GUI (hello, TeamCity), we put a Jenkinsfile in each repository. That file is in Groovy. Groovy is a programming language with all the modularity and libraries of the JVM behind it. Except, you don’t get all that in your Jenkinsfile. You’re limited to what is available when Groovy runs within Jenkins. If you want consistency from one repository to another, you can install custom-shared libraries in Jenkins and call them in each Jenkinsfile. Rolling out change is challenging; any of those disparate Jenkinsfiles can break after the shared library changes. It’s pretty fragile and still limited. It’s progress.

    This is a step up as programming, but I still wouldn’t call it software. When I write an application, I work in more than a programming language; I work in a language system. A language system includes dependency management, frameworks, linters, build systems, editors, community and the whole ecosystem of libraries available in the language. Great language systems exist around the JVM, Node and .NET, for instance. Groovy that you run can participate in the JVM language system, but Groovy running within Jenkins does not have this access. The Jenkinsfile DSL is still fill-in-the-blank.

    What We Need

    Conceiving delivery as software invites us, development team members, to sit down with the local experts on our deployment environment and security and other concerns. It invites us to have a conversation: What does delivery mean for this application, on this team, in this organization? Where do we want consistency and variability? Then we can create and encode a model—a model of the domain of software delivery, within the context of our business and our environment. We can brainstorm events such as a push or a new repository and what we want to always happen. We can divide the parts into team-specific, organization-specific and community-wide (open source).

    Instead of, “How do run the build? How do we run tests?” we can ask, “What needs to happen after a code push?” And we can think carefully about how we work, how our software works and how our business works with it. We are the domain experts in our own delivery.

    This is the top reason I want delivery as software from the top down. By thinking about delivery as software, we think rigorously about our work as software developers. And we get better at it.

    I want more than delivery-with-some-code. I want delivery as software. Controlled top-down by my team, with a rich ecosystem of libraries, with frameworks and services for the parts that are common in the community. I want delivery that’s event-driven, that’s testable and that communicates with me when it’s time for a decision.

    We’ve come a long way since the days before automated deployment. Our existing continuous integration/continuous deployment solutions have served us as far as they can. Incrementally better (fill-in-the-blank) won’t take us to the next level of software delivery, and it won’t help us reach the next level as software developers. It’s time to break out of this local optimum.

    Let’s Go

    To this end, we composed the Software Defined Delivery Manifesto. Say what you want about manifestos: They have a certain flair, a certain impact, a certain dramatic longing for a better world. That is what we are expressing here. Yes, it’s a better world only for development teams, and only for development teams in complex organizations. But we matter! Software has a huge impact on the world, and to make that software better, we need to get better at delivering it. Let’s push past what we’re used to. Accept that our job is not writing code, but running useful software in production. Let’s get better at delivery by modeling it in software.

    If you’re in, join the movement. Sign the manifesto.

    — Jessica Kerr

  • 5 Questions to Ask When Choosing a Cloud-Native Security Platform for DevOps

    5 Questions to Ask When Choosing a Cloud-Native Security Platform for DevOps

    If you work in DevOps, you’ve likely heard the mantra that DevOps success is all about the right people, tools and processes.
    Below, we take a look at how you can address the “tools” part of that equation when it comes to security. Specifically, we’ll discuss which questions to ask and what features DevOps teams should look for when choosing a security tool for today’s cloud-native infrastructures.

    What is Cloud-Native, and What Does It Mean for DevOps?

    Our discussion is informed by two main considerations. First, we’re going to focus on the security challenges that arise in cloud-native environments—which means those built with technologies that include, but are not limited to, containers, serverless functions and (of course) virtual servers.

    Second, we’ll focus on the security needs of DevOps teams. Although DevOps positions may not explicitly involve security, we now live in the age of DevSecOps, and everyone on the DevOps team has a role to play in keeping infrastructure and applications secure. In this article, we’ll consider which types of tools are best-suited to help DevOps teams do that, while also promoting the visibility and collaboration that are essential parts of a healthy DevOps strategy.

    So, those are our goals. Now, let’s look at the questions you should ask as you evaluate cloud-native security platforms for a DevOps organization.

    What Are You Actually Securing?

    This question may seem obvious—so obvious that you may not give it much real thought. But because today’s infrastructures and environments vary so widely, it’s important to step back and figure out what, exactly, you need to secure.

    Are your workloads running in containers? Are you also using serverless functions, or do you plan to add them? How are you orchestrating your workloads? (With the native orchestrator provided by your cloud vendor, Kubernetes running as a service, your own Kubernetes build or something else?) Which new cloud-native technologies do you expect to adopt in the future?

    Answering questions such as these is important to ensure you choose a security platform that can support all of your current and future cloud-native environments. In most cases, you’ll find that security platforms that are purpose-built to secure a range of environments (not just containerized ones, which are usually the focus of most self-proclaimed “modern” security platforms) are the best and safest fit for your needs.

    What Are Your Security Threats (and How Can You Stop Them)?

    This is another question that might seem overly obvious. But here again, the fast-changing nature of threats means that it’s worthwhile to take some time to assess what your threats actually are.

    Keep in mind, too, that the threats faced by your particular team, or the app you deliver, may be different from those that threaten other teams. This is one place where the DevOps principle of cross-organization communication is crucial for effective security management.

    Which Layers Does the Security Platform Secure?

    Scanning container images for known vulnerabilities is good. On its own, however, it hardly amounts to a complete container strategy. The same could be said for setting up a firewall or locking down access control.

    To achieve true security, you need to secure all layers of your infrastructure (including those managed by other teams) against all vectors of attack. For that reason, you want a cloud-native security platform that is designed for holistic security, not a tool that only secures one or two layers.

    Where Does the Security Platform Get Its Vulnerability Information?

    When it comes to identifying vulnerabilities, security platforms can get their information from lots of sources. They could look at a public CVE database or at a list supplied by the tool’s vendor.

    The best security platforms, however, will pull vulnerability data from multiple sources. After all, if you’re only relying on one data source to figure out where the threats are, you are unlikely to catch them all—and just like in the world of Pokémon, catching them all is one of your main priorities for DevOps security.

    How Automated is the Platform?

    Automation is the mother of DevOps (or something like that).

    What I mean is that without automation, you can’t do DevOps very effectively.

    You also, incidentally, can’t do cloud-native security unless you rely heavily on automation. That’s because the highly dynamic nature of containerized, serverless and other cloud-native environments means that trying to interpret all of the data they generate, identify vulnerabilities and react to them manually just doesn’t work. That’s why you want a cloud-native security platform that automates wherever and whenever possible.

    You should still expect to have to perform some tasks manually, of course (which is actually a good thing—if we could automate everything, DevOps engineers wouldn’t need to exist anymore). But to the extent possible, your security platform should automate your security-related workflows.

    — Sonya Koptyev

  • 3 Tips to Build A DevSecOps Organization

    3 Tips to Build A DevSecOps Organization

    For organizations undergoing digital transformation today, modernizing the existing environment can present serious challenges when it comes to security. Whether you’re dealing with a transition from legacy to cloud or hybrid models, or shifting from ITIL and waterfall methodologies to DevOps, the increasing number of technologies and complexity mean you need to rethink your security posture. The traditional approach to security—that is, a static strategy that largely involves patching on piecemeal measures at the 11th hour—is simply not a winning formula anymore, especially when dealing with requirements such as privacy compliance or contractual agreements.

    DevSecOps represents a fundamental shift in which real business needs drive a dynamic, living/breathing approach to security based on continuously changing requirements. To evolve from DevOps to DevSecOps, an organization must focus on integrating security into the very fabric of the software development cycle, and work to increase intelligence, situational awareness, and collaboration.

    Here are three critical ways to consider to ensure your DevSecOps strategy is up to snuff.

    Create a Single Source of Truth

    When you have multiple teams trying to work at breakneck speed, having one absolute source of data is essential. Gone are the days when we could rely on static spreadsheets that lived locally on this or that person’s computer, and even communication mechanisms such as email are too manual and out of sync to be trusted. What’s more, it’s impossible to draw meaningful correlations and map trends if your data is sitting in silos across your organization.

    Creating a single source of truth will ensure the greatest accuracy of information for everyone. You need to pinpoint where your data is coming from, how it should be collected and how it should be shared. You’ll want to integrate your full tool stack and workflow, and harness automation to streamline hand-offs between collaboration tools, system updates, chatbots and more.

    Shift Left: Secure By Design

    Relying on firewalls and antivirus as your primary security measures is a bad, bad habit. The key is instead to shift left of these elements and work to embed privacy from the start. This is the new age of security, using a risk-based approach instead of a reactive one—that is, identifying what needs protection, why it must be protected and how you will do so. It’s also understanding that security should not be just an external threat perspective, but also having visibility into what’s happening internally.

    The key drivers of your secure-by-design approach for DevSecOps should be privacy compliance, contractual obligations and threat models, both known and emerging. The goal here is to be proactive and avoid spending valuable resources to address threats or vulnerabilities that could have been prevented. And in case you didn’t already know, properly written, patched and documented code accounts for about 80 percent of security!

    Understand Your Environment

    If your organization has embraced DevOps, then you’re likely aware of necessities such as process, collaboration and automation. However, these can sometimes come at the expense of other important things, including privacy and security. A lot of this is due to lack of oversight and poor visibility into change management.

    As organizations accelerate their adoption of cloud services, threat vectors are ever-expanding. As such, you need to have complete situational awareness of your organization. You need to know what to monitor for and when, and this cannot be limited to the events directly related to security. Instead, focus on extending your perimeter of knowledge beyond your DevOps pipeline and ensure you’re monitoring everything from operating system logs and directory systems to DNS and servers. Without all of this context, there’s simply no way to correlate security incidents with other data from your IT environment. This is the information you need to document processes, workflows and playbooks, and ensure your teams can communicate and collaborate rapidly to address issues before the business is impacted.

    Remember, when it comes to the ultimate big-picture goal of DevSecOps, it’s always about minimizing the financial impact to your organization. Whether we’re talking about your reputation or lost time and resources, the bottom line is dollars down the drain. And the data is compelling so far—in its “State of DevOps 2017 Report,” Puppet found that high-performing DevOps teams experience a 96X faster MTTR from downtime and a 5X lower change failure rate compared to lower-performing teams.

    This article was co-written by Domnick Eger.

    — Robert Hawk

  • Toyota Research Institute Discusses DevOps Automation and Rapid Innovation

    Toyota Research Institute Discusses DevOps Automation and Rapid Innovation

    How TRI is using DevOps automation to drive its research and engineering

    Toyota Research Institute (TRI) is working to build a future where everyone has the freedom to move, engage and explore with a focus on reducing vehicle collisions, injuries and fatalities. Working tirelessly behind the scenes is TRI’s Infrastructure Engineering team, responsible for designing, deploying and maintaining the infrastructure that makes this possible.

    I recently sat down with Mike Garrison, the technical lead for Infrastructure Engineering at TRI, to talk about DevOps automation and cloud-based deep learning. (Full disclosure: TRI is a customer of Flux7)

    Note: Mike Garrison and Adrien Gaidon, Machine Learning Lead at Toyota Research Institute, will share more about their experience with Distributed Deep Learning & High Resolution Driving Data at this week’s AWS re:Invent.

    Can you start by giving us a broad idea of the project you’re working on at TRI?

    Our mission at TRI is to improve the quality of human life through advances in artificial intelligence, automated driving and robotics. The Infrastructure Engineering team supports the researchers and engineers at TRI by making it easier for them to utilize the power of the cloud in a secure, automated and reliable fashion while not slowing them down. We see ourselves as partners who can help reduce the time and burden, so they can focus more on their deliverables.

    Have you evolved as a team to enable the rapid innovation that this kind of project needs?

    Yes. The biggest change we’re making is an evolution to embrace more DevOps ideas, methods and processes.  Automation is a core tenet of DevOps. We are applying that in two ways. First, to reduce tactical, manual IT operations activities; second, to make it faster and easier for our researchers to test their ideas.

    For example, we are creating a self-service portal that will allow researchers to easily and quickly provision the AWS assets they need to test new ideas. This helps researchers become more productive as they won’t wait for the infrastructure team to spin up resources. Having a secure cloud sandbox environment enables them to try new ideas, fail fast, destroy the sandbox if needed, and start over. It’ll enable our researchers to innovate at velocity and at scale. Modern cloud infrastructure and DevOps automation is empowering us to quickly remove any barriers that get in the way, allowing the team to do their best work, advance research quickly, push boundaries and transform the industry.

    How are you applying the latest technology advancements to help produce cars that are safer, more accessible and more environmentally friendly?

    The process of developing autonomous vehicles is data intensive. At TRI we log a massive amount of data—terabytes of data per car per day—from our fleet of test vehicles. Researchers analyze the data and continuously refine machine learning algorithms to ensure the most accurate outcomes possible across a myriad of driving scenarios. Researchers train new models and update test cars for new test runs. We’re also using simulation to virtually recreate traffic situations, adding adverse conditions such as bad weather and difficult traffic patterns. It’s a process of continuous improvement that simply couldn’t be achieved without the high compute power of the cloud.

    Supporting this high-powered data analysis is a cloud infrastructure that has been optimized to minimize the amount of time researchers take in this gather, learn, tune and test process. For example, at TRI, we have two driving technology modes—Guardian and Chauffeur. Guardian is designed to intervene only when a driver needs help avoiding an accident, whereas Chauffeur is meant as a truly autonomous vehicle. We developed a single technology stack to support both modes.

    And, using the PyTorch deep learning framework, my teammates on the machine learning side developed a series of computer vision models that monitor and control both modes as well. Working together, we are able to collect information from their deep learning models, spinning up new compute and storage resources on demand. We’ve married this elasticity with advanced management and orchestration services to ensure that collected data can be retrieved and analyzed quickly. Through advanced automation, we are able to shrink model training times so they are measured in hours—or even minutes.

    What lessons have you learned along the way? Any barriers or surprises that others could learn from?

    While DevOps automation requires more upfront work, it pays dividends in time savings, staff being able to focus on strategy rather than tactics, and provides logged history which can be critical for security, problem resolution and much more. If I had any words of advice for others looking to adopt DevOps automation, I would advise them: Don’t try to go back a year or two later and try to reverse engineer what was done. Start on the right foot and automate as much as you possibly can.

    — Aater Suleman

  • Upskilling: The Enterprise DevOps Skills Survey and Report

    Upskilling: The Enterprise DevOps Skills Survey and Report

    There has been a lot of recent reference to “T-shaped” IT professionals—individuals whose core competence (the stem) is supplemented by a broad range of other knowledge including soft skills (top of the T). T-shaped talent is in such high demand with a limited supply that recruitment efforts have become very competitive for those with upskilled and cross-skilled transformational know-how.

    Unfortunately, the migration from a deeply skilled specialist to a T-shaped practitioner is leaving many confused about what skills they should be acquiring and hiring and when. That’s why DevOps Institute (DOI) recently launched the 2019 “Upskilling: Enterprise DevOps Skills Report” community survey to help individuals and enterprises better understand the current and future state of DevOps skilling across the global IT community. Using data science coupled with hiring manager interviews, DOI’s goal in fielding the report is to provide tangible insight into why and where IT practitioners, managers, leaders and business stakeholders should focus their efforts in growing personal and organizational skills portfolios.

    Eveline Oehrlich, a former vice president and research analyst at Forrester, is bringing her data science expertise as the lead analyst for the report. Founding Platinum sponsor Electric Cloud and founding Gold sponsor Cloudbees are also sponsoring this important project. The “Upskilling: Enterprise DevOps Skills Report” will be a year over year project, and the results will be published in Q1 2019.

    The global survey will remain open until Dec. 7 and can be accessed through the following link: https://www.research.net/r/K9WD2XC

    We encourage everyone in the DevOps community to complete the survey so that the data samples truly reflect the global state of DevOps skills.  Any organization that has at least 20 people complete the survey will get a personal snapshot of the data results. All respondents who complete the survey will be entered into a drawing for four $100 Amazon gift cards.

    — Jayne Groll