There are two basic design methodologies in the software development world: top down and bottom up. In bottom up, the goal is to piece functionality together to “build up” to the desired result. In top down, the goal is to break the problem into smaller and smaller parts until you have implementable pieces.
Both have strengths, and by correlation, both have weaknesses. Analysis and design phases in pre-agile days were almost ubiquitously top down. They would take business requirements, translate them into functional requirements, then break those down and assign the constituent pieces to developers. As the code was implemented, weaknesses in the design would be exposed, and feedback would go back up to the level of functional requirements—or, in really bad cases, to business requirements—and a mini-iteration would take place to resolve the problems.
When agile came along, this was all too slow a process. To be sure, it was (and is) still used and has produced some of the world’s best software, but an agile team can’t sit idle waiting for a long process of analysis. Kan-Ban and similar tools/processes were introduced to address this issue, making the pieces being designed smaller and thus speeding the process somewhat. But smaller pieces implies bottom up to some extent, and the waters got a bit muddied.
Test driven development (TDD) was designed to make sure the code written met the technical requirement by writing a test for the expected result before the actual code. This was one of the most fruitful things to go into/come out of agile development. While it has flaws—the most significant being the human factor—it does make more thought go into code being written and more rigor come out of code produced. The act of writing the test makes the developer think of ways to solve the problem that satisfy the test. But the source of those requirements still needed to come from some form of process.
Enter behavior driven development (BDD). It’s not really development, though the name is catchy. It would be better named behavioral design for TDD, but that’s a mouthful.
In short, the idea is that business owners and analysts speak in terms of the needs of a given role, while developers speak in terms of functionality or the results of a given test set. This creates a gap that causes misunderstandings. The goal of BDD is to get everyone speaking the same language. It tries to extend the spirit of TDD up the design/analysis stack—”shift it left,” in today’s parlance—so that the tests being developed are a hierarchy stretching from acceptance testing down to unit testing, and everyone is speaking the same language.
En vogue today for the language is Gherkin, a simple requirement specification language that writes requirements and test criteria with (relatively) strictly defined sentences. I say “relatively” because, the language is a bit loose from a unit test perspective. But from role-based specification, it is locking down options to make the environment and test cases clear.
I actually got to thinking about all of this while considering SmartBear’s recent acquisition of HipTest. SmartBear has made an investment in making its tools BDD-compatible, and HipTest is largely a BDD platform. The combination of the two should see some movement in the BDD space that will be interesting. I spoke briefly with SmartBear about the acquisition, and it seems the company really is merging the strengths of the two organizations, so it is worth watching over the next few months.
Thus far, the benefits of BDD that jump out are the ability for everyone to talk about an element of the product in common terminology, while the developer can think “down” to the code required to meet the requirement and the business analyst (or owner) can think “up” to determine how to market this cool new feature.
Tools are out there to generate TDD test classes/cases for a variety of languages—notably from my stable, Python and Java. This might be a huge step in the right direction also. Just the other day I was having a bit of a lighthearted conversation with a peer and said, “Get to the point where you can auto-generate tests at build time for us, so IT managers aren’t stuck deciding which lines of code are/aren’t worth maintaining test code for, and then we’ll talk.” Tools and frameworks such as Cucumber and JBehave actually take steps in that direction, though their approaches are different.
We are not currently using BDD on our in-lab test application (Android/web clients, Spring API server) at work, but will have to consider it moving forward as another bit of DevOps tooling and practice we can experiment with. The tools are there, and the process bridges that sticky part of analysis and design where top down meets functional.
After all, the end goal is giving users what they need. If BDD helps make that process smooth and faster, it is a fit for any DevOps environment.
There may be nothing more difficult in a DevOps transformation than aligning the business with its technology organization—and vice versa. Dealing with skills gaps and information gaps, while addressing communication breakdowns and cultural hurdles can keep even the most seasoned practitioners and leaders up at night. How do we align disparate business units accordingly? What practical tips can people take and implement along their journey?
These are topics we immersed ourselves in during a live video podcast ahead of the DevOps Enterprise Summit (#DOES18) in London. I was able to sit down with Carmen DeArdo of Nationwide and Jayne Groll of DevOps Institute to take a deeper dive into the DevOps transformation patterns and practices. This discussion centered on a major theme of DOES18 this year: business and technology alignment for the modern enterprise.
We hit on many important issues that I think we can all learn from and apply in our organizations respectively. One of the things we’ve learned over the course of the years is that, in DevOps, the focus is heavily cultural, but it has to be squarely on nurturing and getting buy-in across the organization. But at the same time, we need to be orchestrating our strategic alignment initiatives between the disparate functions and organizations within the business. That’s a tall task.
Watch the full video replay below, and/or read through the full transcript to see what key points were made and how we can adopt best practices to move our businesses forward.
Carmen and Jayne will both be speaking at DOES18 London this year and I will be leading video panels that you can see exclusively on DevOps TV / devopsy.kinsta.cloud. Hope to see you there!
Alan Shimel: Hey, good day, everyone. Thanks for joining us. This is Alan Shimel of devopsy.kinsta.cloud. And I’m here to talk to you all today with a few friends of mine about the upcoming DevOps Enterprise Summit in London. This will be a live panel discussion ahead of the London DOES event, which, by the way, is June 25th to the 26th at the Intercontinental Hotel, sometimes called The O2. Our panel today, I’m happy to say, are two really good friends of mine. Well, one is one of my business partners at the DevOps Institute: Co-founder and CEO of DevOps Institute Jayne Groll, and a speaker at DevOps Enterprise Summit London. Jayne, welcome to our hangout today.
Jayne Groll: Hi, Alan. Hi, Carmen. Thanks a lot. Really looking forward to it.
Shimel: Okay. So Jayne has already sort of introduced our next guest there, another good friend of mine, a recognized DevOps leader for many years. He really kind of kicked off a lot of the kind of groundbreaking DevOps work that we’ve seen at Nationwide over the years. Also a great Pittsburgh Steelers and Pittsburgh sports fan, Carmen DeArdo. Carmen, welcome.
Carmen DeArdo: Welcome Alan. Welcome. Hi, Jayne. It’s always great talking to you.
Shimel: Thank you. So all kidding aside, both of you are kind of experts and leaders within the DevOps Enterprise community, and you’re both presenting at DOES London this year. You’ve actually both presented at previous DOES events as well. So thank you for coming here. You know what? When you guys first started kind of preaching in the wilderness about DevOps and so forth, it was a bit of the wilderness, but it’s becoming commonplace in businesses and organizations throughout the world now. Our goal on this chat, guys, is to learn about the patterns and practices that leaders have identified for business and technology alignment, and really what we’d hope is just to share your experience and thoughts with our audience so that maybe they can learn something and they could integrate or assimilate that into their own transformational journeys and escapades.
And just again a reminder, this is about DevOps Enterprise Summit London, June 25th to 26th, and you can get more information on the event at events.itrevolution.com. You’ll find on there, by the way, the entire conference agenda, and beyond Carmen and Jayne, it is a gangbuster lineup of speakers. If you’re interested in DevOps and any kind of related type of technologies and frameworks, etc., it’s really a can’t-miss one for those of us, you know, who follow this space, or even for those of us just getting involved in it.
That being said, though, I’ve spoken enough. Let’s get started. I’ve given a little bit of your background, but maybe I haven’t said everything that needs to be said. So, Carmen, since I see your lovely face on the screen ahead of me here, why don’t you go first? Give us your background, your company role, etc., professional accomplishments.
DeArdo: Okay. Thanks, Alan. So I actually spent most of my career in telecom working for Bell Labs. Thought I would retire from Bell Labs like everyone else at the time, and then the telecom bust happened, and we all had to figure out what we were gonna do with the rest of our careers. But it served me well, because I think a lot of those concepts, even though we didn’t call them DevOps, but, you know, certainly, creating high-availability, high-reliability capacity systems, you know, those concepts are still, I think, pertinent obviously today. I moved into Nationwide to really help with trying to get Agile set up about 12 years ago, and then, you know, as I think you see the case with a lot of companies, they start somewhere in their value stream. They may not even think of it at that point like an end-to-end value stream, but they’re starting somewhere. They start with Agile.
You know, we were very successful getting all our work done there, but then realized, okay, that’s only part of the story. The cards getting into the backlog through iterations is great, but we aren’t getting flow of work from the business into the teams, and we certainly aren’t releasing into production frequently. So about, I’d say, five years ago, you know, somewhat in affiliation, some of the work I was doing with, as you know, Rosalind Radcliffe, Hayden Lindsey, you know, started to talk about what’s this DevOps thing. Was lucky enough to end up at the first Gene (Kim) conference speaking with Hayden. Wasn’t sure what I was getting myself into, but it’s been a wonderful experience. You know, got to meet you, Jayne, and lots of great folks from Gene’s community.
In Nationwide, we’re just continuing to work, how do we make that end-to-end flow and lead time, you know, what do we do to make it better? What DevOps practices can we bring? But as you noted at the beginning, how do you get the business more online? How do you get work to flow? And how do you address culture, which we know is probably more important than technology?
Shimel: Excellent, dead on. Jayne, how about you? You have more hair than both Carmen and I. You’re the lovely face we should see on the screen here. Give us a little background and accomplishments.
Groll: So I call myself an accidental technologist. So I started my career, got out of school, ended up in the legal field for about 15 years working on the business side. So I wasn’t a lawyer, but I was one of the early paralegals for a big Wall Street law firm that adopted UNIX a long, long time ago. I don’t even want to say when, but screens were black and green back then. And so everybody had to learn UNIX. And so I was very fortunate to be given an opportunity to accidentally become a technologist, and I know even with my technical skills, I’m a little self-taught. Was able to move up into some senior management roles across several different verticals, leaving the legal space.
And then in 2004, kind of fell on ITIL and really started to see a paradigm shift in IT that technology is important, but your practices, your principles, your processes are important as well. And people, process and technology became a mantra. And so co-founded ITSM Academy with Lisa Schwartz and really saw the growth of maturity in IT. You know, I always say we love our frameworks. IT, we love our frameworks more than any other business unit, and really started to see that percolate. Met Gene and some others through that experience, but really, landing where we are today. In 2012, I was able to attend an early DevOps day, frankly, at Gene’s invitation and really saw something percolating there that was pretty much common sense, but not common practice.
So we started to see, you know, should Dev and Ops collaborate better? Should there be a faster flow? Should there be less silos? Yes. Should we use automation more? Yes. We automate the rest of the world, we don’t automate ourselves as much. And really started to see something grow there, and, Alan, as you know, in 2015, we launched the DevOps Institute primarily because the practices around DevOps unlike other frameworks were really emerging in the wild. You know, you said it before. They were emerging in the wild and making some sense out of that, you know, as it crossed over into the enterprise space was important and requires a lot of research.
So, you know, we don’t have a single body of knowledge in DevOps. Thankfully, we have a collective body of knowledge in DevOps. So for me, I think part of my, you know, pride is my journey has not been a linear path. Really blessed to be, you know, where I am today, and, you know, DevOps Institute right now is, what, 32 countries that are talking about in doing DevOps. I mean, that’s really kind of surprising and remarkable all at the same time.
Shimel: Yep. And just to excuse, I don’t know if you guys can hear. We have a train, the 11-08 Cannonball Express, going via the office here.
DeArdo: Just like my cousin Vinnie here, Alan? Is that like …?
Shimel: That’s “two yutes.” The way I talk, we’re not far off. Anyway, though, guys, let us move along here. You know, one of the things that I’ve learned in my own exploration of DevOps and digital transformation, etc., is that a lot of businesses, large enterprises, they may think they do, but on the whole, they don’t really understand technology. Yeah, some of the guys in IT understand some of it or what have you, but do you think most businesses understand technology? Carmen?
DeArdo: I think it’s … Obviously, people are exposed to technology. In fact, I talk about this all the time, that sometimes we’re still in the book writing business or book production that we still feel like we got to produce manuals of paper material … The disconnect comes as, How we can effectively utilize the technology that we have to meet the business needs? And I think that’s maybe where the disconnect goes. It’s not that, you know, people certainly are … they have Alexas. They have stuff in their home. They may be even, you know, great at programming their home devices, but how are we actually taking advantage of that to serve our customers? And I still think thinking about things kind of in this … you know, Jayne talked about silos, but I actually think the last frontier really of DevOps is in the left part of the value stream, not the right part.
I think they code the cloud. There’s patterns, maybe not everybody’s done it. I know it’s not easy to do. That’s pretty well-defined from a pattern. The front part is not, and I think understanding that, you know, you have to basically be an IT business. We hear this all the time. If you’re not an … consider yourself an IT business, someone who is good at IT is gonna disrupt you. So it’s not that I don’t think they understand technology. They’re certainly capable of it. It’s just, do we really understand how we can use it the best way in order to meet business needs? And, you know, things like the system of engagement. Well, the system of engagement is gonna change. We’ve already seen it change. It changed from web to mobile. It’s going to voice. It’s gonna go to different engagement models.
That’s fine, but you still have to have the back-end technology, and the microservices, and the APIs, and the ability to deliver more quickly, and I think that’s the part of it, the gap we still haven’t bridged. And how can we actually get …? You know, one of the things that Nicole and I are talking about at the conference is the project-to-product model. And I think large companies like ourselves are kind of flailing at what … you know, that makes sense. It makes sense to align around a value stream, but how do you really define what those value streams represent so that you can actually get that flow and then use technology to help get that quick deployment and, you know, feedback? So I think that’s where we see a lot of the gaps, you know, from that perspective.
Shimel: Sure. Jayne, before I have you answer on that, you know, I got an email of pitch. I get a lot of pitches as you can imagine, but I got one the other day that, to me, really nailed the issue. The issue was they pitched me about how great their solutions and their services are, and they said it makes IT costs less. Instead of … so they’re still thinking of the IT department as a cost center instead of a profit generator. And to me, when you say that, that means you really don’t understand technology, you don’t understand the role of IT in the modern enterprise, and you’re just not doing it right. Jayne, your thoughts.
Groll: So I think you’re right. I think, you know, the modern enterprise has a few challenges ahead of it. Carmen, I absolutely agree with you. I think every business knows they need technology and they need to accelerate their use of technology. I think they don’t understand how to apply that, and more importantly, I think the big paradigm shift, and, Carmen, you said it, is that most companies don’t realize that they are now technology companies. It isn’t even just IT does for you, is the core company as a technology company. And those that kind of recognize that, I think, are those that move ahead. Those that are still kind of hanging on to a legacy … Now, you know, I’ll add kind of something into that recipe that’s a little scary.
With the way technology is moving and how fast it’s going, I mean, the acceleration is remarkable, and, you know, Carmen, you said it. It was cloud. It was mobile. Now it’s voice. That it feels like, you know, this kind of moving target that if I’m a business leader, not a technology leader, although Lord knows that, I think, has a plan here, you know, where do you put the dart? You know, do you invest in mobile? Do you invest in voice? Do you invest …? So it’s such a moving target, and it’s going so fast that I think it can be very daunting to figure out, okay, where do we make our play, and, you know, where do we use our technology spend? And I would say something funny, Alan, because you’re right.
You know, in my years in senior IT management, I almost always reported to the CFO, because I was a call center. And so it was nobody understood quite what I was spending money on, but they figured if I reported to the finance guy, that somewhere along the way the cost would be controlled as opposed to let’s optimize the capability. Let’s look at this in investment. So I really think organizations are trying to figure out, first of all, are we a technology company? And I would say unequivocally the answer is yes regardless of what vertical you’re in. And then where do I invest? What technology do I start looking at? And if what I do today is gonna be obsolete tomorrow, how do I make that investment for my company further?
Shimel: Excellent. So, guys, you both have a little bit of a background, more than a little bit, in training and so forth. Do we train people outside of the IT organizations on what technology can do for you? Do we put up sort of Uncle Sam’s “We want you. What can I do for you?” posters? How do we get the business to understand the technology aspects, the technology side of things? Go ahead.
Groll: I mean, you know, the “we want you …” unless you can promote something that says what’s in it for me, me as an organization, me as an individual, me as a leader, I think that’s difficult. You know, doing road shows on here’s what technology can do for you isn’t something that you can tell. I think it’s something that you have to prove. And I think IT sometimes runs into an uphill battle trying to prove that, because it is costly and it does change a lot of the way people perform on a daily basis. I know what I did today, and tomorrow you want me to do something differently, and I may not be as successful as I was yesterday.
So there’s a little bit of a proof of concept usually, you know, done through piloting, whatever, but I think the, “How do we tell them?” I don’t think we tell them. I think we have to show them.
Shimel: Fair enough. Carmen?
DeArdo: Yeah. I mean, I think we’ve gotten a lot of appreciation. I mean, I’ve told … you know, and Cindy Payne and Jim Grafmeyer told the story at the last conference about Hurricane Harvey and how the business was able to respond within a day to what was a wind event and came a rain event with all our flooded cars and how we got the website optimized to handle those, to take the load off of our customer service center. They’re starting to get that, “Gee, having this capability to deliver things more frequently …” We now have some of us who deliver daily whereas before everything was going into a monthly release. So that’s a great accomplishment.
On the other hand, we’re running into the things that, you know, Topos talked about with compliance, with legal, with … and just, you know, how do we … Most of our time now on lead time is going from the idea to the backlog of the team, not from going from the … you know, the team getting the work to deployment. So I think going into, you know, that view of how do we start to think more about, you know, the other things that are getting in our way of lead time, because it’s not just the IT part of it. It’s now the business part of it, and we have a couple areas now who actually have true product owners sitting with their lines and are almost at the point now where they can deliver, you know, stories and storyboard them and move them all the way through production, you know, much more quickly.
So I think that’s an example, and I think that’s what you need. I always talk about you need stories, you need examples of how this can actually be done at your company, because reading about it, hearing about it is great, but then when you see the team next to you doing it, and even if it’s not perfect and even if you know it can be improved, that’s where really you start to change the culture of not just IT but of the business. You want something like that. And then, you know, I think you got something going that you can really build on top of. Jayne.
Groll: Yeah. I mean, I think that conferences like DevOps Enterprise London, Las Vegas, other events, local events, I think, also let individuals and organizations know they’re not alone in this journey. You know, part of the challenge in DevOps right now, I think, is we’re going from what is DevOps to how do we do this. And it’s not finite. It’s not, you know, you open a recipe book and it says, “Start here. Add these ingredients. You know, stir, bake for 350 degrees, and you have DevOps.” It requires a lot of higher-level thinking, requires some experimentation. And so I think that, you know, access to what have other people or organizations done … Carmen, I think you’re absolutely right, what have other teams done, really kind of making it a community of enablement is very, very important in this space, because everybody’s trying to figure out where do they start. And unfortunately or fortunately, the answer is it depends. It depends where you are, it depends what you’re trying to achieve. So I think that’s part of it as well.
Shimel: Yep. So, guys, let me try to tie a bow on this section before we move on. The reason I asked about the technology and technology education issue is I think it’s imperative for organizations to understand that in order for us to get what I call business and tech alignment, and what I mean by business and tech alignment is that we’re all pulling in the same directions for the same reasons with the same goals, and we’re following hopefully a road that we have agreed upon, and we all know the road we’re traveling, because, again, you get back to DevOps, guys. You know, when you guys are out there talking about it and we were covering it at devopsy.kinsta.cloud, what do we always hear? What exactly is DevOps?
There’s no real definition. What does it mean? You know, how do we define DevOps and so forth? And I think one of the things we’ve learned over the course of the years is that, you know, in DevOps, the focus is, you know, heavily cultural, but it has to be squarely on nurturing, buying across the organization for this “DevOps initiatives,” you know, that we’re starting. But at the same time, we need to be orchestrating our strategic alignment initiatives between the disparate functions and organizations within the business. And that to me is business and tech alignment. If you guys don’t agree with that, speak up, but if you do, let’s move on to our next part, but can we wrap up, Carmen, Jayne? Are you fair to say that what we’re really talking about here is business and tech alignment?
Groll: So I’m gonna jump in just for a second, because … So you know I come from the ITIL space. And so in the ITIL space, they were coining the term BITA, business IT alignment. And so for years and years and years, there was a lot of conversation about BITA and its semantics. But alignment means that we’re kind of standing side by side in lanes next to each other. The reality is it isn’t alignment. It’s really unification. It’s really we don’t need to align with the business, we need to become the business. And, again, I know it’s semantics, but that word alignment gets tossed around a lot, and it implies something like, “Well, I’ll drive my car next to yours, and we’ll travel at the same pace, and we’ll get to the same journey,” as opposed to, “Let’s be in the same car.” And so you understand what I’m saying?
Shimel: Yeah.
Groll: And I said the term BITA is sort of falling away a little bit, but it was thrown around a lot during the years when ITIL in particular, you know, was kind of front and center.
Shimel: Yap. Carmen, any thoughts on that before we move on?
DeArdo: No, that was great points. And, you know, I refer back to Mark Schwartz’ last book, “The Seat at the Table,” and where he really talks about IT really should move beyond trying to get a seat at the table and move towards, you know, as Jayne said, you’re actually moving together, and you have to merge really these concepts of what the business wants, you know, has to be very much aligned, more than aligned. It’s part of what IT need to be doing. So that merging has to take a fact, and I think as much as we’ve seen the culture issues in Dev and Ops, that’s gonna be an interesting culture issue. But we’re seeing the companies who are successful are already doing it. They are working together. And, you know, if you walked up to some of the teams I just talked about, you wouldn’t be able to say, “Oh, well, such-and-such is a business person, and, you know, Jayne’s a business person, and John’s an IT person.” No, they’re a team working together to achieve something for the good of the company, and that’s the way it has to be.
Shimel: Yep. Guys, let me broach another subject then, because I think we’ve covered this one. You know, I read an article yesterday about AI, and automation, machine learning, and we’ve all seen these before, and, you know, how it’s changing. But automation really is changing the job market, especially for people younger than the three of us. I mean, we’ve both seen our … all three of us have seen our 21st birthday recently and …
Groll: Just recently.
Shimel: Recently, but really the folks coming up behind us, and we’re talking about our children and so forth, automation and the job market that they’ll grow into is paramount. And one of the reasons is that we have a tremendous skill gap. You know, my background is obviously in infosec, and now we call it cybersecurity. I hate the term. But there’s a tremendous gap in the security space. There’s also a tremendous gap in the DevOps space, obviously, and that’s one of the reasons we see such a high demand, even for people who may not be certified or classically trained in DevOps, I mean, take some sort of DevOps class. If they’ve learned to work Puppet or Chef or Jenkins, you know, they’re in high demand here.
And so with this skills gap, it doesn’t hinder our DevOps transformation. Is it a drag on the entire DevOps industry or community? Jayne, I’m gonna ask you to take first crack at that. I mean, you must see this with DOI stuff.
Groll: I do, and, you know, it’s kind of like good news, bad news. So the great news is, where else do you get to work in an industry or an environment where you are now encouraged to learn something other than what you’re essentially good at? I mean, you’re now encouraged to grow new skills to evolve. Organizations are funding skills modernization programs. So, you know, we groomed … and Carmen referenced that. You know, we groomed an IT based on silos. And in every silo, you were a developer, or you were a QA person, or you were a security person, and you lived in your silo with other people who had the same skills as you did. And now, cross-functional skills and, you know, what they call T-shaped people where you need to still have a specialty but you’re absolutely required to have very large breadth of different skills means that organizations are supporting and funding that.
So the continuous learner is more the norm today than it was before where you will be a developer … experience, and I think it’s a great day to be in technology truthfully, because you are encouraged to think, and learn, and practice, and train, and do, you know, all sorts of other things. And I think companies are starting to see that. I hear the term “skilling” a lot. We not only talk about training, we talk about skilling. We need to skill our force. We need to skill our teams. We need to reskill or upskill. You know, those are really terms that are starting to enter our common vocabulary. And the day of the generic, “I’m a master, I’m an expert, I’m a practitioner,” and I’ve got a few of those credentials as well, those are so ambiguous that now the skills are very, very tangible, and organizations are hiring, I think, hiring and funding individuals to develop not only technical skills but practice and principal skills, too. So we’re all gonna be upskilled, maybe outskilled.
Shimel: Perhaps both. Carmen, how about you on this one?
DeArdo: Yeah. I mean, automation is obviously nothing new. I mean, I’m really dating myself, but I can remember when people didn’t trust the output of compilers. Like, are we sure the compilers are producing the right machine language or assembly language? I don’t know. I can bring the assembly language better than this compiler. I mean, I think automation and learning is just … obviously, you know, if you spent, you know, four decades like I have, and, you know, none of us are spring chickens, but you’ve got to learn a lot and keep learning. Now, I think the great thing, and I agree with you and Jayne, the great thing about Nationwide is they have invested in learning and upskilling their people. We had this thing called Teaching Thursdays where twice a month we have two hours which is set aside for peer-to-peer learning, and it’s really not … and it’s peers teaching people, and people love to be the teacher.
I mean, at the last forum, Jason Cox was talking about … We had a subgroup focused on operations, and Jayne was there. And talk about … you know, he had some folks on his team, that they have to become engineers. They have to become coders. And one of the guys who initially maybe wasn’t so crazy about it, once they became kind of the expert in their area and they were teaching other people, well, you know, they were really engaged then, because, you know, they were involved and they were the one now from a peer-to-peer learning perspective.
So companies, I think, like Nationwide, Disney, other companies that are leaders in this space need obviously to invest in that. And then, you know, it also means that the associate themselves, you know, needs to take advantage of that. I mean, we can offer those capabilities. We know the jobs are gonna change. We can make those skills available, and then folks need to take advantage of them, but yes, I mean, people have to become more broadly skilled and be able to become more of what I would call problem-solving engineering mindset as we move forward.
Shimel: Sure. So, guys, though, you know, so we have skills over here. And on this side, there’s another thing that creates a tension with the skills, and that’s what I call the information gap. So we have a skills gap, and you guys have both addressed that really well, but we also have an information gap. And I alluded to it when I talked about training. If an employer or an organization thinks that someone who can write a recipe in Chef or has heard of Jenkins is a DevOps engineer, if there’s such a thing as a DevOps engineer, or is someone who, you know, is gonna really help build their DevOps team, to me, that’s not a skills gap, that’s an information gap. How do we understand … How do we make organizations … How do we fill that gap on information so that organizations recognize, A, that there may, in fact, be a skills gap, B, how to fill that skills gap and bring the organization up to it?
I mean, this tension, this chicken-and-egg, if you don’t know it, you don’t know. How are you supposed to know? And so I’ll put it to you. What do you think is more important? The information gap, the skills gap? Are they equally important? Do we tackle them concurrently, individually? Carmen, how do you guys do it at Nationwide?
DeArdo: I mean, I think one of the things that IT hasn’t been good at is treating IT as … we’ve been a very bad customer to ourselves. So if you look at our pipeline, we don’t treat it as a product. If you actually want to get faster at delivering a capability from idea through deployment, you need to consider that as a product, not a bunch of different tools. It’s not Jenkins, and New Relic, and UrbanCode or, you know, the AWS. It’s not that. It’s a pipeline that’s gonna enable you, and it doesn’t start at code, it starts at idea to get you all the way through that deployment. So then what … do you have a team? Are you doing thinking across the pipeline? Are you doing architecture? We don’t do things like value stream architecture or architecting the pipeline as a product. We want to buy a bunch of technologies. We can hire people good in those technologies, but as you said, that’s not really gonna get us where we need to go.
So I think the first thing is, do you really treat your product … One of the first things, do you treat your pipeline as a product? Do you have a team that can actually support that product all the way? Do you have a product owner who can drive continuous improvement across that with funding and prioritization? And do you have one of your best architects, if not the best, to actually create the architecture and do value stream architecture, which is something, you know, that Mik Kersten and I have been talking a lot about. I think that’s one side of it that we don’t get, is teams could … if you ask a team, they can kind of figure out, tell you like a Deming quality circle, how they can go faster. And you can experiment, but then if you’re gonna operationalize that, you don’t want 200 teams doing things in 200 different ways. You can then take that and say, “How do I address that as part of my delivery capability?”
And then I think what you need your leadership to do is apply systems thinking, and that’s a whole another subject we could get into, but I don’t think executives and leaders know how to really address the, “How do I optimize this as a whole?” They want to get into the weeds. They want to get into the tool selections. There’s people that can figure that out. You need to really focus on the strategy and how you can optimize all this great information you’re getting to make sure you’re removing barriers and allowing teams, you know, doing the learning, making sure they can be upskilled, investing in their ability, and allowing them, you know, to be able to innovate and improve your delivery capability. And I think that’s what I just said quickly. It’s easier said than done. And I think that’s where I see a lot of the struggles, and I think where we’re trying to address this is how do we start to put some of those pieces in place so it’s not just a bunch of, you know, divergent disparate technologies we’re trying to bring in and hope that that’s gonna make us get better. Because those things and having skills just around those things is not gonna actually get us to where we need to go.
Shimel: Fair enough. Jayne, I’m gonna give you a chance to comment on the DevOps skills versus information gap.
Groll: So I think the two are really related, and I think Carmen brings up some good points. You know, I think it all comes down to silos. So let’s look at our silos. Well, within our silos, we speak the same language. We use the same acronyms. We use the same tools. We live in a common community where everybody kind of understands what’s going on and what’s happening. And then when we leave our silo and we try to talk to a different silo, we don’t speak the same language. We don’t use the same tools. We don’t have the same processes. So I think the skills gap doesn’t fully overcome the issues with the information gap, but, certainly, there’s help there that if we’re looking to share information, we have to share it in a way that everybody kind of has some type of uncommon understanding.
The other thing with information is, you know, information is really interesting to me, because everyone thinks they communicate a lot. People generally think that they communicate well. You know, teams generally think they communicate … like I told you that. You told it to me maybe in language I don’t understand or it wasn’t clear. It was some report that, you know, I don’t even know where it is. So I think, you know, kind of the flow of information and from a vocabulary perspective, from an understanding perspective, from a “Did you understand what I shared?” is something we all need to overcome. And I think, you know, again, they’re so interrelated to each other that if we start to create a broader set of skills, hopefully, that helps to improve the communication. I think the other side of it is we talk a lot about collaboration, and there’s a really fundamental difference between communication and collaboration.
And so information flows through communication, but communication is pretty much one way. I tell you, you tell me, and somewhere along the way. Collaboration is I start to ask your opinion, and that’s a really big, big difference. And so when we’re starting to share information, there’s a respect that comes across the skills gap. There’s a lot of things that go across that that, again, there’s separate gaps, but hopefully, the long term, and as Carmen said, not easy to fix, is by overcoming some of the skills gaps as well.
Shimel: Fair enough. I think we’ve covered that one well. Let’s move on to another area if we can. We have 20 minutes or so. You know, one of the things that really does my heart good is this concept of DevOps everywhere as far as the eye can see, like the meme with Buzz and Woody. We have seen DevOps move from just Dev and Ops to QA, DevSecOps exploding on the scene where, you know, a couple days, the 28th, a day or two after DOES London, John Willis and team are rolling out our second DevSecOps Days, which I’m involved in, and devsecopsdays.com for those interested. But even beyond IT, we’re seeing what I call DevBizOps, where DevOps’ Agile sort of practices are starting to take hold in HR, in sales, in finance, in, you know, accounting, etc.
And let’s be fair. This is not unique. The fact of the matter is a lot of the lean terminology, a lot of the lean stuff that was built into DevOps didn’t come from IT, it came from manufacturing. And so, you know, it’s like pay it forward. Now we’re seeing these kinds of DevOps principles move into other areas of the business. First of all, is that a good thing? I think it is. I can’t imagine it’s not, but I’ll defer to the both of you. And then, secondly, you know, hey, at the end of the day, remember, it’s all about the culture, stupid. And so what human factors, cultural factors should we be thinking of when we talk about a broader DevBizOps opportunity? And, Jayne, you’re on my screen, so I’m gonna tell you to go first on this one if you can.
Groll: You know, it’s business transformation. You know, it’s not really DevOps transformation. Digital transformation is not limited to IT. It’s really business transformation. And so it goes back to, you know, we talked about a little while ago, where, you know, every organization has to recognize that technology is their business, and so some of the practices from the business start to assimilate into IT and beyond. I saw this in service management where there’s now movement towards enterprise service management, where you’re gonna make changes, you do it the same way.
I think when we start to look … I mean, I can tell you from the content perspective, you know, we use tools in content development, even though it’s not software development, that are very, very similar to that. So I think it’s fantastic. I was talking to a gentleman from an organization on the Netherlands recently, and they actually took the Spotify squad model and applied it to their entire business, and not a small company, by the way. And so they applied it to their entire business where their business is now broken into squads. And, you know, some think that that’s only for a tech company, and by the way, this is not a tech company. It’s a large financial organization.
So I think that’s great. I do think there’s some … I think there’s good news, bad news. I think there’s a little bit of risk that, you know, with DevOps and some of DevOps practice is still emerging that it becomes, “This week, we do it this way, and this week, we do it another way.” And, you know, “Oh, that didn’t work because …” You know, fill in the blanks, but I do think that kind of having a one business and getting away from this we love our frameworks and IT, and every year, we have a new framework that supposedly supersedes the other framework, you know, that’s a maturity issue. And I think if the business embraces it … you’re right, Alan. Lean Six Sigma. That was a business initiative that a lot of organizations underwent that weren’t necessarily manufacturing organizations. So, you know, as we get closer to that, I think the unification becomes stronger.
Shimel: Great. Carmen, how about you?
DeArdo: Yeah. So I think of things like … It is a great thing. So I think of things like lean concepts. We have leaders across the company that we have gone through lean education and lean concepts. So when you talk about value streams, when you talk about lead time, when you talk about wait states, people understand those concepts, whether they be on business or IT. I mean, this whole idea that Dominica, I’m sure, would have talked about, about making work visible, Kanbans. I mean, we actually had an explosion of people who wanted to use some of our tools that initially were brought in for work out of management because they wanted … you know, it’s like, “Gee, who knew, you know, this Kanban stuff would work for us and WIP.” WIP limits. And so people certainly want to apply those kind of concepts.
So I do agree that, you know, the basic concepts, you know, really can spread everywhere, and the whole concepts of even the things, like we talked about it, the three ways. The three ways don’t just apply to IT. The three ways apply to anything that you’re trying to do and get better at. I think when it comes to information sometimes, though, you know, it’s kind of like the robot story and the goal. We have lots of KPIs, but are we actually focused on the goal? So the goal is not around keeping this person utilized or that team utilized. The goal is about, can I get faster from one end of my value stream to the other? And do we have data that shows flow? And that’s why I’m excited with some of the things that I’ve been … you know, that Mik’s talking about. You know, he has a book coming out that is gonna show actual flow of work, because up until now, we have not been able to actually see work flow all the way through that delivery pipeline. And so we guess a lot of times. We think, “Oh, well, if we just add more developers. Oh, we’ll add more testers.”
Well, how do we know? We don’t really know, because we’re making maybe educated guesses, but we’re really not all focused on understanding where the work is and those concepts that can shine a light on it. So I think it all does come back to a lot of those concepts that, growing up in Bell Labs, I was lucky enough to experience with Deming, even going back to Shewhart, around doing those experiments, getting better, seeing the entire end-to-end value stream, but these concepts apply. I mean, “The Phoenix Project” is really not a book about IT even though I teach part of it. It’s a book about how do you get better at doing whatever business you’re in and trying to, you know, stay in business and improve what you’re doing for your customer. So I think those concepts will continue to go, and I think, again, as we’ve talked about today, there’s a thread here of everybody needs to be on board with these concepts: legal, compliance, the business, everybody, because they all play a role in allowing you to do a better job, delivering more quickly to your customers.
Shimel: Fair enough. Guys, let me … You know, Carmen, you hit on some great things there. The Kanban board or even here at devopsy.kinsta.cloud, frankly, we run our entire customer engagement, not practice, but now into farming, on Trello cards, which is sort of an electronic Kanban. And it’s a great project manager. I’ll give you another thing. You know, there’s the cultural aspect of this of taking that sort of DevOps culture and expanding it throughout the organization, but there’s also the bits and bytes and tools aspect of it. Kanban’s a great example. I’ll throw another one out at you. Get up. Why does it have to be source code for programs? Why can’t it be source code for contracts or any kind of digital document or, you know, things that we work on?
So I think we’re just starting to scratch the surface on how we can take that DevOps can do or Kanban and, you know, move it out across the entire organization. And we’ll all … I think we’ll have better organizations as a result, but let me now turn it back around. How can we take some of the business culture positives, some of the things that have worked in other parts of our business, and reflect it back into our DevOps and IT processes? How can we enhance our DevOps by adopting broader business principles and practices that seem to work? Carmen, I’ll let you take first crack at that.
DeArdo: So I think, actually, a lot of our test and learn activities started with the business who had actually hypotheses around, “Well, if we offered a policy in a different way or if we made a tweak to something, would we be able to sell more of those things?” So we had people who … and there’s a lot of statistical rigor that you have to bring to that. So we actually had people on the business side who had these ideas about, “I have a hypothesis. You know, is there a way you can allow me to test that hypothesis to see if it, in fact, will yield some positive result in terms of actual policies being sold?” So it was a very scientific-type experiment that they actually wanted to run, and at the time, we really didn’t have a way to support that from an IT perspective. And it kind of forced us to do some things to support, you know, how could we actually get this out and offer it to only a subset of people, which we now might call canary launch or some other terms that are DevOp-ish terms, but really nobody was reading a DevOps book at this point. They were just trying to figure out, “How can we support the business’s need here to try to adapt more quickly and have a high plan around?”
And that started, I think, some of the things that actually were successful that then led into once we started to get into DevOps, like, “Okay, what we actually did there was DevOps.” We didn’t call it DevOps, but those are principles that really we need to support not just in this one’s case, but for all kinds of business hypotheses that we want to get out and turn around real quickly. And doing that really does bring everybody together, because it’s exciting. It’s exciting if you can sit at a business and you can think up, “I have ideas for this product or that capability. You know, how can we more quickly try to see if that would be a good idea?” rather than writing a 12-month business case and trying to do, you know, a protracted study.
So the whole idea of experiments and gain things out quickly, I mean, everybody will resonate and get behind that if you talk about it in that way. If you start talking about Puppet and Chef and things like that, that’s not gonna cut it. But if you talk about it in terms of business and what it can mean for the business, then that resonates and gets a lot of people excited.
Shimel: Excellent. Jayne, how about you?
Groll: I mean, I think Carmen is right on point. And, Carmen, you said something I think that was … you’ve said a lot of very insightful things, but I think in particular that the experiment came down to, “How can we sell more policies? How can we make our business more successful?” And I think if you look at kind of the past in IT, sometimes, we didn’t know how we were contributing to the bottom line. So we were introducing a lot of technology, and we were doing all this cool stuff. But how did that convert into an increase in revenue, an increase in market share, increase in competitive advantage, things like that? And so I think that this unification with the business becomes more significant when IT sees themselves as an enabler of the goal. The goal of every business is to make money. And so sees themselves, and it goes back to information and communication as well, that they understand that what they’re doing is actually helping to sell more policies or increase market share or stand above the herd or anything along those lines.
Now, again, if you’re the CIO, you probably know that. If you’re in senior management, you probably know that. If you’re a developer and you’re one of a few hundred developers, you may not know that. And so I think that it is that kind of bottom-up top-down approach where understanding that what you do every day is contributing in a very, very substantial and tangible way. And then I think the other part of it, and, Carmen, I think you said it really well, is encouraging those experiments where the business and IT work together. And maybe it is that 1 in 100 coder that’s working as part of that team to say, “You know what? I have an idea.” Everything starts with a great idea. It could be from the business. Hopefully, from the business, that says, “I have an idea. Let’s see if we can, you know, bring that to life and what it will do to our bottom line.” And then there’s really a great collaboration between IT and the business.
Shimel: Agreed. Agreed. So, guys, we’re coming up to the top of the hour here, and I guess we need to wrap up. Otherwise, we’ll just be on air all day, and I know Jayne has a show in Vegas and so forth, and Carmen has stuff to do. But first of all, Carmen DeArdo, Jayne Groll, thanks for being our guests here on this hangout as we, you know, continue to build up to DevOps Enterprise London, DevOps Enterprise Summit London. Just a reminder for anyone watching, it is June 25th and 26th in London. There are still some tickets left, I think. I don’t know how many. I wouldn’t wait. You can get updates about the event as well as get tickets at events.itrevolution.com. It’s obviously on the web.
By the way, this year, as in past DevOps summits, devopsy.kinsta.cloud is a media sponsor. We’re partners. We’ve partnered with Gene from the beginning. We will be doing videos again this year at DevOps Enterprise Summit. I think it’s gonna be a little different this year. We’re doing some interesting roundtables. You’ll see them on our YouTube channel, DevOps TV. So stay tuned for that. All right. Guys and lady, or guy and lady, I had a great time today. I’ve learned a whole bunch. I hope our folks listening and watching this have learned as well. You know, you guys have done, not just today, but, Carmen and Jayne, you’ve been instrumental in helping to bring the gospel of DevOps to the world, if you may, or at least maybe not the gospel, but at least some really good learning and understanding.
So thanks again, guys, for being with us. We’re gonna call it a wrap. Carmen, any last word? Three words.
DeArdo: Three words. Project to product.
Shimel: Okay. Jayne? Last words from you.
Groll: Pittsburgh Steelers. I don’t know.
Shimel: We finally got her. Carmen, we converted her.
DeArdo: I should have said, “Let’s go Tens.” That’s three words.
Shimel: Okay. Hey, everyone. I will see you both in London June 25th and 26th. Hope to see many of you listening or watching this in London as well. If you’re there, stop by, say hello. I always like to meet people who have watched us online. But until then, this is Alan Shimel for devopsy.kinsta.cloud and IT Revolution. Thanks for watching, everyone, and hope to see you soon. Bye-bye.
The downfall of the all-in-one IT management suite and the rise of the customized IT Ops toolchain
Here is how enterprise IT Ops used to work: you would make your choice among the Big Four vendors and buy in (quite literally!) to their strategy. This meant a suite of products that came preintegrated (more or less) out of the box, and—at least on the sales proposal—covered all of an IT organization’s needs across a huge spectrum of activities.
Photo by Barn Images on Unsplash
Broadly speaking, each of the Big Four’s suites included some flavor of the following: monitoring, ticketing, configuration management and asset tracking and automation and orchestration. Some vendors were stronger in one area than in another, but unless you had pretty specific needs, each vendor could be your one-stop shop.
Development and improvement inside these suites was mainly incremental and occurred at a fairly leisurely pace. Meanwhile, startups with bright ideas would come along occasionally and challenge the incumbents in specific areas, only to be bought by one of the mega-vendors if they showed enough promise.
There was a symbiotic relationship between vendor and IT buyer, as IT professionals built decades-long careers as experts in this or that vendor’s technology stack, with the wall of framed certifications to prove it. CIOs were reassured by the fact that there was a readily available pool of experts to hire, and continued reinvesting in the next iteration of their chosen vendor’s stack.
The Old Model No Longer Fits Modern IT
In the last decade or so, this comfortable status quo has broken down. The cracks were already showing before then, but only the most agile and forward-thinking organizations were moving fast enough to trip over them. As more and more businesses began to expect greater and greater responsiveness and speed from their IT departments, the huge sprawling suites of the mega-vendors were no longer able to keep up with the required rate of change. Leisurely 18-month release cycles no longer cut it in an agile world, where that time span may encompass several entire strategies.
In the old world, to call something “best of breed” was a sniffy dismissal of a one-trick pony, as opposed to the more supposedly versatile offerings of the Big Four. In the new world, it became obvious that aside from a handful of tentpole products that were genuinely good and fit for purpose, most of the individual components of the big suites was distinctly unexciting. Worse, the main value proposition of the suites—that they came preintegrated out of the box—started to break down as well. Many IT professionals were still fiercely loyal to individual products, but they became more willing to pick and choose, rather than buying into the whole vision.
In parallel, the rise of simple integrations—primarily REST APIs—made it easier than ever before to connect disparate solutions together into a single IT Ops toolchain. Worse (from the point of view of the incumbent vendors), sometimes it was easier to connect modern point solutions than their own component pieces, weighted down by the baggage of decades on the market.
How to Deliver the Flexibility and Agility the Business Expects?
These days, it is becoming commonly accepted even among the most staid and risk-averse organizations that the way to go is no longer to align to a single IT vendor and trust them to deliver on their vision, but rather to design custom-fit, multisolution toolchains, including home-grown, open source and commercial tools.
Photo by Max Bender on Unsplash
But what to do if you find yourself dealing with the consequences of this sort of IT monoculture?
Look Back At How You Got Here
The first step is to re-evaluate past buying decisions and strategic alignment. It is very important to do this without assigning blame; it was a different time, and those choices may well have been the right ones for the prevailing environment at the time, when integration between tools from different vendors was legitimately difficult.
Don’t Just Throw Everything Out
In many environments, too, the correct solution may well not be a clean sweep. One or more of those tentpole products that kept the overall suite looking good past its sell-by date may well still be perfectly fit for purpose and not worth the hassle of swapping out. However, other components are probably legitimately lagging behind the state of the art, in which case it is worth replacing them with something more fit for purpose.
Count Costs – All Costs
Include cost in these calculations, but remember to include all costs. Maybe you or a predecessor signed a big enterprise license agreement (ELA), so Product X is “free” in that context. But what does it cost to keep Product X running, in terms of ancillary licenses, hardware requirements and, last but not least, skill and effort on the part of the IT administrators? I don’t want to name specific products or vendors as examples, to avoid the discussion devolving into particulars, but it’s worth bearing in mind that many legacy products have shallow talent pools—not that many people are available on the market, and many of those people are coming to the ends of their careers, which makes them both expensive and reluctant to change employers. Instead, up-and-coming technologies often have much deeper and wider talent pools, with many more experienced specialists available at all salary and skill levels. If you don’t believe me, try hiring half a dozen COBOL programmers and half a dozen Node.js coders, and then we’ll talk …
Photo by Giu Vicente on Unsplash
Your IT Ops Toolchain, For Your IT Ops Needs
Moving from one of the big all-in-one suites to an assemblage of different tools, all flying in loose formation, is not easily done. Any technology transition of this sort is a Project with a capital P, and needs to be planned and managed with due care and attention. However, there is a large payoff at the end in terms of increased agility and flexibility within IT Ops. In turn, this affords IT the ability to offer their business counterparts the sort of responsive and even proactive IT support that moves IT from being a cost center, on a par with Facilities, to a strategic differentiator that enables new business initiatives.
Business as usual doesn’t cut it any more, and assumptions that used to be valid have fallen apart. Decisions based on those assumptions need to be re-evaluated. New, more flexible approaches will set up IT Ops for success in the newly flexible world we are already living in.
Most organizations are not fully compliant with the European Union’s (EU) General Data Protection Regulation (GDPR) before it takes effect in May, analysts report.
While DevOps certainly gives organizations the mean to turbo charge the compliance process, stakeholders outside of IT departments often are failing to greenlight this necessary project ahead of the deadline, due largely to uncertainties about the regulation.
Not being in compliance could hit organizations hard, with fines up to 4 percent of their annual revenues or €20 million.
The Cloud Security Alliance (CSA) and Netskope, for example, recently released the results of a survey of more than 1,000 respondents about their organizations’ GPDR readiness. The survey revealed a full 83 percent were not very prepared for GDPR. While 85 percent had taken steps for compliancy, only 31 percent had well-defined plans for meeting the provisions of GDPR.
The GDPR’s “right to erasure,” (53 percent) “data protection by design and by default,” (42 percent), and “records of processing activities” (39 percent) were deemed the biggest stumbling blocks for compliancy, according to the study.
A separate survey of 700 network security and IT professionals by Enterprise Strategy Group (ESG) showed only 11 percent of firms were completely ready for GDPR ahead of the May 25 deadline. Thirty-three percent said they were prepared to publicly disclose data breaches within the GDPR-mandated 72-hour period, and 33 percent and 44 percent percent of the firms surveyed were mostly or somewhat prepared, respectively.
Most organizations ESG surveyed also reported they are investing in GDPR, reflecting the seriousness in which they are regarding the mandate. While only 10 percent said they had made or were planning to make major investments in GDPR, 63 percent said they would make or had made incremental investments.
Analyst firm Varonis made similar findings after completing a survey of 500 IT security experts in the United States and Europe in October, out of which 200 were in the United States and the rest of those surveyed were equally split between the UK, France and Germany.
Varonis also found 60 percent in the EU and 50 percent in the United States faced serious challenges before they could be GDPR-compliant. However, 70 percent of the firms represented were also convinced GDPR compliancy would give them a competitive edge.
IM000302.JPG
Where Is the Data?
Much of the uncertainty about GDPR reflects how organizations often fail to understand where exactly their data is located. While not specific to GDPR, Varonis found many organizations lacked fundamental data-protection protocols. Those surveyed revealed, for example, 21 percent of their organizations’ electronic folders were “open to everyone.” The survey also revealed 54 percent of the firms’ data was stale. Effectively tracking data extending from on-premises servers to across the cloud was often cited as an issue, making it difficult to track the exact origins of customer data and to determine whether it might be subject to GDPR protection or not.
“We live in a hybrid world in which data isn’t stored entirely on-premises or completely in the cloud. The hybrid model creates additional challenges and potential risk from a security and data governance standpoint,” Ken Spinner, vice president of global field engineering at Varonis, said. “You can’t assume your data is safe in the cloud. If your cloud environment isn’t secure, your data won’t just be in danger of being exposed to your entire organization—it could be accessible to the world.”
Uncertainties Abound
A lack of understanding about what GDPR compliance involves is often cited an impediment to making necessary changes for compliancy. Much has been reported, for example, about whether EU citizens, residents, or simply those who are only temporarily on European soil are protected by GDPR. How the EU will impose fines against non-EU organizations also remains murky. According to ESG data, less than one-third of the respondents of the organizations that took part in its survey fully understand all the GDPR’s requirements.
These questions, of course, largely need to be resolved by legal counsel. Meanwhile, many legal departments are waiting for more guidance from the EU about uncertainties mentioned above, as well as other remaining questions about the GDPR’s scope.
In other words, DevOps is ready to do the necessary, while legal and other stakeholders who fully understand what needs to be done must pull the trigger first. Only then can DevOps, consisting of IT operations, QA, InfoSec and development teams take over. When that happens, DevOps’ role will remain critical for making compliance more agile, automated and, ultimately, reliable.
“DevOps can assist companies in complying with regulations like GDPR by ensuring that key regulatory requirements are part of the initial design criteria, and taken into account when applications are updated,” Varonis’ Spinner said. “GDPR, manual methods of enforcing data policy aren’t realistic solutions because large organizations could have millions of files. DevOps can support GDPR requests with processes and solutions that ensure personal data that falls under the new regulations is protected and used appropriately.”
“As with any new technology, it’s really important that we be thinking now about how to do [it] ethically and responsibly.” —Robert High, Vice President and Chief Technology Officer, IBM Watson
Here’s something to think about: According to an article in The Wall Street Journal, more than 30 countries have defensive weapons run by automation. South Korea has robotic sentries along the DMZ, while Israeli robots patrol the border at Gaza. Russia has robotic vehicles armed with 30mm cannons and anti-tank guided missiles that are well-equipped to battle it out with NATO forces should the need arise.
Got your attention? Wait, there’s more.
The SeaHunter is a U.S. military vessel that can travel the high seas on its own for months without a crew to feed or protect. It’s completely unmanned. And, it costs only $20 million per boat, less than half of the $49 million it costs to fire 59 Tomahawk missiles. At that price, the SeaHunter is practically throwaway.
The SeaHunter, an unmanned oceangoing vessel, can remain at sea for months at a time.
The implications are profound. The value proposition is hard to resist, Why spend billions on building and manning traditional warships when, for the same expense or less, you can have a navy of predominantly robotic ships patrolling the oceans of the world? And, you can have robotic aircraft, capable of automated takeoff and landing, using these ships as mobile bases. An air force untethered to any land mass is a game changer, particularly when non-state actors can get their hands on a few of them. It sort of brings new meaning to the word “pirate”—no need to board a vessel by force, just get the access credentials.
And, at the core of it all this modern weaponry is automation and growing capabilities in AI. That should give you something to think about.
We in DevOps are all about automation. It’s our raison d’être. Our fondness for, and dedication to, automation is admirable. We really have changed the way tech works. Yet, many of us think of our activities as mundane. It’s akin to the nonchalance of a Major League baseball player. They’ve hit the ball so much that many players have forgotten how skillful one needs to be just to foul off a bad pitch. Most people on the planet can’t hit a Big League pitch, let alone make it go foul by intention. But for the pro, it’s no big deal—it’s just part of a day’s work.
It’s the same for us. We’re so accustomed to automating our world that it’s become no big deal. But, it is a big deal. It’s a very big deal.
The easiest thing in the world is to trivialize the incredible impact our work has on others. After all, we’re “just working on” an ecommerce site, a dating app, or IoT endpoint for wired bicycles, right? Those of us working in the defense industry might grasp the significance of the work, but for most of us, it’s business as usual.
Here’s the raw truth: Those of us doing the automation are changing the world. Each line of code we write is intelligence that alters the digital infrastructure of some part of the planet. The code we write will live on well after we’re gone, whether it’s from a company or from the planet. Are we really clear about the impact we have? Yes, some of our tech superstars are asking us to pay attention, but how many of us really are?
Every day that passes we give automation more power. It might be a little piece of power, such as letting only authorized users access to an application. It might be significant power, such as determining whether a person qualifies for a loan to buy a house. It might be enormous power, such as manipulating political forces toward a desired behavior. Or, it might be the ultimate power: Deciding who gets to live and who gets to die. But, regardless of degree, it is power and we have it. And, we continue to make automation more powerful. As surprising as it may sound, despite appearances otherwise, we’re the ones the world trusts intrinsically to do our work safely and responsibly. A lot of the world’s safety depends on how we do our work. Political leaders don’t write the code. We do.
Originally, I was going to call this piece “Death and Destruction in the Age of Automation” and write about how we’ve taken automation way beyond commercial IT into the realm of life-and-death decision-making. I decided otherwise. The world does not need another gloom-and-doom piece, nor does it need another piece of sensational journalism. We’d do better to focus more on the positive things we’ve achieved. Technology is doing a whole lot more good than bad.
But the fact remains that technology has accompanied death and destruction since Roman catapults assaulted the frontiers of ancient Europe. The thing that’s really different now is that the technology thinks—or, at least, it’s doing a really good job of emulating thinking. And, we’re the ones making it think better. Thus, we should ask, Are we making technology think in ways that are safe and trustworthy? Are we acting in ways that are safe and trustworthy?
These are big questions. Do most of us really care about them? Or, is it easier to accept our work as mundane, with little consequence in the Big Picture? The nice thing about mundane is that it doesn’t require a lot of ethical contemplation. But, when you consider that the script that deploys an updated container to a Kubernetes cluster is not that far from the one that deploys an unmanned warship to a regional hotspot, things look different. And, contemplating the implications thereof becomes more than an academic exercise in a computer science ethics class.
More than 2,000 years ago the philosopher Socrates said, “The unexamined life is not worth living.” Given the immense role that technology plays in today’s world and the power it wields—power that continues to move further away from our direct control—we might do well to say, “The unexamined technology is not worth doing.” It’s something to think about, while we still can.
Here I am at the Fontainebleau Miami Beach for Appian World 2018, where the theme is “Transform.” The event, which is the biggest one yet with 1,500 registered attendees, is all about low code, and in his keynote address, CEO Matt Calkins emphasized that low code app-building isn’t for unsophisticated, simple apps—we don’t need powerful tools such as Appian for those. Rather, low code is for sophisticated apps to set free the renaissance that will happen with easier app-building.
This is the first Appian World since the company went public last year, and the event has a public-company feel. I didn’t see a lot of people in jeans and t-shirts—most attendees were fairly well-dressed in business-casual, the mark of enterprises.
One of the touch points in Calkins’ keynote was reliance on open standards. Another was to make use of artificial intelligence (AI). I know AI and machine learning (ML) seem to be everywhere. It was interesting the company announced AI integrations with Amazon, Microsoft and Google, but no Apple or IBM Watson. I will ask Calkins about this when I interview him for DevOps TV later today.
Calkins inherently believes that low code will set off a revolution as software continues to disrupt and empower the digital transformation revolution. The logic behind it is simple: “Making applications easier to make because you don’t have to code as much.” But Calkins emphasizes that does not mean you are making unsophisticated applications. The world doesn’t need low-code tools to make unsophisticated applications. It is the higher calling for sophisticated apps that Appian is all about.
Another mantra from the keynote was, “Host anywhere, run anywhere, data anywhere.” The idea here is a true hybrid reality where data can be on-prem or apps hosted on public cloud, on containers or not. In fact, Calkins was very bullish on containers (who isn’t?). Docker integration is already available and he hinted at announcements regarding Kubernetes as well.
Appian seems to have succeeded by becoming the low-code platform. With lots of integration and focus on making application development, management and deployment easier, the company has risen to the top of its market.
Stay tuned for more updates from Appian World today and tomorrow.
My family has been playing a fair amount of the board game Settlers of Catan lately and, as happens with me, I got to considering the balancing act of the game versus the balancing act of moving to full-on DevOps. Most orgs have some amount of DevOps running at this point, but most have not made the transition, and I think the similarities have something to do with why. So, let’s consider …
In Catan, you put a settlement at the intersection of three hexagonal tiles. Each tile has a number on it, and those numbers run the quasi-bell curve created by rolling 2d6. There never seems to be a great set of numbers, so you go for the best you can arrange (there are more considerations, such as the type of resources you get for a given hex, but we’ll get to that later). Then you hope that your numbers come up enough to keep you in the game. The game is pretty hot/cold, with the dice seeming to pick a number (regardless of bell curve) and roll it more often in a given game.
The resources part is important. In Catan, each hex represents one of five resources, each of which you need to win the game. But you can trade resources (at a loss) for other resources, so you don’t have to sit on a hex for each type. This leads to some interesting attempts to get around the fact that the dice rolled against you or there were no good resource/number combinations when it was your turn to set up.
The parallels are intriguing, though not perfect. You have resources (people/systems/software) and you have goals (project/product/portfolio) that you are trying to reach. Random things (system failures/hackers/budgeting changes) interrupt you more than is convenient (this last one is equivalent to die rolls in the game, but also another game concept called the robber that I only mention for those who play, since explaining the game would make for a long blog). And you have to be a success.
Choosing to move people to DevOps creates a short-term problem with an eye to long-term gains, since those people moving are both out of their comfort zone and using tools that are new to them. Choosing to dedicate systems to DevOps processes/deployments removes them from the pool of resources available to longstanding systems/processes. It creates a tension of temporarily stretching things so that in the long run you will be better off (much like saving to build a city in Catan when the dice really want you to build roads).
The difference is, of course, obvious. Catan is a game, and if you lose horribly you just reset the board and demand a rematch. You can laugh at the pernicious nature of the dice when they turn on you just as you were about to do something cool. But in DevOps transformation, you just can’t do that. People’s jobs are on the line, and even if you are at a place where job security isn’t an issue, people’s stress levels still will change based on such wild events. If you haven’t experienced this, have your company announce budget cuts that keep you from implementing part of a transformation stage or have your systems compromised that require the attention of staff that were in charge of moving an application portfolio. It isn’t pretty.
The key is to keep the goal in everyone’s mind. The end result of a DevOps transformation should be the ability to get changes done faster with better focus on business needs, more complete testing and better communications. Keep talking this up, as there are always roadblocks to large change in any organization. Those improved communications should help people work together to overcome issues and move on. Even if something such as a data breach brings the effort to a halt for a while, when it resumes, an increased focus on the security of data will only make the transformation stronger.
So figure out how to deploy those resources to the best advantage, keep rolling the dice (so to speak) and keep trying to improve on what the last iteration did. The odyssey will continue unless the org gives up on it; your task is to make the most of each step while you continue to keep those core business systems running.
The General Data Protection Regulation (GDPR) will have an effect on how Google Analytics collects and processes data and how DevOps uses the tool—at least, that is what is known for sure at this time. These changes also should almost certainly impact DevOps for any organization that relies on the tool to collect and analyze user data. It is also known DevOps teams will have to reconfigure their Google Analytics implementations, even if an organization has no users based in the European Union (EU).
However, while Google has recently offered some guidance about modifications to Google Analytics that impact DevOps, the company has yet to clarify some key points about changes ahead of when the new mandate takes effect May 25. In the immediate, here are some changes Google has communicated so far and actions DevOps will need to take ahead of the regulation’s deadline.
Immediate Action Required
The GDPR’s “right to be forgotten” provision offers users options to demand the deletion of personal data third parties store and share. To meet the terms of this provision, Google Analytics will allow organizations to manage how long their users’ data remains on Google’s servers. In this way, Google Analytics will automatically delete user and event data older than the date specified. Organizations can specify that the data remains on Google Analytic servers for 14 months, 26 months, 38 months and so on, for a maximum 50 months. It is also possible to opt for data to not expire automatically. Google Analytics says it will not delete information based on aggregate user data.
Google Analytics also will allow organizations to delete data for individual users in an EU country that can be used to identify them under the GDPR’s “right be forgotten” clause. This data will include common identifiers that Google Analytics stores—including standard Google Analytics first-party cookies, user ID or App Instance ID.
However, Google Analytics has yet to specify details about how organizations will be able to delete user data. While the process will be animated, Google says more information “will be available shortly,” presumably ahead of the GDPR deadline.
Google says it is also drafting a new “EU User Consent Policy” ahead of GDPR. This policy that customers will have to agree covers organizations’ “responsibilities for making disclosures to, and obtaining consent from, end users of your sites and apps in the EU.”
Another key modification Google has yet to reveal is the interfaces Google Analytics will offer that will allow organizations to make changes to how their users’ data is stored. Whether DevOps will be able to automate these processes or not will depend largely on whether Google Analytics will offer a GUI, an API or both.
“If Google Analytics offers a GUI with no API, there is not much a DevOps team can do to automate it outside the existing functionality, since you need a person to do that. But if it’s an API, DevOps teams can weave the controls into their existing frameworks,” said John L. Myers, an analyst for Enterprise Management Associates (EMA). “If both are available, then you’ve got the best of both worlds.”
Additionally, an API would allow the retention and deletion tools to work within existing DevOps frameworks and to become automated as part of business processes, Myers noted.
The DevOps Challenge
DevOps will obviously need to be prepare for GDPR in other ways outside of the scope of changes to Google Analytics or face fines up to 4 percent of their annual revenues or €20 million ($25 million), whichever is greater. Indeed, GDPR applies to all organizations worldwide that collect data from users in EU countries. In fact, it is often incorrectly reported the new privacy mandate applies to just EU citizens, while instead, the regulation concerns data collected from any user who is physically located in an EU country, regardless of nationality. Regardless, it is also obviously impossible to determine whether user data protected by GDPR including cookies, names, email addresses, social media posts and other information belongs to a user with an EU IP address who might just be visiting the region or is a full-fledged EU national.
Given the far-reaching provision of GDPR that places strict controls on how third parties store and share data and protect their online privacy, GDPR likely will fall under an urgent DevOps project category, requiring the input of the change advisory board (CAB), as well an organization’s IT operations, QA, InfoSec and development teams. The degree to which GDPR will change code deployments and processes will depend on the organization’s business model, of course. GDPR compliance should also require review of qualified legal counsel.
Meanwhile, Google Analytics says its tools can help organizations to comply with GDPR for data collection policies. These features include cookie settings, privacy controls, data sharing settings, data deletion on account termination and IP anonymization.
However, as mentioned above, Google still need to provide DevOps with more clarification about certain changes to Google Analytics affecting organization ahead of the May 25 deadline. Many in the DevOps community are also very likely waiting to learn about other ways DevOps can use Google Analytics for GDPR compliance.
Helping out a friend, I had reason to revisit low-code/no-code solutions and poke around a bit. The industry has been relatively quiet on the topic, but changes in systems and marketing made me feel the need to offer an update on the relation of current solutions to DevOps.
Low-code/no-code solutions have been a panacea in the eyes of some both in and outside the industry for decades. 4GLs were the earliest widespread attempt to make an environment that eliminated the need for developers, and while some years are quieter than others, the movement has not slackened.
There are even some things we should consider adjuncts to low-code/no-code environments that normally are not considered. Traditionally the drive for low-code/no-code solutions is to move faster than IT normally turns out solutions. This would force us to consider the spreadsheet in accounting with 50,000 lines of special functions as a companion solution—faster than waiting for IT and more agile.
The thing is that low-code/no-code environments have increasingly realized that IT is going to be their best customer, adjusting their offerings to focus more in that direction. At the same time, agile development and DevOps have made low-code/no-code solutions more palatable to an IT shop that is looking to continue the gains brought about by these technologies.
The future appears to trend toward low-code/no-code solutions being an adjunct of agile and DevOps. I don’t say this lightly, but the signs do seem to point that way. Agile shops, having become more responsive to business needs, will hit a limit on what they can deliver, while the appetite for faster and more responsive systems delivery will not abate. Low-code environments that offer the following will become a very real option to extending the gains of agile and DevOps:
A clear and understandable way to code when needed in the language of choice for the IT environment.
A high level of feedback on what is wrong, possibly including suggestions on how to fix it.
Transparency in what is actually happening behind the scenes with the “no-code” part of the system.
The idea is simple. By handling tasks that we, as developers, do over and over and over, we are freed to spend more time doing the truly innovative and environment specific work. This would seem to offer the next step in the growth of agile.
Those tools that offer options such as automated deployment will step into ARA land also, making the Ops side of DevOps happy—in shops that haven’t invested a fortune in setting up ARA already, anyway. If low-code can offer the ability to say, “Here is a problem,” then the developer can submit a fix, and it is submitted to test, and if tests pass is sent to production (and some of the tools do), then the only questions are, “Does the code generated by the low-code part perform? and, Is it easy to integrate existing libraries, etc.?” If the answers are yes, then we also have an add-on for DevOps that delivers value.
We live in interesting times. It will be very cool to see this all play out, and interesting to see where it heads after this. At least one vendor is betting on artificial intelligence (AI) and machine learning (ML) to offer increased productivity.
Of course, it isn’t a panacea. I read where a vendor said, “We automatically update your queries when a table is modified,” which spawns another dozen questions for me, starting with “Really? What do you do about code that was counting on that field much later in processing? Do you check that screens are displaying defaults, or that variables used to make decisions aren’t always the default value? Do you flag this stuff so devs can ‘fix’ what automated query changes ‘broke’?” I’ll be digging in some more, because now they’ve got my attention. So you’ll probably be hearing more from me about this topic.
Meanwhile, let the future worry about the future, and keep kicking it today. Just stay aware of what else is going on out there that might be handy in your toolbox.
Culture and resistance to DevOps processes appears to be growing at many organizations where DevOps projects have started going mainstream and involving more enterprise groups.
Cloud automation vendor Quali recently surveyed 1,300 DevOps and IT professionals on some of the key barriers to DevOps adoption and found more respondents identifying culture as a top issue compared to last year. Twenty-two percent of the professionals in Quali’s most recent survey cited stagnant organizational culture as the biggest impediment to DevOps—a substantial 8 percent increase over the 14 percent who said the same thing last year.
Quali interpreted the trend as likely the result of more groups getting pulled into broadening DevOps projects at many organizations. Innovation cultures and processes are being impacted in a broader manner as this is has happened. The trend shows why managing culture has to become a top-down responsibility, the firm said.
“As technologies, processes and tools go mainstream, they touch a larger number of people who may need to change the things they do or how they do it,” said Shashi Kiran, chief marketing officer at Quali. “Eventually, change is never easy. It is actually a good thing that culture is there at the top, which means change is touching a lot of people across the organization, which eventually should be net positive.”
An October 2017 report from analyst firm Forrester Research also suggests that more organizations are at least beginning to recognize the need for a cultural shift. Its research shows organizations are making culture a core focus when implementing DevOps projects. The focus is on shifting the mindset of teams to a culture of experimentation, where failure is used as an opportunity to learn and evolve processes instead of as a reason to add more controls and checks. “These mindset transitions are imperative for successful DevOps, and the market trends are encouraging,” Forrester noted.
Other Barriers to DevOps Adoption
While culture is a top barrier to DevOps adoption, it is not the only one. Quali’s survey showed application complexity and legacy processes to be major factors as well, with 20 percent and 21 percent respectively identifying the two issues as among their top barriers to DevOps adoption.
The survey also found that DevOps teams at many businesses are still heavily dependent on their IT organization for infrastructure and continue to use a ticketing system to request and receive required resources.
Nearly 50 percent, for instance said they had to wait for up to one month to get access to infrastructure resources that they had requested from IT, while the wait was even longer at about 24 percent of the organizations in the Quali survey. Only 27 percent—or barely more than 1 in 4—of the DevOps teams represented in the survey get access to requested infrastructure in less than one day.
The numbers are important because slow access to required infrastructure often could have a big impact on the speed and productivity of DevOps teams, Kiran said.
“DevOps teams don’t want to be bogged down by infrastructure. To a large extent, it needs to be invisible to them,” Kiran said.
However, the fact is that environments need to be set up for DevOps and many times there is an infrastructure dependency on the back end. “This may be someone else’s responsibility, but the DevOps team ends up waiting for infrastructure to be set up and that’s frustrating for them,” he noted.
Predictability is important in DevOps environment as is planning and being able to accommodate dynamic change environments, Kiran said. “Self-service is a preferred option where people are involved in environment buildouts, modeling different components and in testing and deploying them,” he said. One-third of the respondents, for instance, said they were strongly inclined to adopt platforms that enabled self-service automation.
The Quali survey found that the degree to which an organization automates processes has a big impact on its ability to enable a self-service capability. More than two-thirds of the organizations surveyed had some form of automation. Of that, 33 percent had implemented a self-service capability for R&D, development and testing initiatives. Generally, organizations with automation and self-service reported improvements in production and agility compared to organizations that had fewer automation and self-service capabilities.
So, where in the development life cycle is most of the automation happening in DevOps environments? The answer isn’t clear because automation is happening in a somewhat siloed manner, Kiran said. “Continuous integration is a starting point and larger organizations adopt continuous deployment as well, based on their internal methodology.” But, for the moment at least, there is no platform that can navigate creation of DevOps environments, expose them for testing and push them into production—all as a continuous life cycle—with full-on visibility, Kiran said.
Kiran’s take on automation is similar to that reported by Forrester last year, which also found silos of automation continuing to be a barrier to DevOps success at many organizations. Forrester has identified “purposeful end-to-end automation” as foundational to DevOps success, especially at scale.
According to the analyst firm, organizations that have implemented complete automation of the CI/CD pipeline have been able to enable rapid deployment. However, despite progress, plenty of opportunity still exists for automation and for deeper collaboration between infrastructure/operations teams and development teams, Forrester has noted.
Quali also created an infographic illustrating some of the finer points of the survey, which can be accessed here.