Blockchain is one of the hottest topics in the IT world today—and that creates a challenge for DevOps professionals. Here’s why, and what DevOps engineers can do to remain relevant.
Blockchain, Fat Protocols and a New Generation of Applications
To understand why blockchain threatens to decrease the role played by DevOps engineers in software platforms, you have to understand the difference between blockchain-based applications and traditional applications.
That difference was summed up most famously in an article by Joel Monegro about “fat protocols.” Monegro convincingly argued that prior to the introduction of blockchain technology, most software stacks were composed of “thin protocols” and “fat applications.”
What he meant was that in most traditional web applications, the majority of the value and functionality lies in the application itself, rather than in the protocols on which it runs.
In other words, if you write a PHP application, most of the time that you invest and the value you create center around the application code. The protocols that allow the application to work, such as HTTP, are relatively simple and uninteresting.
In blockchain-based applications, by contrast, the magic happens chiefly at the level of the protocol. The main job of these applications is simply to access that protocol. As a result, the applications themselves are relatively unimportant.
How It Changes DevOps
What this means for DevOps is that, if blockchain becomes as central to the software platforms of the future as its proponents hope, application code will become much less important than it is today.
By extension, the people who write and manage that code—DevOps engineers—will have less of a role to play in software deployment.
If you take a look at how most existing blockchain platforms operate, it’s easy enough to see how this trend plays out. The amount of effort required to write and maintain a basic “smart contract” application that runs on the Ethereum blockchain, for example, is pretty minimal. You don’t exactly have to be a DevOps genius to churn out some if-this-then-that code that interacts with Ethereum.
The same is true of an application that writes data to the Bitcoin blockchain to record a real estate transaction, for instance, or one that timestamps a document to a blockchain to create a record of the document’s existence.
In these use cases—as with almost all use cases associated with the blockchain—the role played by DevOps engineers is relatively small, because there is not much code to write or manage to deliver the platform’s core functionality.
Blockchain-Proofing DevOps
Yet, if the blockchain revolution does come to pass, it won’t necessarily be bad news for DevOps experts.
Even though the core application code required to interact is minimal, crafting platforms that are secure and provide a solid user experience takes more work. This is where DevOps engineers can help.
One of the most glaring problems with many existing blockchain platforms is that their interfaces tend to be difficult to use. Related software security problems are also well-known.
In addition, some platforms have performance and scalability problems. Bitcoin advocates like to talk about how digital currency will make transactions much smoother, but the reality is that it can take hours to confirm a Bitcoin transaction. That’s a big problem if you want to buy and sell things with Bitcoin.
Arguably, these issues originate in part from the fact that many of the people currently designing blockchain platforms are good at recognizing how decentralized, distributed databases can drive innovation, but less adept at writing, maintaining and scaling software and the databases that power it. They are not, in other words, DevOps people.
By helping to build out blockchain applications so that they are actually usable, secure, efficient and scalable, DevOps professionals can ensure themselves a role to play in the platforms of the future. At the same time, they’ll help the technology transition from its current state of hype into something that can be used to solve everyday problems.
Of the tried-and-tested frameworks that allow enterprises to assess DevOps in their organization and how it can be improved, the CALMS model remains particularly useful.
CALMS, which stands for Collaboration, Automation, Lean, Measurement and Sharing, is particularly helpful for analyzing an organization’s DevOps structure, and ultimately, its utility in any organization. The CALMS framework covers all stakeholders within DevOps, including business, IT operations, QA, InfoSec and development teams, and how they collectively deliver, deploy and integrate automated processes that make business sense.
“DevOps as a technique is slowly but steadily maturing, so it’s good to see general assessment methods like the CALMS model,” Holger Mueller, an analyst for Constellation Research, said. “The CALMS model gives a good frame of reference to compare the maturity of a DevOps team, and as such, is invaluable for assessing the state of teams for the transformational change that goes with it.”
Applying CALMS
Here are a few ways CALMS can be applied to DevOps for enterprises either thinking about shifting to a DevOps structure or seek to make improvements to an existing deployment.
Culture
It is long-established that technology adoption must be to serve a business need rather than investing in technology for its own sake. This shift in mindset applies to how the “culture” component of CALMS supports how the forecasted ROI associated with automating a process must be championed across the DevOps team. This mindset is especially important for getting the non-technical business teams onboard, Mueller said.
“DevOps must have the backing from the business executives. That’s a crucial factor in the culture part,” he said. “One thing software teams can do is use successful software deployments to help convince the business executives in DevOps to make the investments. Those are the kinds of tribal events that every DevOps organization needs.“
Automation
Netflix is well-known as an exemplary case study for DevOps automation. Among other things, it can deploy code and software updates in a matter of minutes across its entire massive infrastructure, many times throughout the day. But Netflix’s example of automation does not take into account what can go wrong at firms that want to follow Netflix’s example when automating their software deployments and updates, Mueller said.
“The risk is always deploying too fast and you end up deploying bad code,” Mueller said. “Automation, of course, needs to take this into account and compensate. But not everybody can do that.”
Lean
Once completed, it is relatively easy to document cost savings by eliminating resources. Within the CALMS framework, the corresponding “lean” component in DevOps is thus relatively easy to assess, Mueller said.
The good thing about DevOps, Mueller said, is companies can come to a consensus ahead of time regarding what they consider to be a lean operation.
However, he added, “There is always the risk of being so lean you begin to err on the QA side.”
Measurement
The success of software deployments is relatively easy to measure once completed. However, attempting to measure future deployments as part of a forecast model is, of course, much more challenging.
“It can be surprisingly tricky to measure how good DevOps is ahead of time,” Mueller said. “Measurements should thus allow errors to be pinpointed quickly during the post-deployment stage to fix the code if needed very quickly.”
Sharing
Sharing, in many ways, overlaps with culture. This is where assessing whether all of the different teams within DevOps are working together and communicating comes into play.
“Thanks to DevOps, the days of the lone developer are long gone. But sharing is not always easy,” Mueller said. “Even within separate teams, it is easy to fall into the trap of wanting to get your own work done without necessarily thinking about the team as much as you should.”
The End Result
While applying CALMS to assess DevOps never hurts, the framework is still not as important as one of the main objectives of DevOps—namely, using technology to make organizations’ operations more agile and reactive to customer needs, Mueller said.
“At the end of the day, assessment methods such as CALMS are great, but they are never as important as focusing on implementing next-generation applications,” he said.
The age of functional programming is dawning. The hype cycle has heated up, and IT departments are getting both real and perceived benefits out of the idea that a function runs on demand only.
For some, the benefits are very real. Bursty processing, or things that are frequent and quickly resolved, see a benefit from being implemented on functional architectures, and the cost of functional architectures is arguably far better than the cost of cloud instances. I say “arguably” because you can always cram more onto an underused cloud instance, but you pay as you go—even more literally for functional programming, so additions each have an incremental cost. But it is incremental, and you have the power to choose whether functional is right for any given bit of the architecture.
So why the title? Remember The Island of Misfit Toys from the much-beloved Christmas story, “Rudolph the Red Nosed Reindeer”? They were all just a little off. In a world flooded with perfectly good toys, they were left unattended and unwanted.
That is a risk functional runs in the long run, also. The idea that you can pop a function out there, configure the access point (almost ubiquitously a web application firewall entry point, but things change so we’ll stick with the generic “entry point”), and start sending requests is great. The fact that you only pay for that bit of time when your request is actually being serviced is great, too.
But this ideas assumes security is baked in at the functional level. If you are deploying this function anywhere with a public IP, it assumes hardcore security is baked in.
And failure to have that security baked in is the first reason you might have a misfit toy.
By definition, functional programming-based systems are complex systems. They are the ultimate evolution of microservices. They will suffer all of the headaches and heartaches associated with SOA and add on the issues with REST versioning, etc. There will be 50 or 500 functions, each clipping along and doing its thing, composed to make a single application (in a purist world, composed in a functional programming app).
Some of these functions will become outdated and unused. You can’t leave them laying around, waiting to be abused. Indeed, a new form of financial attack might be to look for just such abandoned toys and start calling them repeatedly and rapidly.
Imagine coming in one day and having Finance ask you why your bill at functional provider X tripled in the last month, only to find a function you left hanging around was being called for no good reason?
But the complexity inherent in lots of small bits of code, each doing a specific task, pretty much guarantees there will be cruft left lying around unless you are very careful.
And that, my friends, is (finally, I know) the point of this blog. Our monitoring tools today lean toward watching the performance of the application, but you need more than that before you go all-in on functional programming. You need build tools that can tell you when a given function is no longer called—from anywhere. That is a tall order which none of the previous distributed methodologies has managed to master, but it is more important in a world where you have functions sitting in various public clouds with public-facing IP addresses.
There also needs to be a definitive source code scanner. Those are slowly getting better, but imagine if a library the build system pulls in and uses for 20 or 30 different functional programming microbits becomes compromised? We’ve seen enough instances of ignoring the problem is begging to be a victim, so scanning on build is more important with more systems using external libraries. Or more accurately, with external libraries being pulled into more microservices. There are companies/service providers that will validate your external dependencies and only let you load open source libraries they have certified as safe. I’ve never used one, but they’re worth looking into.
A list of entry points and places they are accessed from is useful for internal-only applications. You can’t do that with a publicly exposed REST API that anyone can call, of course. I’m a developer and a D&D player, so I’ll use that example: Wizards/Hasbro cannot control where this API is being called from. The same is true for any web services APIs that exist to give others access. But internal APIs that are an integral part of internal systems and not called externally? Those you need an automated way to find references to them in internal source and keep a count. Much like garbage collection, you’ll want to take down ones whose count reaches zero.
The thing is that in many ways functional programming reduces operational complexity. It certainly reduces operational overhead. But that complexity is replaced with a different kind of operational complexity; maintenance becomes a bigger issue, and the already occurring breakup of code into smaller chunks increases, bringing cross-reference complexity along with it.
Keep that in mind. You will have orphaned functions, you will have security concerns, you will have changed operational complexity. Plan for these things from day one. They are not at all deal breakers, and the benefits of functional are already clear in certain verticals with more general benefits being discovered as I type. But they’re the new problems that will require answers in this new architecture.
So, build in oversight and tools from the beginning and go see how functional programming could benefit your project. At this time, it isn’t the answer to every problem, but—like most cool new IT things—it is another tool to have at hand when solving problems for the business.
And don’t let your functional programming microservices end up on the Island of Misfit Functional Programming.
As the cycle of DevOps is repeated in a growing list of organizations, I feel compelled to reiterate the warning that this is not the pinnacle. Anyone who has been around for a while knows that the current fad is not the last fad.
[[Cue DevOps people screaming at my choice of the word “fad”]]
Of course it’s a fad. DevOps (and to a much lesser extent, agile) is the cool toy in the midst of Gartners’ (in)famous hype cycle. That doesn’t mean nothing will come of it; it means that people freak out when you tell them it doesn’t solve every problem.
All of the process/solution fads that have come along have offered some benefit to IT. DevOps promises to offer more, and in many environments it does. But grabbing your broadest brush and screaming, “DEEEEVVVVVOOOOOPPPPPSSSS!” while painting everything in sight is not problem-solving; it is thinking there is only one path to quality and it flows through the Chosen Process.
The reason DevOps is different from previous hype cycles is obvious: It has the built-in assumption that this is not the end state. It assumes you will keep moving forward and things will change. Previous “new way of doing things” ideas have assumed that there was some nirvana you would get to and everyone would relax there, sipping ambrosia and enjoying immortality. DevOps makes no such claim. About as close to a final state as it gets is increased communications between the various parts of the DevOps life cycle. And that is because there is only so far you can go with increased communications before you’re flooded. And even that leaves room for new tools such as Slack to augment communications.
But don’t look at every problem with DevOps blinders. This is the real world, and while some will (possibly correctly) claim that every software dev/deploy cycle can benefit from DevOps, the real question is not “Can it benefit?” but rather, “Is the cost/investment/risk less than those benefits?” And to the chagrin of those who live and breathe DevOps/agile, I’m afraid the answer to that question isn’t always a resounding “Yes.”
I’m undeniably a fan of both agile and DevOps. What I’m not a fan of is finding the nearest round hole and cramming everything—including square pegs—into it. Look at each project/process objectively. As a simple and blog-length example, let’s examine standalone app that does some important back-end process and is only updated once or twice a year: Why ever would you rip the process for that apart, invest man-hours to bring it into the DevOps and agile folds, then … only update it once or twice a year?
If you are building a new project, and you have DevOps processes in the building, it would take a convincing argument to not use them. The same should be true the opposite direction. If there is an existing project that doesn’t have delivery delays, it should take a convincing argument to move it to a whole new development/deployment methodology. Choose wisely. Invest resources where they’re required, not where ever the broad brush falls. IT has enough work to do, don’t add to it, just so you can say “We’re a DevOps shop.” Better to say “We’re a right tool for the job shop.”
And keep kicking it. Business doesn’t run itself, IT runs business. Keep making it work.
There are two basic design methodologies in the software development world: top down and bottom up. In bottom up, the goal is to piece functionality together to “build up” to the desired result. In top down, the goal is to break the problem into smaller and smaller parts until you have implementable pieces.
Both have strengths, and by correlation, both have weaknesses. Analysis and design phases in pre-agile days were almost ubiquitously top down. They would take business requirements, translate them into functional requirements, then break those down and assign the constituent pieces to developers. As the code was implemented, weaknesses in the design would be exposed, and feedback would go back up to the level of functional requirements—or, in really bad cases, to business requirements—and a mini-iteration would take place to resolve the problems.
When agile came along, this was all too slow a process. To be sure, it was (and is) still used and has produced some of the world’s best software, but an agile team can’t sit idle waiting for a long process of analysis. Kan-Ban and similar tools/processes were introduced to address this issue, making the pieces being designed smaller and thus speeding the process somewhat. But smaller pieces implies bottom up to some extent, and the waters got a bit muddied.
Test driven development (TDD) was designed to make sure the code written met the technical requirement by writing a test for the expected result before the actual code. This was one of the most fruitful things to go into/come out of agile development. While it has flaws—the most significant being the human factor—it does make more thought go into code being written and more rigor come out of code produced. The act of writing the test makes the developer think of ways to solve the problem that satisfy the test. But the source of those requirements still needed to come from some form of process.
Enter behavior driven development (BDD). It’s not really development, though the name is catchy. It would be better named behavioral design for TDD, but that’s a mouthful.
In short, the idea is that business owners and analysts speak in terms of the needs of a given role, while developers speak in terms of functionality or the results of a given test set. This creates a gap that causes misunderstandings. The goal of BDD is to get everyone speaking the same language. It tries to extend the spirit of TDD up the design/analysis stack—”shift it left,” in today’s parlance—so that the tests being developed are a hierarchy stretching from acceptance testing down to unit testing, and everyone is speaking the same language.
En vogue today for the language is Gherkin, a simple requirement specification language that writes requirements and test criteria with (relatively) strictly defined sentences. I say “relatively” because, the language is a bit loose from a unit test perspective. But from role-based specification, it is locking down options to make the environment and test cases clear.
I actually got to thinking about all of this while considering SmartBear’s recent acquisition of HipTest. SmartBear has made an investment in making its tools BDD-compatible, and HipTest is largely a BDD platform. The combination of the two should see some movement in the BDD space that will be interesting. I spoke briefly with SmartBear about the acquisition, and it seems the company really is merging the strengths of the two organizations, so it is worth watching over the next few months.
Thus far, the benefits of BDD that jump out are the ability for everyone to talk about an element of the product in common terminology, while the developer can think “down” to the code required to meet the requirement and the business analyst (or owner) can think “up” to determine how to market this cool new feature.
Tools are out there to generate TDD test classes/cases for a variety of languages—notably from my stable, Python and Java. This might be a huge step in the right direction also. Just the other day I was having a bit of a lighthearted conversation with a peer and said, “Get to the point where you can auto-generate tests at build time for us, so IT managers aren’t stuck deciding which lines of code are/aren’t worth maintaining test code for, and then we’ll talk.” Tools and frameworks such as Cucumber and JBehave actually take steps in that direction, though their approaches are different.
We are not currently using BDD on our in-lab test application (Android/web clients, Spring API server) at work, but will have to consider it moving forward as another bit of DevOps tooling and practice we can experiment with. The tools are there, and the process bridges that sticky part of analysis and design where top down meets functional.
After all, the end goal is giving users what they need. If BDD helps make that process smooth and faster, it is a fit for any DevOps environment.
There may be nothing more difficult in a DevOps transformation than aligning the business with its technology organization—and vice versa. Dealing with skills gaps and information gaps, while addressing communication breakdowns and cultural hurdles can keep even the most seasoned practitioners and leaders up at night. How do we align disparate business units accordingly? What practical tips can people take and implement along their journey?
These are topics we immersed ourselves in during a live video podcast ahead of the DevOps Enterprise Summit (#DOES18) in London. I was able to sit down with Carmen DeArdo of Nationwide and Jayne Groll of DevOps Institute to take a deeper dive into the DevOps transformation patterns and practices. This discussion centered on a major theme of DOES18 this year: business and technology alignment for the modern enterprise.
We hit on many important issues that I think we can all learn from and apply in our organizations respectively. One of the things we’ve learned over the course of the years is that, in DevOps, the focus is heavily cultural, but it has to be squarely on nurturing and getting buy-in across the organization. But at the same time, we need to be orchestrating our strategic alignment initiatives between the disparate functions and organizations within the business. That’s a tall task.
Watch the full video replay below, and/or read through the full transcript to see what key points were made and how we can adopt best practices to move our businesses forward.
Carmen and Jayne will both be speaking at DOES18 London this year and I will be leading video panels that you can see exclusively on DevOps TV / DevOps.com. Hope to see you there!
Alan Shimel: Hey, good day, everyone. Thanks for joining us. This is Alan Shimel of DevOps.com. And I’m here to talk to you all today with a few friends of mine about the upcoming DevOps Enterprise Summit in London. This will be a live panel discussion ahead of the London DOES event, which, by the way, is June 25th to the 26th at the Intercontinental Hotel, sometimes called The O2. Our panel today, I’m happy to say, are two really good friends of mine. Well, one is one of my business partners at the DevOps Institute: Co-founder and CEO of DevOps Institute Jayne Groll, and a speaker at DevOps Enterprise Summit London. Jayne, welcome to our hangout today.
Jayne Groll: Hi, Alan. Hi, Carmen. Thanks a lot. Really looking forward to it.
Shimel: Okay. So Jayne has already sort of introduced our next guest there, another good friend of mine, a recognized DevOps leader for many years. He really kind of kicked off a lot of the kind of groundbreaking DevOps work that we’ve seen at Nationwide over the years. Also a great Pittsburgh Steelers and Pittsburgh sports fan, Carmen DeArdo. Carmen, welcome.
Carmen DeArdo: Welcome Alan. Welcome. Hi, Jayne. It’s always great talking to you.
Shimel: Thank you. So all kidding aside, both of you are kind of experts and leaders within the DevOps Enterprise community, and you’re both presenting at DOES London this year. You’ve actually both presented at previous DOES events as well. So thank you for coming here. You know what? When you guys first started kind of preaching in the wilderness about DevOps and so forth, it was a bit of the wilderness, but it’s becoming commonplace in businesses and organizations throughout the world now. Our goal on this chat, guys, is to learn about the patterns and practices that leaders have identified for business and technology alignment, and really what we’d hope is just to share your experience and thoughts with our audience so that maybe they can learn something and they could integrate or assimilate that into their own transformational journeys and escapades.
And just again a reminder, this is about DevOps Enterprise Summit London, June 25th to 26th, and you can get more information on the event at events.itrevolution.com. You’ll find on there, by the way, the entire conference agenda, and beyond Carmen and Jayne, it is a gangbuster lineup of speakers. If you’re interested in DevOps and any kind of related type of technologies and frameworks, etc., it’s really a can’t-miss one for those of us, you know, who follow this space, or even for those of us just getting involved in it.
That being said, though, I’ve spoken enough. Let’s get started. I’ve given a little bit of your background, but maybe I haven’t said everything that needs to be said. So, Carmen, since I see your lovely face on the screen ahead of me here, why don’t you go first? Give us your background, your company role, etc., professional accomplishments.
DeArdo: Okay. Thanks, Alan. So I actually spent most of my career in telecom working for Bell Labs. Thought I would retire from Bell Labs like everyone else at the time, and then the telecom bust happened, and we all had to figure out what we were gonna do with the rest of our careers. But it served me well, because I think a lot of those concepts, even though we didn’t call them DevOps, but, you know, certainly, creating high-availability, high-reliability capacity systems, you know, those concepts are still, I think, pertinent obviously today. I moved into Nationwide to really help with trying to get Agile set up about 12 years ago, and then, you know, as I think you see the case with a lot of companies, they start somewhere in their value stream. They may not even think of it at that point like an end-to-end value stream, but they’re starting somewhere. They start with Agile.
You know, we were very successful getting all our work done there, but then realized, okay, that’s only part of the story. The cards getting into the backlog through iterations is great, but we aren’t getting flow of work from the business into the teams, and we certainly aren’t releasing into production frequently. So about, I’d say, five years ago, you know, somewhat in affiliation, some of the work I was doing with, as you know, Rosalind Radcliffe, Hayden Lindsey, you know, started to talk about what’s this DevOps thing. Was lucky enough to end up at the first Gene (Kim) conference speaking with Hayden. Wasn’t sure what I was getting myself into, but it’s been a wonderful experience. You know, got to meet you, Jayne, and lots of great folks from Gene’s community.
In Nationwide, we’re just continuing to work, how do we make that end-to-end flow and lead time, you know, what do we do to make it better? What DevOps practices can we bring? But as you noted at the beginning, how do you get the business more online? How do you get work to flow? And how do you address culture, which we know is probably more important than technology?
Shimel: Excellent, dead on. Jayne, how about you? You have more hair than both Carmen and I. You’re the lovely face we should see on the screen here. Give us a little background and accomplishments.
Groll: So I call myself an accidental technologist. So I started my career, got out of school, ended up in the legal field for about 15 years working on the business side. So I wasn’t a lawyer, but I was one of the early paralegals for a big Wall Street law firm that adopted UNIX a long, long time ago. I don’t even want to say when, but screens were black and green back then. And so everybody had to learn UNIX. And so I was very fortunate to be given an opportunity to accidentally become a technologist, and I know even with my technical skills, I’m a little self-taught. Was able to move up into some senior management roles across several different verticals, leaving the legal space.
And then in 2004, kind of fell on ITIL and really started to see a paradigm shift in IT that technology is important, but your practices, your principles, your processes are important as well. And people, process and technology became a mantra. And so co-founded ITSM Academy with Lisa Schwartz and really saw the growth of maturity in IT. You know, I always say we love our frameworks. IT, we love our frameworks more than any other business unit, and really started to see that percolate. Met Gene and some others through that experience, but really, landing where we are today. In 2012, I was able to attend an early DevOps day, frankly, at Gene’s invitation and really saw something percolating there that was pretty much common sense, but not common practice.
So we started to see, you know, should Dev and Ops collaborate better? Should there be a faster flow? Should there be less silos? Yes. Should we use automation more? Yes. We automate the rest of the world, we don’t automate ourselves as much. And really started to see something grow there, and, Alan, as you know, in 2015, we launched the DevOps Institute primarily because the practices around DevOps unlike other frameworks were really emerging in the wild. You know, you said it before. They were emerging in the wild and making some sense out of that, you know, as it crossed over into the enterprise space was important and requires a lot of research.
So, you know, we don’t have a single body of knowledge in DevOps. Thankfully, we have a collective body of knowledge in DevOps. So for me, I think part of my, you know, pride is my journey has not been a linear path. Really blessed to be, you know, where I am today, and, you know, DevOps Institute right now is, what, 32 countries that are talking about in doing DevOps. I mean, that’s really kind of surprising and remarkable all at the same time.
Shimel: Yep. And just to excuse, I don’t know if you guys can hear. We have a train, the 11-08 Cannonball Express, going via the office here.
DeArdo: Just like my cousin Vinnie here, Alan? Is that like …?
Shimel: That’s “two yutes.” The way I talk, we’re not far off. Anyway, though, guys, let us move along here. You know, one of the things that I’ve learned in my own exploration of DevOps and digital transformation, etc., is that a lot of businesses, large enterprises, they may think they do, but on the whole, they don’t really understand technology. Yeah, some of the guys in IT understand some of it or what have you, but do you think most businesses understand technology? Carmen?
DeArdo: I think it’s … Obviously, people are exposed to technology. In fact, I talk about this all the time, that sometimes we’re still in the book writing business or book production that we still feel like we got to produce manuals of paper material … The disconnect comes as, How we can effectively utilize the technology that we have to meet the business needs? And I think that’s maybe where the disconnect goes. It’s not that, you know, people certainly are … they have Alexas. They have stuff in their home. They may be even, you know, great at programming their home devices, but how are we actually taking advantage of that to serve our customers? And I still think thinking about things kind of in this … you know, Jayne talked about silos, but I actually think the last frontier really of DevOps is in the left part of the value stream, not the right part.
I think they code the cloud. There’s patterns, maybe not everybody’s done it. I know it’s not easy to do. That’s pretty well-defined from a pattern. The front part is not, and I think understanding that, you know, you have to basically be an IT business. We hear this all the time. If you’re not an … consider yourself an IT business, someone who is good at IT is gonna disrupt you. So it’s not that I don’t think they understand technology. They’re certainly capable of it. It’s just, do we really understand how we can use it the best way in order to meet business needs? And, you know, things like the system of engagement. Well, the system of engagement is gonna change. We’ve already seen it change. It changed from web to mobile. It’s going to voice. It’s gonna go to different engagement models.
That’s fine, but you still have to have the back-end technology, and the microservices, and the APIs, and the ability to deliver more quickly, and I think that’s the part of it, the gap we still haven’t bridged. And how can we actually get …? You know, one of the things that Nicole and I are talking about at the conference is the project-to-product model. And I think large companies like ourselves are kind of flailing at what … you know, that makes sense. It makes sense to align around a value stream, but how do you really define what those value streams represent so that you can actually get that flow and then use technology to help get that quick deployment and, you know, feedback? So I think that’s where we see a lot of the gaps, you know, from that perspective.
Shimel: Sure. Jayne, before I have you answer on that, you know, I got an email of pitch. I get a lot of pitches as you can imagine, but I got one the other day that, to me, really nailed the issue. The issue was they pitched me about how great their solutions and their services are, and they said it makes IT costs less. Instead of … so they’re still thinking of the IT department as a cost center instead of a profit generator. And to me, when you say that, that means you really don’t understand technology, you don’t understand the role of IT in the modern enterprise, and you’re just not doing it right. Jayne, your thoughts.
Groll: So I think you’re right. I think, you know, the modern enterprise has a few challenges ahead of it. Carmen, I absolutely agree with you. I think every business knows they need technology and they need to accelerate their use of technology. I think they don’t understand how to apply that, and more importantly, I think the big paradigm shift, and, Carmen, you said it, is that most companies don’t realize that they are now technology companies. It isn’t even just IT does for you, is the core company as a technology company. And those that kind of recognize that, I think, are those that move ahead. Those that are still kind of hanging on to a legacy … Now, you know, I’ll add kind of something into that recipe that’s a little scary.
With the way technology is moving and how fast it’s going, I mean, the acceleration is remarkable, and, you know, Carmen, you said it. It was cloud. It was mobile. Now it’s voice. That it feels like, you know, this kind of moving target that if I’m a business leader, not a technology leader, although Lord knows that, I think, has a plan here, you know, where do you put the dart? You know, do you invest in mobile? Do you invest in voice? Do you invest …? So it’s such a moving target, and it’s going so fast that I think it can be very daunting to figure out, okay, where do we make our play, and, you know, where do we use our technology spend? And I would say something funny, Alan, because you’re right.
You know, in my years in senior IT management, I almost always reported to the CFO, because I was a call center. And so it was nobody understood quite what I was spending money on, but they figured if I reported to the finance guy, that somewhere along the way the cost would be controlled as opposed to let’s optimize the capability. Let’s look at this in investment. So I really think organizations are trying to figure out, first of all, are we a technology company? And I would say unequivocally the answer is yes regardless of what vertical you’re in. And then where do I invest? What technology do I start looking at? And if what I do today is gonna be obsolete tomorrow, how do I make that investment for my company further?
Shimel: Excellent. So, guys, you both have a little bit of a background, more than a little bit, in training and so forth. Do we train people outside of the IT organizations on what technology can do for you? Do we put up sort of Uncle Sam’s “We want you. What can I do for you?” posters? How do we get the business to understand the technology aspects, the technology side of things? Go ahead.
Groll: I mean, you know, the “we want you …” unless you can promote something that says what’s in it for me, me as an organization, me as an individual, me as a leader, I think that’s difficult. You know, doing road shows on here’s what technology can do for you isn’t something that you can tell. I think it’s something that you have to prove. And I think IT sometimes runs into an uphill battle trying to prove that, because it is costly and it does change a lot of the way people perform on a daily basis. I know what I did today, and tomorrow you want me to do something differently, and I may not be as successful as I was yesterday.
So there’s a little bit of a proof of concept usually, you know, done through piloting, whatever, but I think the, “How do we tell them?” I don’t think we tell them. I think we have to show them.
Shimel: Fair enough. Carmen?
DeArdo: Yeah. I mean, I think we’ve gotten a lot of appreciation. I mean, I’ve told … you know, and Cindy Payne and Jim Grafmeyer told the story at the last conference about Hurricane Harvey and how the business was able to respond within a day to what was a wind event and came a rain event with all our flooded cars and how we got the website optimized to handle those, to take the load off of our customer service center. They’re starting to get that, “Gee, having this capability to deliver things more frequently …” We now have some of us who deliver daily whereas before everything was going into a monthly release. So that’s a great accomplishment.
On the other hand, we’re running into the things that, you know, Topos talked about with compliance, with legal, with … and just, you know, how do we … Most of our time now on lead time is going from the idea to the backlog of the team, not from going from the … you know, the team getting the work to deployment. So I think going into, you know, that view of how do we start to think more about, you know, the other things that are getting in our way of lead time, because it’s not just the IT part of it. It’s now the business part of it, and we have a couple areas now who actually have true product owners sitting with their lines and are almost at the point now where they can deliver, you know, stories and storyboard them and move them all the way through production, you know, much more quickly.
So I think that’s an example, and I think that’s what you need. I always talk about you need stories, you need examples of how this can actually be done at your company, because reading about it, hearing about it is great, but then when you see the team next to you doing it, and even if it’s not perfect and even if you know it can be improved, that’s where really you start to change the culture of not just IT but of the business. You want something like that. And then, you know, I think you got something going that you can really build on top of. Jayne.
Groll: Yeah. I mean, I think that conferences like DevOps Enterprise London, Las Vegas, other events, local events, I think, also let individuals and organizations know they’re not alone in this journey. You know, part of the challenge in DevOps right now, I think, is we’re going from what is DevOps to how do we do this. And it’s not finite. It’s not, you know, you open a recipe book and it says, “Start here. Add these ingredients. You know, stir, bake for 350 degrees, and you have DevOps.” It requires a lot of higher-level thinking, requires some experimentation. And so I think that, you know, access to what have other people or organizations done … Carmen, I think you’re absolutely right, what have other teams done, really kind of making it a community of enablement is very, very important in this space, because everybody’s trying to figure out where do they start. And unfortunately or fortunately, the answer is it depends. It depends where you are, it depends what you’re trying to achieve. So I think that’s part of it as well.
Shimel: Yep. So, guys, let me try to tie a bow on this section before we move on. The reason I asked about the technology and technology education issue is I think it’s imperative for organizations to understand that in order for us to get what I call business and tech alignment, and what I mean by business and tech alignment is that we’re all pulling in the same directions for the same reasons with the same goals, and we’re following hopefully a road that we have agreed upon, and we all know the road we’re traveling, because, again, you get back to DevOps, guys. You know, when you guys are out there talking about it and we were covering it at DevOps.com, what do we always hear? What exactly is DevOps?
There’s no real definition. What does it mean? You know, how do we define DevOps and so forth? And I think one of the things we’ve learned over the course of the years is that, you know, in DevOps, the focus is, you know, heavily cultural, but it has to be squarely on nurturing, buying across the organization for this “DevOps initiatives,” you know, that we’re starting. But at the same time, we need to be orchestrating our strategic alignment initiatives between the disparate functions and organizations within the business. And that to me is business and tech alignment. If you guys don’t agree with that, speak up, but if you do, let’s move on to our next part, but can we wrap up, Carmen, Jayne? Are you fair to say that what we’re really talking about here is business and tech alignment?
Groll: So I’m gonna jump in just for a second, because … So you know I come from the ITIL space. And so in the ITIL space, they were coining the term BITA, business IT alignment. And so for years and years and years, there was a lot of conversation about BITA and its semantics. But alignment means that we’re kind of standing side by side in lanes next to each other. The reality is it isn’t alignment. It’s really unification. It’s really we don’t need to align with the business, we need to become the business. And, again, I know it’s semantics, but that word alignment gets tossed around a lot, and it implies something like, “Well, I’ll drive my car next to yours, and we’ll travel at the same pace, and we’ll get to the same journey,” as opposed to, “Let’s be in the same car.” And so you understand what I’m saying?
Shimel: Yeah.
Groll: And I said the term BITA is sort of falling away a little bit, but it was thrown around a lot during the years when ITIL in particular, you know, was kind of front and center.
Shimel: Yap. Carmen, any thoughts on that before we move on?
DeArdo: No, that was great points. And, you know, I refer back to Mark Schwartz’ last book, “The Seat at the Table,” and where he really talks about IT really should move beyond trying to get a seat at the table and move towards, you know, as Jayne said, you’re actually moving together, and you have to merge really these concepts of what the business wants, you know, has to be very much aligned, more than aligned. It’s part of what IT need to be doing. So that merging has to take a fact, and I think as much as we’ve seen the culture issues in Dev and Ops, that’s gonna be an interesting culture issue. But we’re seeing the companies who are successful are already doing it. They are working together. And, you know, if you walked up to some of the teams I just talked about, you wouldn’t be able to say, “Oh, well, such-and-such is a business person, and, you know, Jayne’s a business person, and John’s an IT person.” No, they’re a team working together to achieve something for the good of the company, and that’s the way it has to be.
Shimel: Yep. Guys, let me broach another subject then, because I think we’ve covered this one. You know, I read an article yesterday about AI, and automation, machine learning, and we’ve all seen these before, and, you know, how it’s changing. But automation really is changing the job market, especially for people younger than the three of us. I mean, we’ve both seen our … all three of us have seen our 21st birthday recently and …
Groll: Just recently.
Shimel: Recently, but really the folks coming up behind us, and we’re talking about our children and so forth, automation and the job market that they’ll grow into is paramount. And one of the reasons is that we have a tremendous skill gap. You know, my background is obviously in infosec, and now we call it cybersecurity. I hate the term. But there’s a tremendous gap in the security space. There’s also a tremendous gap in the DevOps space, obviously, and that’s one of the reasons we see such a high demand, even for people who may not be certified or classically trained in DevOps, I mean, take some sort of DevOps class. If they’ve learned to work Puppet or Chef or Jenkins, you know, they’re in high demand here.
And so with this skills gap, it doesn’t hinder our DevOps transformation. Is it a drag on the entire DevOps industry or community? Jayne, I’m gonna ask you to take first crack at that. I mean, you must see this with DOI stuff.
Groll: I do, and, you know, it’s kind of like good news, bad news. So the great news is, where else do you get to work in an industry or an environment where you are now encouraged to learn something other than what you’re essentially good at? I mean, you’re now encouraged to grow new skills to evolve. Organizations are funding skills modernization programs. So, you know, we groomed … and Carmen referenced that. You know, we groomed an IT based on silos. And in every silo, you were a developer, or you were a QA person, or you were a security person, and you lived in your silo with other people who had the same skills as you did. And now, cross-functional skills and, you know, what they call T-shaped people where you need to still have a specialty but you’re absolutely required to have very large breadth of different skills means that organizations are supporting and funding that.
So the continuous learner is more the norm today than it was before where you will be a developer … experience, and I think it’s a great day to be in technology truthfully, because you are encouraged to think, and learn, and practice, and train, and do, you know, all sorts of other things. And I think companies are starting to see that. I hear the term “skilling” a lot. We not only talk about training, we talk about skilling. We need to skill our force. We need to skill our teams. We need to reskill or upskill. You know, those are really terms that are starting to enter our common vocabulary. And the day of the generic, “I’m a master, I’m an expert, I’m a practitioner,” and I’ve got a few of those credentials as well, those are so ambiguous that now the skills are very, very tangible, and organizations are hiring, I think, hiring and funding individuals to develop not only technical skills but practice and principal skills, too. So we’re all gonna be upskilled, maybe outskilled.
Shimel: Perhaps both. Carmen, how about you on this one?
DeArdo: Yeah. I mean, automation is obviously nothing new. I mean, I’m really dating myself, but I can remember when people didn’t trust the output of compilers. Like, are we sure the compilers are producing the right machine language or assembly language? I don’t know. I can bring the assembly language better than this compiler. I mean, I think automation and learning is just … obviously, you know, if you spent, you know, four decades like I have, and, you know, none of us are spring chickens, but you’ve got to learn a lot and keep learning. Now, I think the great thing, and I agree with you and Jayne, the great thing about Nationwide is they have invested in learning and upskilling their people. We had this thing called Teaching Thursdays where twice a month we have two hours which is set aside for peer-to-peer learning, and it’s really not … and it’s peers teaching people, and people love to be the teacher.
I mean, at the last forum, Jason Cox was talking about … We had a subgroup focused on operations, and Jayne was there. And talk about … you know, he had some folks on his team, that they have to become engineers. They have to become coders. And one of the guys who initially maybe wasn’t so crazy about it, once they became kind of the expert in their area and they were teaching other people, well, you know, they were really engaged then, because, you know, they were involved and they were the one now from a peer-to-peer learning perspective.
So companies, I think, like Nationwide, Disney, other companies that are leaders in this space need obviously to invest in that. And then, you know, it also means that the associate themselves, you know, needs to take advantage of that. I mean, we can offer those capabilities. We know the jobs are gonna change. We can make those skills available, and then folks need to take advantage of them, but yes, I mean, people have to become more broadly skilled and be able to become more of what I would call problem-solving engineering mindset as we move forward.
Shimel: Sure. So, guys, though, you know, so we have skills over here. And on this side, there’s another thing that creates a tension with the skills, and that’s what I call the information gap. So we have a skills gap, and you guys have both addressed that really well, but we also have an information gap. And I alluded to it when I talked about training. If an employer or an organization thinks that someone who can write a recipe in Chef or has heard of Jenkins is a DevOps engineer, if there’s such a thing as a DevOps engineer, or is someone who, you know, is gonna really help build their DevOps team, to me, that’s not a skills gap, that’s an information gap. How do we understand … How do we make organizations … How do we fill that gap on information so that organizations recognize, A, that there may, in fact, be a skills gap, B, how to fill that skills gap and bring the organization up to it?
I mean, this tension, this chicken-and-egg, if you don’t know it, you don’t know. How are you supposed to know? And so I’ll put it to you. What do you think is more important? The information gap, the skills gap? Are they equally important? Do we tackle them concurrently, individually? Carmen, how do you guys do it at Nationwide?
DeArdo: I mean, I think one of the things that IT hasn’t been good at is treating IT as … we’ve been a very bad customer to ourselves. So if you look at our pipeline, we don’t treat it as a product. If you actually want to get faster at delivering a capability from idea through deployment, you need to consider that as a product, not a bunch of different tools. It’s not Jenkins, and New Relic, and UrbanCode or, you know, the AWS. It’s not that. It’s a pipeline that’s gonna enable you, and it doesn’t start at code, it starts at idea to get you all the way through that deployment. So then what … do you have a team? Are you doing thinking across the pipeline? Are you doing architecture? We don’t do things like value stream architecture or architecting the pipeline as a product. We want to buy a bunch of technologies. We can hire people good in those technologies, but as you said, that’s not really gonna get us where we need to go.
So I think the first thing is, do you really treat your product … One of the first things, do you treat your pipeline as a product? Do you have a team that can actually support that product all the way? Do you have a product owner who can drive continuous improvement across that with funding and prioritization? And do you have one of your best architects, if not the best, to actually create the architecture and do value stream architecture, which is something, you know, that Mik Kersten and I have been talking a lot about. I think that’s one side of it that we don’t get, is teams could … if you ask a team, they can kind of figure out, tell you like a Deming quality circle, how they can go faster. And you can experiment, but then if you’re gonna operationalize that, you don’t want 200 teams doing things in 200 different ways. You can then take that and say, “How do I address that as part of my delivery capability?”
And then I think what you need your leadership to do is apply systems thinking, and that’s a whole another subject we could get into, but I don’t think executives and leaders know how to really address the, “How do I optimize this as a whole?” They want to get into the weeds. They want to get into the tool selections. There’s people that can figure that out. You need to really focus on the strategy and how you can optimize all this great information you’re getting to make sure you’re removing barriers and allowing teams, you know, doing the learning, making sure they can be upskilled, investing in their ability, and allowing them, you know, to be able to innovate and improve your delivery capability. And I think that’s what I just said quickly. It’s easier said than done. And I think that’s where I see a lot of the struggles, and I think where we’re trying to address this is how do we start to put some of those pieces in place so it’s not just a bunch of, you know, divergent disparate technologies we’re trying to bring in and hope that that’s gonna make us get better. Because those things and having skills just around those things is not gonna actually get us to where we need to go.
Shimel: Fair enough. Jayne, I’m gonna give you a chance to comment on the DevOps skills versus information gap.
Groll: So I think the two are really related, and I think Carmen brings up some good points. You know, I think it all comes down to silos. So let’s look at our silos. Well, within our silos, we speak the same language. We use the same acronyms. We use the same tools. We live in a common community where everybody kind of understands what’s going on and what’s happening. And then when we leave our silo and we try to talk to a different silo, we don’t speak the same language. We don’t use the same tools. We don’t have the same processes. So I think the skills gap doesn’t fully overcome the issues with the information gap, but, certainly, there’s help there that if we’re looking to share information, we have to share it in a way that everybody kind of has some type of uncommon understanding.
The other thing with information is, you know, information is really interesting to me, because everyone thinks they communicate a lot. People generally think that they communicate well. You know, teams generally think they communicate … like I told you that. You told it to me maybe in language I don’t understand or it wasn’t clear. It was some report that, you know, I don’t even know where it is. So I think, you know, kind of the flow of information and from a vocabulary perspective, from an understanding perspective, from a “Did you understand what I shared?” is something we all need to overcome. And I think, you know, again, they’re so interrelated to each other that if we start to create a broader set of skills, hopefully, that helps to improve the communication. I think the other side of it is we talk a lot about collaboration, and there’s a really fundamental difference between communication and collaboration.
And so information flows through communication, but communication is pretty much one way. I tell you, you tell me, and somewhere along the way. Collaboration is I start to ask your opinion, and that’s a really big, big difference. And so when we’re starting to share information, there’s a respect that comes across the skills gap. There’s a lot of things that go across that that, again, there’s separate gaps, but hopefully, the long term, and as Carmen said, not easy to fix, is by overcoming some of the skills gaps as well.
Shimel: Fair enough. I think we’ve covered that one well. Let’s move on to another area if we can. We have 20 minutes or so. You know, one of the things that really does my heart good is this concept of DevOps everywhere as far as the eye can see, like the meme with Buzz and Woody. We have seen DevOps move from just Dev and Ops to QA, DevSecOps exploding on the scene where, you know, a couple days, the 28th, a day or two after DOES London, John Willis and team are rolling out our second DevSecOps Days, which I’m involved in, and devsecopsdays.com for those interested. But even beyond IT, we’re seeing what I call DevBizOps, where DevOps’ Agile sort of practices are starting to take hold in HR, in sales, in finance, in, you know, accounting, etc.
And let’s be fair. This is not unique. The fact of the matter is a lot of the lean terminology, a lot of the lean stuff that was built into DevOps didn’t come from IT, it came from manufacturing. And so, you know, it’s like pay it forward. Now we’re seeing these kinds of DevOps principles move into other areas of the business. First of all, is that a good thing? I think it is. I can’t imagine it’s not, but I’ll defer to the both of you. And then, secondly, you know, hey, at the end of the day, remember, it’s all about the culture, stupid. And so what human factors, cultural factors should we be thinking of when we talk about a broader DevBizOps opportunity? And, Jayne, you’re on my screen, so I’m gonna tell you to go first on this one if you can.
Groll: You know, it’s business transformation. You know, it’s not really DevOps transformation. Digital transformation is not limited to IT. It’s really business transformation. And so it goes back to, you know, we talked about a little while ago, where, you know, every organization has to recognize that technology is their business, and so some of the practices from the business start to assimilate into IT and beyond. I saw this in service management where there’s now movement towards enterprise service management, where you’re gonna make changes, you do it the same way.
I think when we start to look … I mean, I can tell you from the content perspective, you know, we use tools in content development, even though it’s not software development, that are very, very similar to that. So I think it’s fantastic. I was talking to a gentleman from an organization on the Netherlands recently, and they actually took the Spotify squad model and applied it to their entire business, and not a small company, by the way. And so they applied it to their entire business where their business is now broken into squads. And, you know, some think that that’s only for a tech company, and by the way, this is not a tech company. It’s a large financial organization.
So I think that’s great. I do think there’s some … I think there’s good news, bad news. I think there’s a little bit of risk that, you know, with DevOps and some of DevOps practice is still emerging that it becomes, “This week, we do it this way, and this week, we do it another way.” And, you know, “Oh, that didn’t work because …” You know, fill in the blanks, but I do think that kind of having a one business and getting away from this we love our frameworks and IT, and every year, we have a new framework that supposedly supersedes the other framework, you know, that’s a maturity issue. And I think if the business embraces it … you’re right, Alan. Lean Six Sigma. That was a business initiative that a lot of organizations underwent that weren’t necessarily manufacturing organizations. So, you know, as we get closer to that, I think the unification becomes stronger.
Shimel: Great. Carmen, how about you?
DeArdo: Yeah. So I think of things like … It is a great thing. So I think of things like lean concepts. We have leaders across the company that we have gone through lean education and lean concepts. So when you talk about value streams, when you talk about lead time, when you talk about wait states, people understand those concepts, whether they be on business or IT. I mean, this whole idea that Dominica, I’m sure, would have talked about, about making work visible, Kanbans. I mean, we actually had an explosion of people who wanted to use some of our tools that initially were brought in for work out of management because they wanted … you know, it’s like, “Gee, who knew, you know, this Kanban stuff would work for us and WIP.” WIP limits. And so people certainly want to apply those kind of concepts.
So I do agree that, you know, the basic concepts, you know, really can spread everywhere, and the whole concepts of even the things, like we talked about it, the three ways. The three ways don’t just apply to IT. The three ways apply to anything that you’re trying to do and get better at. I think when it comes to information sometimes, though, you know, it’s kind of like the robot story and the goal. We have lots of KPIs, but are we actually focused on the goal? So the goal is not around keeping this person utilized or that team utilized. The goal is about, can I get faster from one end of my value stream to the other? And do we have data that shows flow? And that’s why I’m excited with some of the things that I’ve been … you know, that Mik’s talking about. You know, he has a book coming out that is gonna show actual flow of work, because up until now, we have not been able to actually see work flow all the way through that delivery pipeline. And so we guess a lot of times. We think, “Oh, well, if we just add more developers. Oh, we’ll add more testers.”
Well, how do we know? We don’t really know, because we’re making maybe educated guesses, but we’re really not all focused on understanding where the work is and those concepts that can shine a light on it. So I think it all does come back to a lot of those concepts that, growing up in Bell Labs, I was lucky enough to experience with Deming, even going back to Shewhart, around doing those experiments, getting better, seeing the entire end-to-end value stream, but these concepts apply. I mean, “The Phoenix Project” is really not a book about IT even though I teach part of it. It’s a book about how do you get better at doing whatever business you’re in and trying to, you know, stay in business and improve what you’re doing for your customer. So I think those concepts will continue to go, and I think, again, as we’ve talked about today, there’s a thread here of everybody needs to be on board with these concepts: legal, compliance, the business, everybody, because they all play a role in allowing you to do a better job, delivering more quickly to your customers.
Shimel: Fair enough. Guys, let me … You know, Carmen, you hit on some great things there. The Kanban board or even here at DevOps.com, frankly, we run our entire customer engagement, not practice, but now into farming, on Trello cards, which is sort of an electronic Kanban. And it’s a great project manager. I’ll give you another thing. You know, there’s the cultural aspect of this of taking that sort of DevOps culture and expanding it throughout the organization, but there’s also the bits and bytes and tools aspect of it. Kanban’s a great example. I’ll throw another one out at you. Get up. Why does it have to be source code for programs? Why can’t it be source code for contracts or any kind of digital document or, you know, things that we work on?
So I think we’re just starting to scratch the surface on how we can take that DevOps can do or Kanban and, you know, move it out across the entire organization. And we’ll all … I think we’ll have better organizations as a result, but let me now turn it back around. How can we take some of the business culture positives, some of the things that have worked in other parts of our business, and reflect it back into our DevOps and IT processes? How can we enhance our DevOps by adopting broader business principles and practices that seem to work? Carmen, I’ll let you take first crack at that.
DeArdo: So I think, actually, a lot of our test and learn activities started with the business who had actually hypotheses around, “Well, if we offered a policy in a different way or if we made a tweak to something, would we be able to sell more of those things?” So we had people who … and there’s a lot of statistical rigor that you have to bring to that. So we actually had people on the business side who had these ideas about, “I have a hypothesis. You know, is there a way you can allow me to test that hypothesis to see if it, in fact, will yield some positive result in terms of actual policies being sold?” So it was a very scientific-type experiment that they actually wanted to run, and at the time, we really didn’t have a way to support that from an IT perspective. And it kind of forced us to do some things to support, you know, how could we actually get this out and offer it to only a subset of people, which we now might call canary launch or some other terms that are DevOp-ish terms, but really nobody was reading a DevOps book at this point. They were just trying to figure out, “How can we support the business’s need here to try to adapt more quickly and have a high plan around?”
And that started, I think, some of the things that actually were successful that then led into once we started to get into DevOps, like, “Okay, what we actually did there was DevOps.” We didn’t call it DevOps, but those are principles that really we need to support not just in this one’s case, but for all kinds of business hypotheses that we want to get out and turn around real quickly. And doing that really does bring everybody together, because it’s exciting. It’s exciting if you can sit at a business and you can think up, “I have ideas for this product or that capability. You know, how can we more quickly try to see if that would be a good idea?” rather than writing a 12-month business case and trying to do, you know, a protracted study.
So the whole idea of experiments and gain things out quickly, I mean, everybody will resonate and get behind that if you talk about it in that way. If you start talking about Puppet and Chef and things like that, that’s not gonna cut it. But if you talk about it in terms of business and what it can mean for the business, then that resonates and gets a lot of people excited.
Shimel: Excellent. Jayne, how about you?
Groll: I mean, I think Carmen is right on point. And, Carmen, you said something I think that was … you’ve said a lot of very insightful things, but I think in particular that the experiment came down to, “How can we sell more policies? How can we make our business more successful?” And I think if you look at kind of the past in IT, sometimes, we didn’t know how we were contributing to the bottom line. So we were introducing a lot of technology, and we were doing all this cool stuff. But how did that convert into an increase in revenue, an increase in market share, increase in competitive advantage, things like that? And so I think that this unification with the business becomes more significant when IT sees themselves as an enabler of the goal. The goal of every business is to make money. And so sees themselves, and it goes back to information and communication as well, that they understand that what they’re doing is actually helping to sell more policies or increase market share or stand above the herd or anything along those lines.
Now, again, if you’re the CIO, you probably know that. If you’re in senior management, you probably know that. If you’re a developer and you’re one of a few hundred developers, you may not know that. And so I think that it is that kind of bottom-up top-down approach where understanding that what you do every day is contributing in a very, very substantial and tangible way. And then I think the other part of it, and, Carmen, I think you said it really well, is encouraging those experiments where the business and IT work together. And maybe it is that 1 in 100 coder that’s working as part of that team to say, “You know what? I have an idea.” Everything starts with a great idea. It could be from the business. Hopefully, from the business, that says, “I have an idea. Let’s see if we can, you know, bring that to life and what it will do to our bottom line.” And then there’s really a great collaboration between IT and the business.
Shimel: Agreed. Agreed. So, guys, we’re coming up to the top of the hour here, and I guess we need to wrap up. Otherwise, we’ll just be on air all day, and I know Jayne has a show in Vegas and so forth, and Carmen has stuff to do. But first of all, Carmen DeArdo, Jayne Groll, thanks for being our guests here on this hangout as we, you know, continue to build up to DevOps Enterprise London, DevOps Enterprise Summit London. Just a reminder for anyone watching, it is June 25th and 26th in London. There are still some tickets left, I think. I don’t know how many. I wouldn’t wait. You can get updates about the event as well as get tickets at events.itrevolution.com. It’s obviously on the web.
By the way, this year, as in past DevOps summits, DevOps.com is a media sponsor. We’re partners. We’ve partnered with Gene from the beginning. We will be doing videos again this year at DevOps Enterprise Summit. I think it’s gonna be a little different this year. We’re doing some interesting roundtables. You’ll see them on our YouTube channel, DevOps TV. So stay tuned for that. All right. Guys and lady, or guy and lady, I had a great time today. I’ve learned a whole bunch. I hope our folks listening and watching this have learned as well. You know, you guys have done, not just today, but, Carmen and Jayne, you’ve been instrumental in helping to bring the gospel of DevOps to the world, if you may, or at least maybe not the gospel, but at least some really good learning and understanding.
So thanks again, guys, for being with us. We’re gonna call it a wrap. Carmen, any last word? Three words.
DeArdo: Three words. Project to product.
Shimel: Okay. Jayne? Last words from you.
Groll: Pittsburgh Steelers. I don’t know.
Shimel: We finally got her. Carmen, we converted her.
DeArdo: I should have said, “Let’s go Tens.” That’s three words.
Shimel: Okay. Hey, everyone. I will see you both in London June 25th and 26th. Hope to see many of you listening or watching this in London as well. If you’re there, stop by, say hello. I always like to meet people who have watched us online. But until then, this is Alan Shimel for DevOps.com and IT Revolution. Thanks for watching, everyone, and hope to see you soon. Bye-bye.
The downfall of the all-in-one IT management suite and the rise of the customized IT Ops toolchain
Here is how enterprise IT Ops used to work: you would make your choice among the Big Four vendors and buy in (quite literally!) to their strategy. This meant a suite of products that came preintegrated (more or less) out of the box, and—at least on the sales proposal—covered all of an IT organization’s needs across a huge spectrum of activities.
Photo by Barn Images on Unsplash
Broadly speaking, each of the Big Four’s suites included some flavor of the following: monitoring, ticketing, configuration management and asset tracking and automation and orchestration. Some vendors were stronger in one area than in another, but unless you had pretty specific needs, each vendor could be your one-stop shop.
Development and improvement inside these suites was mainly incremental and occurred at a fairly leisurely pace. Meanwhile, startups with bright ideas would come along occasionally and challenge the incumbents in specific areas, only to be bought by one of the mega-vendors if they showed enough promise.
There was a symbiotic relationship between vendor and IT buyer, as IT professionals built decades-long careers as experts in this or that vendor’s technology stack, with the wall of framed certifications to prove it. CIOs were reassured by the fact that there was a readily available pool of experts to hire, and continued reinvesting in the next iteration of their chosen vendor’s stack.
The Old Model No Longer Fits Modern IT
In the last decade or so, this comfortable status quo has broken down. The cracks were already showing before then, but only the most agile and forward-thinking organizations were moving fast enough to trip over them. As more and more businesses began to expect greater and greater responsiveness and speed from their IT departments, the huge sprawling suites of the mega-vendors were no longer able to keep up with the required rate of change. Leisurely 18-month release cycles no longer cut it in an agile world, where that time span may encompass several entire strategies.
In the old world, to call something “best of breed” was a sniffy dismissal of a one-trick pony, as opposed to the more supposedly versatile offerings of the Big Four. In the new world, it became obvious that aside from a handful of tentpole products that were genuinely good and fit for purpose, most of the individual components of the big suites was distinctly unexciting. Worse, the main value proposition of the suites—that they came preintegrated out of the box—started to break down as well. Many IT professionals were still fiercely loyal to individual products, but they became more willing to pick and choose, rather than buying into the whole vision.
In parallel, the rise of simple integrations—primarily REST APIs—made it easier than ever before to connect disparate solutions together into a single IT Ops toolchain. Worse (from the point of view of the incumbent vendors), sometimes it was easier to connect modern point solutions than their own component pieces, weighted down by the baggage of decades on the market.
How to Deliver the Flexibility and Agility the Business Expects?
These days, it is becoming commonly accepted even among the most staid and risk-averse organizations that the way to go is no longer to align to a single IT vendor and trust them to deliver on their vision, but rather to design custom-fit, multisolution toolchains, including home-grown, open source and commercial tools.
Photo by Max Bender on Unsplash
But what to do if you find yourself dealing with the consequences of this sort of IT monoculture?
Look Back At How You Got Here
The first step is to re-evaluate past buying decisions and strategic alignment. It is very important to do this without assigning blame; it was a different time, and those choices may well have been the right ones for the prevailing environment at the time, when integration between tools from different vendors was legitimately difficult.
Don’t Just Throw Everything Out
In many environments, too, the correct solution may well not be a clean sweep. One or more of those tentpole products that kept the overall suite looking good past its sell-by date may well still be perfectly fit for purpose and not worth the hassle of swapping out. However, other components are probably legitimately lagging behind the state of the art, in which case it is worth replacing them with something more fit for purpose.
Count Costs – All Costs
Include cost in these calculations, but remember to include all costs. Maybe you or a predecessor signed a big enterprise license agreement (ELA), so Product X is “free” in that context. But what does it cost to keep Product X running, in terms of ancillary licenses, hardware requirements and, last but not least, skill and effort on the part of the IT administrators? I don’t want to name specific products or vendors as examples, to avoid the discussion devolving into particulars, but it’s worth bearing in mind that many legacy products have shallow talent pools—not that many people are available on the market, and many of those people are coming to the ends of their careers, which makes them both expensive and reluctant to change employers. Instead, up-and-coming technologies often have much deeper and wider talent pools, with many more experienced specialists available at all salary and skill levels. If you don’t believe me, try hiring half a dozen COBOL programmers and half a dozen Node.js coders, and then we’ll talk …
Photo by Giu Vicente on Unsplash
Your IT Ops Toolchain, For Your IT Ops Needs
Moving from one of the big all-in-one suites to an assemblage of different tools, all flying in loose formation, is not easily done. Any technology transition of this sort is a Project with a capital P, and needs to be planned and managed with due care and attention. However, there is a large payoff at the end in terms of increased agility and flexibility within IT Ops. In turn, this affords IT the ability to offer their business counterparts the sort of responsive and even proactive IT support that moves IT from being a cost center, on a par with Facilities, to a strategic differentiator that enables new business initiatives.
Business as usual doesn’t cut it any more, and assumptions that used to be valid have fallen apart. Decisions based on those assumptions need to be re-evaluated. New, more flexible approaches will set up IT Ops for success in the newly flexible world we are already living in.
Most organizations are not fully compliant with the European Union’s (EU) General Data Protection Regulation (GDPR) before it takes effect in May, analysts report.
While DevOps certainly gives organizations the mean to turbo charge the compliance process, stakeholders outside of IT departments often are failing to greenlight this necessary project ahead of the deadline, due largely to uncertainties about the regulation.
Not being in compliance could hit organizations hard, with fines up to 4 percent of their annual revenues or €20 million.
The Cloud Security Alliance (CSA) and Netskope, for example, recently released the results of a survey of more than 1,000 respondents about their organizations’ GPDR readiness. The survey revealed a full 83 percent were not very prepared for GDPR. While 85 percent had taken steps for compliancy, only 31 percent had well-defined plans for meeting the provisions of GDPR.
The GDPR’s “right to erasure,” (53 percent) “data protection by design and by default,” (42 percent), and “records of processing activities” (39 percent) were deemed the biggest stumbling blocks for compliancy, according to the study.
A separate survey of 700 network security and IT professionals by Enterprise Strategy Group (ESG) showed only 11 percent of firms were completely ready for GDPR ahead of the May 25 deadline. Thirty-three percent said they were prepared to publicly disclose data breaches within the GDPR-mandated 72-hour period, and 33 percent and 44 percent percent of the firms surveyed were mostly or somewhat prepared, respectively.
Most organizations ESG surveyed also reported they are investing in GDPR, reflecting the seriousness in which they are regarding the mandate. While only 10 percent said they had made or were planning to make major investments in GDPR, 63 percent said they would make or had made incremental investments.
Analyst firm Varonis made similar findings after completing a survey of 500 IT security experts in the United States and Europe in October, out of which 200 were in the United States and the rest of those surveyed were equally split between the UK, France and Germany.
Varonis also found 60 percent in the EU and 50 percent in the United States faced serious challenges before they could be GDPR-compliant. However, 70 percent of the firms represented were also convinced GDPR compliancy would give them a competitive edge.
IM000302.JPG
Where Is the Data?
Much of the uncertainty about GDPR reflects how organizations often fail to understand where exactly their data is located. While not specific to GDPR, Varonis found many organizations lacked fundamental data-protection protocols. Those surveyed revealed, for example, 21 percent of their organizations’ electronic folders were “open to everyone.” The survey also revealed 54 percent of the firms’ data was stale. Effectively tracking data extending from on-premises servers to across the cloud was often cited as an issue, making it difficult to track the exact origins of customer data and to determine whether it might be subject to GDPR protection or not.
“We live in a hybrid world in which data isn’t stored entirely on-premises or completely in the cloud. The hybrid model creates additional challenges and potential risk from a security and data governance standpoint,” Ken Spinner, vice president of global field engineering at Varonis, said. “You can’t assume your data is safe in the cloud. If your cloud environment isn’t secure, your data won’t just be in danger of being exposed to your entire organization—it could be accessible to the world.”
Uncertainties Abound
A lack of understanding about what GDPR compliance involves is often cited an impediment to making necessary changes for compliancy. Much has been reported, for example, about whether EU citizens, residents, or simply those who are only temporarily on European soil are protected by GDPR. How the EU will impose fines against non-EU organizations also remains murky. According to ESG data, less than one-third of the respondents of the organizations that took part in its survey fully understand all the GDPR’s requirements.
These questions, of course, largely need to be resolved by legal counsel. Meanwhile, many legal departments are waiting for more guidance from the EU about uncertainties mentioned above, as well as other remaining questions about the GDPR’s scope.
In other words, DevOps is ready to do the necessary, while legal and other stakeholders who fully understand what needs to be done must pull the trigger first. Only then can DevOps, consisting of IT operations, QA, InfoSec and development teams take over. When that happens, DevOps’ role will remain critical for making compliance more agile, automated and, ultimately, reliable.
“DevOps can assist companies in complying with regulations like GDPR by ensuring that key regulatory requirements are part of the initial design criteria, and taken into account when applications are updated,” Varonis’ Spinner said. “GDPR, manual methods of enforcing data policy aren’t realistic solutions because large organizations could have millions of files. DevOps can support GDPR requests with processes and solutions that ensure personal data that falls under the new regulations is protected and used appropriately.”
“As with any new technology, it’s really important that we be thinking now about how to do [it] ethically and responsibly.” —Robert High, Vice President and Chief Technology Officer, IBM Watson
Here’s something to think about: According to an article in The Wall Street Journal, more than 30 countries have defensive weapons run by automation. South Korea has robotic sentries along the DMZ, while Israeli robots patrol the border at Gaza. Russia has robotic vehicles armed with 30mm cannons and anti-tank guided missiles that are well-equipped to battle it out with NATO forces should the need arise.
Got your attention? Wait, there’s more.
The SeaHunter is a U.S. military vessel that can travel the high seas on its own for months without a crew to feed or protect. It’s completely unmanned. And, it costs only $20 million per boat, less than half of the $49 million it costs to fire 59 Tomahawk missiles. At that price, the SeaHunter is practically throwaway.
The SeaHunter, an unmanned oceangoing vessel, can remain at sea for months at a time.
The implications are profound. The value proposition is hard to resist, Why spend billions on building and manning traditional warships when, for the same expense or less, you can have a navy of predominantly robotic ships patrolling the oceans of the world? And, you can have robotic aircraft, capable of automated takeoff and landing, using these ships as mobile bases. An air force untethered to any land mass is a game changer, particularly when non-state actors can get their hands on a few of them. It sort of brings new meaning to the word “pirate”—no need to board a vessel by force, just get the access credentials.
And, at the core of it all this modern weaponry is automation and growing capabilities in AI. That should give you something to think about.
We in DevOps are all about automation. It’s our raison d’être. Our fondness for, and dedication to, automation is admirable. We really have changed the way tech works. Yet, many of us think of our activities as mundane. It’s akin to the nonchalance of a Major League baseball player. They’ve hit the ball so much that many players have forgotten how skillful one needs to be just to foul off a bad pitch. Most people on the planet can’t hit a Big League pitch, let alone make it go foul by intention. But for the pro, it’s no big deal—it’s just part of a day’s work.
It’s the same for us. We’re so accustomed to automating our world that it’s become no big deal. But, it is a big deal. It’s a very big deal.
The easiest thing in the world is to trivialize the incredible impact our work has on others. After all, we’re “just working on” an ecommerce site, a dating app, or IoT endpoint for wired bicycles, right? Those of us working in the defense industry might grasp the significance of the work, but for most of us, it’s business as usual.
Here’s the raw truth: Those of us doing the automation are changing the world. Each line of code we write is intelligence that alters the digital infrastructure of some part of the planet. The code we write will live on well after we’re gone, whether it’s from a company or from the planet. Are we really clear about the impact we have? Yes, some of our tech superstars are asking us to pay attention, but how many of us really are?
Every day that passes we give automation more power. It might be a little piece of power, such as letting only authorized users access to an application. It might be significant power, such as determining whether a person qualifies for a loan to buy a house. It might be enormous power, such as manipulating political forces toward a desired behavior. Or, it might be the ultimate power: Deciding who gets to live and who gets to die. But, regardless of degree, it is power and we have it. And, we continue to make automation more powerful. As surprising as it may sound, despite appearances otherwise, we’re the ones the world trusts intrinsically to do our work safely and responsibly. A lot of the world’s safety depends on how we do our work. Political leaders don’t write the code. We do.
Originally, I was going to call this piece “Death and Destruction in the Age of Automation” and write about how we’ve taken automation way beyond commercial IT into the realm of life-and-death decision-making. I decided otherwise. The world does not need another gloom-and-doom piece, nor does it need another piece of sensational journalism. We’d do better to focus more on the positive things we’ve achieved. Technology is doing a whole lot more good than bad.
But the fact remains that technology has accompanied death and destruction since Roman catapults assaulted the frontiers of ancient Europe. The thing that’s really different now is that the technology thinks—or, at least, it’s doing a really good job of emulating thinking. And, we’re the ones making it think better. Thus, we should ask, Are we making technology think in ways that are safe and trustworthy? Are we acting in ways that are safe and trustworthy?
These are big questions. Do most of us really care about them? Or, is it easier to accept our work as mundane, with little consequence in the Big Picture? The nice thing about mundane is that it doesn’t require a lot of ethical contemplation. But, when you consider that the script that deploys an updated container to a Kubernetes cluster is not that far from the one that deploys an unmanned warship to a regional hotspot, things look different. And, contemplating the implications thereof becomes more than an academic exercise in a computer science ethics class.
More than 2,000 years ago the philosopher Socrates said, “The unexamined life is not worth living.” Given the immense role that technology plays in today’s world and the power it wields—power that continues to move further away from our direct control—we might do well to say, “The unexamined technology is not worth doing.” It’s something to think about, while we still can.
Here I am at the Fontainebleau Miami Beach for Appian World 2018, where the theme is “Transform.” The event, which is the biggest one yet with 1,500 registered attendees, is all about low code, and in his keynote address, CEO Matt Calkins emphasized that low code app-building isn’t for unsophisticated, simple apps—we don’t need powerful tools such as Appian for those. Rather, low code is for sophisticated apps to set free the renaissance that will happen with easier app-building.
This is the first Appian World since the company went public last year, and the event has a public-company feel. I didn’t see a lot of people in jeans and t-shirts—most attendees were fairly well-dressed in business-casual, the mark of enterprises.
One of the touch points in Calkins’ keynote was reliance on open standards. Another was to make use of artificial intelligence (AI). I know AI and machine learning (ML) seem to be everywhere. It was interesting the company announced AI integrations with Amazon, Microsoft and Google, but no Apple or IBM Watson. I will ask Calkins about this when I interview him for DevOps TV later today.
Calkins inherently believes that low code will set off a revolution as software continues to disrupt and empower the digital transformation revolution. The logic behind it is simple: “Making applications easier to make because you don’t have to code as much.” But Calkins emphasizes that does not mean you are making unsophisticated applications. The world doesn’t need low-code tools to make unsophisticated applications. It is the higher calling for sophisticated apps that Appian is all about.
Another mantra from the keynote was, “Host anywhere, run anywhere, data anywhere.” The idea here is a true hybrid reality where data can be on-prem or apps hosted on public cloud, on containers or not. In fact, Calkins was very bullish on containers (who isn’t?). Docker integration is already available and he hinted at announcements regarding Kubernetes as well.
Appian seems to have succeeded by becoming the low-code platform. With lots of integration and focus on making application development, management and deployment easier, the company has risen to the top of its market.
Stay tuned for more updates from Appian World today and tomorrow.
