Mindless testing, whether it is manually or automatically executed, can cover only a very narrow scope of possible bugs: the ones you’ve already thought of. Exploratory testing however, endeavors to think outside that scope. Thoughtful testers look at a problem from all angles, weighing factors such as feature usage, psychology, user stories and more. Their goal is to chart the uncharted territory of an application.
The faster our development cycles become, the more thoughtful our testing should be. It’s only through techniques such as exploratory testing that we’ll be able to get there.
Continuous testing is the process of executing automated tests to obtain rapid feedback on the business risks associated with a software release. Where does that leave exploratory testing? It’s not automated, but it’s certainly critical for determining whether a release candidate has an acceptable level of risk.
Test automation is perfect for repeatedly checking whether incremental application changes break your existing functionality. However, where test automation falls short is in helping you determine whether new functionality truly meets expectations. Does it address the business needs behind the user story? Does it do so in a way that’s easy to use, resource-efficient, reliable and consistent with the rest of your application?
Exploratory testing promotes the creative, critical testing required to answer these questions. Obviously, it doesn’t make sense to repeat the same exploratory tests continuously across and beyond a sprint, but exploratory testing can be a continuous part of each delivery cycle.
Exploratory Testing in Action
Here are a few ways teams embed exploratory testing throughout their process.
Perform Ad Hoc Exploratory Testing as Each User Story is Implemented
This is the exploratory testing equivalent of peer code review. When a developer completes a user story, they sit down with a tester. The tester starts testing while providing a running commentary on what they are doing and why. Next, the developer takes control, explaining how they would test the software given their knowledge of the implementation details and challenges. The developer gains a user- and business-focused perspective of the functionality, and the tester learns about the inherent technical risks.
Another tactic is to have the developer and a tester separately test the same feature simultaneously, then discuss their findings at the end of the session. Often, this turns testing into a competition, where each participant tries to uncover the most or “best” issues in the allotted time.
Align Exploratory Testing Sessions with Full Regression Testing
It’s simply not possible to perform exploratory testing or full regression testing on every code commit. That’s what smoke testing is for. Instead, many teams run full regression testing and session-based exploratory testing in parallel a few times per week, whenever they’ve implemented new functionality that a user could feasibly exercise.
For optimal results, these sessions should be lightly planned and tightly timeboxed, include diverse perspectives and really take the six thinking hats theory seriously.
Host Blitz Exploratory Sessions for Critical Functionality
The best way to uncover user experience issues before users do is to get a broad array of feedback prior to release. One way is to host “blitz” exploratory testing sessions. When you’re wrapping up work on critical new functionality, invite people from a variety of backgrounds and teams to participate in a short timeboxed session. Incentives can help drive participation, maximize results and make testing fun.
Using test automation to continuously check the integrity of existing functionality is certainly critical. However, if you’re not also making exploratory testing a continuous part of your process, how will you know if the new functionality meets expectations?
The goal of continuous testing is to understand whether a release candidate has an acceptable level of risk. Exploratory testing is perfectly suited for helping you answer that critical question.
As the cycle of DevOps is repeated in a growing list of organizations, I feel compelled to reiterate the warning that this is not the pinnacle. Anyone who has been around for a while knows that the current fad is not the last fad.
[[Cue DevOps people screaming at my choice of the word “fad”]]
Of course it’s a fad. DevOps (and to a much lesser extent, agile) is the cool toy in the midst of Gartners’ (in)famous hype cycle. That doesn’t mean nothing will come of it; it means that people freak out when you tell them it doesn’t solve every problem.
All of the process/solution fads that have come along have offered some benefit to IT. DevOps promises to offer more, and in many environments it does. But grabbing your broadest brush and screaming, “DEEEEVVVVVOOOOOPPPPPSSSS!” while painting everything in sight is not problem-solving; it is thinking there is only one path to quality and it flows through the Chosen Process.
The reason DevOps is different from previous hype cycles is obvious: It has the built-in assumption that this is not the end state. It assumes you will keep moving forward and things will change. Previous “new way of doing things” ideas have assumed that there was some nirvana you would get to and everyone would relax there, sipping ambrosia and enjoying immortality. DevOps makes no such claim. About as close to a final state as it gets is increased communications between the various parts of the DevOps life cycle. And that is because there is only so far you can go with increased communications before you’re flooded. And even that leaves room for new tools such as Slack to augment communications.
But don’t look at every problem with DevOps blinders. This is the real world, and while some will (possibly correctly) claim that every software dev/deploy cycle can benefit from DevOps, the real question is not “Can it benefit?” but rather, “Is the cost/investment/risk less than those benefits?” And to the chagrin of those who live and breathe DevOps/agile, I’m afraid the answer to that question isn’t always a resounding “Yes.”
I’m undeniably a fan of both agile and DevOps. What I’m not a fan of is finding the nearest round hole and cramming everything—including square pegs—into it. Look at each project/process objectively. As a simple and blog-length example, let’s examine standalone app that does some important back-end process and is only updated once or twice a year: Why ever would you rip the process for that apart, invest man-hours to bring it into the DevOps and agile folds, then … only update it once or twice a year?
If you are building a new project, and you have DevOps processes in the building, it would take a convincing argument to not use them. The same should be true the opposite direction. If there is an existing project that doesn’t have delivery delays, it should take a convincing argument to move it to a whole new development/deployment methodology. Choose wisely. Invest resources where they’re required, not where ever the broad brush falls. IT has enough work to do, don’t add to it, just so you can say “We’re a DevOps shop.” Better to say “We’re a right tool for the job shop.”
And keep kicking it. Business doesn’t run itself, IT runs business. Keep making it work.
Abraham Maslow proposed in 1943 that healthy human beings have a number of needs arranged in a hierarchy, which he constructed in the form of a pyramid. The theory suggests that certain needs, such as physiological and safety needs, are more primitive or basic than social or ego needs. The premise is that higher-level needs are addressed only once lower, more basic needs have been met.
By FireflySixtySeven – Own work using Inkscape, based on Maslow’s paper, A Theory of Human Motivation., CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=36551248
A similar path can be uncovered when we map out the evolution of technologies. As technologies fulfill their intended roles, developers pursue greater achievements and solutions to more complex problems. It’s a pattern that exists at every new stage of technological growth as we are driven to satisfy loftier motivations. It’s also happening now in organizational structures as our work becomes more tied to the capabilities of technology.
This progression deserves particular examination in the role that DevOps strategy plays as organizations grow and mature. DevOps encourages a mindset of communication, collaboration, integration and automation among software developers and IT operations, but it can also be swept up by a motivation to go beyond the scope of its basic needs and seek constant betterment. Through this lens, it’s helpful to look at how DevOps culture strives to keep up with the need for a more holistic approach to the end-to-end software delivery life cycle (SDLC), and how it will continue to mature as needs change and evolve.
Physiological Needs
Maslow begins at the base of the pyramid with basic needs. For humans, these needs are physiological, the physical requirements for survival: air, water, food, shelter, etc. To make the analogy in technology, this is the realm of manual processes, where operators flick a switch, crank a handle or spin a wheel. It’s the basic functions of machinery.
The base of the DevOps pyramid is the notion that I have to use software at all, much less manipulate it to fit my business objectives. Not until that basic need is recognized can further development begin to take shape. Initially, Dev teams built applications and passed them to QA teams to test them. The production environment in these early stages got the job done, but at a basic level it lacked cohesion. Teams could harbor mistrust and production environments could become destabilized by poor communication or diverging points of view. Like a simple machine, tasks were performed separately and passed from team to team only after the basics were completed.
Recognizing the need for software was the first step, in fact it’s a basic requirement for most of today’s businesses and organizations. To make progress and mature an organization must take the next step toward addressing more complex problems and make choices that promote structural growth and organizational progress.
Safety Needs
Moving up Maslow’s pyramid, human motivation addresses concerns for safety and well-being as choices of preference are made and a degree of order is sought. Organizationally speaking, burgeoning DevOps teams seek greater usefulness and functionality. It’s not enough for teams to perform a basic function, they need to adapt and find a way to get beyond a lack of cohesion. Organizations need a level of comfort (compared to Maslow’s concept of safety) in knowing that the development and operations teams can work together and have the best intentions for the project. This stage promotes an upgraded approach to software delivery and a reduction in production issues.
Once the realization that software is a basic need for companies and organizations, their sights turn to how best to make it happen across multiple teams with a common goal. And, if a level of internal stability and order can be achieved, the organization can begin to look beyond itself to develop outside relationships in the manner that Maslow describes in the third layer of his pyramid: belonging.
Social Belonging
In the third stage of Maslow’s pyramid, after the physiological and safety needs are met, human motivation shifts to interpersonal relationships and feelings of belonging. In terms of an organization that services customers or constituencies, once internal structures are in place, greater resources can be allocated to outward pursuits. This stage in the business mindset is when the focus on outcomes shifts from being vendor-focused and satisfying internal needs to the servicing the customer. They ask the question: What answers or solutions does the customer want from me? Just like in Maslow’s acceptance stage when humans consider what it takes to be a friend or to find love, organizations try to answer what it takes to provide customers what they need.
In this stage, the automation of DevOps contributes as teams integrate into a structured or organized community that can provide those answers. Teams begin working together and develop the early stages of a feedback loop and full visibility into the SDLC. The workflow takes on an increasingly automated role as a cultural shift toward full lifecycle ownership occurs and teams take responsibility throughout the deployment life cycle, not just in stages.
Esteem
If organizations succeed in putting customers first, they can take the next step to reacting to the customer’s needs at an accelerated pace. In Maslow’s pyramid, the next stage, commonly referred to as the esteem or status level, is divided into two elements: the internal and the external.
Maslow classified these two categories as: (1) esteem for oneself through dignity, achievement, mastery and independence, and (2) the desire for reputation or respect from others. To achieve everything that organizations want to for the customer, internally they must improve how the organization functions, and externally they must demonstrate to the customer that they are listening and the goal is to respond to the customer’s needs directly.
Internally, the organization requires its teams to communicate, collaborate and integrate better, and it puts pressure on its production activity to be responsive not just prolific. If this is achieved, it not only demonstrates that an organization is listening to the customer, but also that it can provide solutions at a cadence that makes the customer more successful.
Self-actualization
The final stage of Maslow’s hierarchy is self-actualization: the point of realizing potential and being the most that one can be. In any organizational structure, this may not be an end goal, because striving to be better should always be a core tenet. However, it might be argued that when the brain trust at the top—the CEO, the CIO, the board, etc.—are fully aware that the components of their organizations are working together and of what they are achieving, a level of corporate-actualization can be claimed.
Far from being the end, this is when thoughtful soul-searching should occur by asking the essential question: Is it all working? This is the most important question an organization can ask itself with regularity, and doing so is a sign of great maturity.
You can recognize your basic needs; build structure into your organization; service your customers, listen to their needs, respond promptly and make them more successful; and recognize internally what make your organization successful. But if you don’t consistently and rigorously ask if it’s all working, you risk slipping backward.
Once an organization reaches a mature level, how does it maintain or even enhance its hard-earned growth? The best thing it can do is to consistently define itself with the use of qualified actualization tools and reporting engines. True self-actualization and corporate-actualization require tools that can quantify how mature an organization is and where it stands by comparing how it’s doing against its competitors. Organizations need tools that can correlate data against a variety of peers in the vertical or in common geography, etc., and provide comfort in knowing what works and that they are leading the way.
Organizational requirements change rapidly. The strategic plan you created in the past and previously only updated annually is now out of date within a quarter.
Agile and DevOps practices have been adopted to help resolve complex business issues in these times of flux through technology improvement, providing organizations with an increased certainty of intent, quality and safety.
However, integrating the potential impact of Agile or DevOps into business continuity practices isn’t always considered a sound and highly valuable idea. The continuity of an organization is often considered important only after a crisis has occurred—the proverbial closing of the gate after the horse has bolted.
Below are a few ways that business continuity capabilities need to change in light of the impact of Agile and DevOps.
First, a Business Continuity Primer
Maintaining business continuity in your organization:
Provides the ability to adapt as required to meet competitive, regulatory, organizational, technology or new market opportunities.
Provides your stakeholders, suppliers and, most importantly, customers and staff with the assurance the organization will remain a sound, going concern.
For example, many organizations have a data repository in an application such as SAP, Oracle or SharePoint. This repository is saved into the cloud or other media daily or as needed.
That is great. But if you mapped the processes used inside a department, you might find that employees download this data into Excel or some other tool and manipulate it for their own purposes. This is where the real work is being performed, and if this data is lost, the business continuity impact could be quite severe.
DevOps and Business Continuity
Agile and DevOps methodologies suggest that you design, create, test and release your technology changes in rapid cycles of two weeks or less, with any technology change introduced probably having a business process—and thus business continuity—impact.
Therefore, whatever you release must always be ready for production. But how can you provide this level of assurance? This is how:
All “epics”—high-level business needs that are broken down into smaller stories—should also have a simple statement highlighting the impact of request.
All user stories should have tests to confirm the work being performed is as requested but also that they would be usable no matter where needed.
Feedback is provided to the development, infrastructure, cloud provider and product owner regarding the success of the tests.
Any issues need to be resolved before the work/change is released.
Issues and other lessons become part of the backlog of new work.
If a continuity test was not performed, then this user story is prioritized for completion in near future sprints.
DevOps and Business Continuity Maturity Assessment Questions
There’s a relatively simple way to understand how your organization’s business continuity capabilities stack up: Ask a number of focused questions (to the people who should be able to provide suitable answers).
Start by asking:
How often do you test your code as you create it? (“Oh yes, we do that.”)
Do you always test before you go live? (“Oh yes, we do that.”)
How many of you create your code—be it infrastructure virtualization, new applications, internet, etc.—in environments that replicate, as close as possible, the production environment? (“Oh, hmm, no our environments are all different.”)
Step it up with the “killer” questions:
How can you ensure that what you’re about to release is safe?
How can you ensure that the work you’ve done is not a waste?
How do you know that your organization can be kept “in business” and will not be exposed to an adverse effect from the change, or a business continuity situation?
Understand How the Organization Relies on Technology
As mentioned, business continuity is ultimately all about keeping your business in business, and thus there’s a need to understand how technology is employed to maintain:
Regulatory compliance.
Competitive advantage.
A safe place for your people and suppliers to work.
Map the technology used at each step, down to workstation applications.
Quickly assess the impact of that stream being blocked or unavailable by a facilities, people or technology issue.
Create a plan to make that value stream as available as required.
Organizations that have mature business continuity practices will, of course, also use technology to automate these activities. But they will also ensure that changes needed are tested as soon as possible in the product life cycle.
If your tests are not performed in a production-like environment continuously, then how do you know you are ready and safe to go live? Go-live should not just encompass the new code but also the processes, training, security, continuity testing, organizational change and supplier update, if needed.
Fortunately, the DevOps tools that support these capabilities are readily available and thus need to become part of any development, testing framework or infrastructure.
Ultimately, it’s important to sufficiently invest in keeping your business in business. DevOps practices are another element to factor in to your organization’s business continuity management policies, processes and practices.
Safety, security and controls are the foundations upon which 135-year-old ExxonMobil is built. As such, quality has become a key component in delivering software. The company is currently undergoing a shift from traditional project management organizations and support organizations to product teams, which has highlighted the need for testing early in the development cycle and to combine new development with ongoing maintenance.
Testing, the team “in the middle,” has become an integral component of the ExxonMobil development life cycle as “Quality Engineering.”
To do this, ExxonMobil started with the basic idea—quality control manages risk—and then explored what is tracked today and whether quality is being measured. Another key activity was to remove the roadblock to continuous testing.
Ann Lewis from ExxonMobil discusses in this video how the Quality Engineering team has created the expectation that testing is a part of everyone’s mindset, as well as the company’s view that continuous testing and deployment can only be accomplished through model-based testing and that managing quality reduces overall cost of ownership.
The full transcript of her discussion is below:
“At ExxonMobil, we have presence in six continents, so that means we get to deal with a lot of people. There’s over 72,000 employees plus another 20 percent contract staff, and that’s a problem. Guess what they want to do? Manual testing. Yeah, because they’re not gonna give it up. We have been on an adventure this last year that I’m gonna talk about and explain to you how we help train these 72,000 plus people that there’s a better way to do it.
“In addition to the numbers of people that we have, the upstream, I always … Well, someone referred to it about sticking a straw in the ground. Yeah, it was actually somebody from Tricentis. Yeah. I was just like, ‘There’s a little bit more to it than that.’ No, we do produce all of the product from under the ground. We get it out, and then we send it downstream to our refineries so they can do something with it like sell it. We also have a chemical company, that … I think right now are second or third to DuPont. We have a lot of product and a lot of things to do with it. That’s a problem, too. Because when it comes to the applications that test, that need to be tested, we have something like 5,000 applications that run our business all technologies. We’ll talk a little bit about that as we go through, too.
“The title is The Middle Child Rises. Why did I choose this? Because in the world of Dev and Ops, test is in the middle.
“We have a few other friends in the middle, and they are a little bit like a middle child, so we’ll talk about that. For today, your takeaways are that our organizational design has shifted. We’re going to explain why and how that happened. We will explain that Tosca is strategic for ExxonMobil and we will explain why that is.
“Tosca is an enabler to speedy delivery.
“You’ve heard about that this morning from many of our Tricentis friends, and it is true and I’m here to share that story. We now have within ExxonMobil, a quality philosophy, yeah, and it’s a lot more than just testing. Yes indeed, it is. We’re going to take a look at what we did and have a lot of fun stories in between.
“Being the middle child, you really didn’t get as much attention. How many are middle children, by the way? A few, okay. Do you have babies in here? Yeah, yeah, you kind of get away with everything, don’t you? The oldest, I have a favorite, I know. Oh, there’s another one out there of my fave, okay, cool.
“It’s very interesting in family relationships how these children get along. The oldest kind of sets the stage for everybody. That middle one comes along and it’s like, ‘Oh, it’s the second kid.’ The third one comes along and it’s like, ‘Oh, it’s the baby, it’s the baby,’ and the middle child just kind of sneaks through, okay? They don’t get quite as much attention. We’re going to talk about the middle child as they relate to testing.
“Typically, it’s the middle child that ended up with rules. That oldest child was all about, if you just tell them once, they usually would stop doing it. That’s the case I had. Then the second one was born, that turned out to be the baby, but I swear he’s got components of being the baby and the middle child. It was like, ‘Adam, we have rules in this household. Yes, we have rules.’ He was like, ‘I just don’t get the same respect as that older sister, Jessica.’
“Now within the corporate world, these middle children, you can think of them as being security and controls, data migration, data conversion. It’s like, ‘Oh, yeah, we forgot about you.’ Last but not least, testing. How many times has it been when suddenly that project shows up, ‘Oh, yeah, I think I might need to engage your team to help me organize my testing.’ Yes, yes, those of us that live in the testing world.
“The big deal right now, though, is that these middle children, our testing organizations, our quality assurance organizations, are becoming the key enabler, like we heard this morning, 50 percent of our problem is with testing. Oh, I could do that but I can’t get testers. I could do that but, ugh, we’ve got to do the testing. It takes so long.
“According to the World Quality Report, the reliance on manual testing is the top technical challenge in app development.
“How many of you knew about the World Quality Report? Yes? Yes? Good, good, good. We’re all the same kind of people, yeah. I like that.
“Businesses see test automation as a way to address one of their top challenges to delivering better software faster, by adopting Agile, DevOps and continuous delivery practices to shorten delivery cycles.” – Forrester
“By 2020, DevOps initiatives will cause 50% of enterprises to implement continuous testing using frameworks.” – Gartner
“The need to support faster time to market with higher quality is driving the demand for effective functional test automation tools. This is why we’re here. Tricentis Tosca, it is a framework. Yay! It’s not scripting. Boo!
“Tricentis Tosca will address that top technical challenge of delivering software faster.
“The need to support faster time to market with higher quality. It’s higher quality in the form of risk coverage. That is what we do with Tricentis Tosca. That is what we do with Tricentis Tosca.
From a framework perspective, the last year and a half we have been on a mission to explain the difference on automation tools. We have gone from record and replay, to test automation from a framework perspective using scripting, such as with the Hewlett-Packard products and Selenium. It is programming, it takes developer time to do automation when you’re talking about some of these historical tools. We have now moved into model-based, and I hope everyone in here has been to at least some of the Tricentis’ demos on what model-based is. I’m not going to go into detail here, but I will tell you that it has taken our automation from a 40-hour event down to a 8 to 16-hour event. No joke.
“I have been able to take Tricentis Tosca and put it into non-developer, normal people’s hands.
“I actually sat down with one of the Tricentis’ architects and did automation myself. I said, ‘Oh my gosh, I haven’t touched a keyboard to do anything technical in so long,’ and I actually did it. That is key, that is significant. That has been the story that has been going on with ExxonMobil for the past year and a half.
“It has become strategic for us. It’s automated screen scanning, it’s lower maintenance, it’s targeted testing. I’m doing it in half of the time, if not less. It’s faster, it’s easier to support, it’s talking about prevention versus detection. We have created a road map.
“What you’re seeing in our colorful TFS diagrams on the top, yes, we are doing our monitoring in TFS. You’re seeing a migration and better put a re-engineering of our existing scripts into Tosca. Re-engineering because for the last two years that I’ve been in this job, people kept telling me, ‘Ann, did you know that those existing Hewlett-Packard scripts really don’t test anything?’ ‘Well, gee, I don’t think I knew that.’ ‘Well, gee willikers, let’s see if it’s really true.’ Lord, have mercy. Twenty percent of those Hewlett-Packard scripts that were driven by the business to automate didn’t test anything. Well okay, they tested something but it wasn’t worthwhile.
“All of these little blue dots on your right beyond the solid blue vertical line are things that don’t really matter because we didn’t do a risk analysis. When I talk about re-engineering, we’re not just taking what we had and push it into Tosca, which you can do, by the way, there’s a tool for that. It’s up to you to use it, I’m not going to say anything. All I’m telling you is all our experience has been it made no sense.
“What we have found is when we sit down with our subject matter experts and we had this really cool dialogue like, ‘What do you do all day long?’ ‘Oh, I use VA-01 and NM-16 all day long.’ I said, ‘Cool. And so what happens when one of them doesn’t work? Dead in the water. You’re in customer service, aren’t you?’ ‘Yeah.’ ‘How many sales? Millions of dollars of sales do you do every day?’ ‘Well … ‘ This is the user, I mean they are totally into explaining to us what they do. Through this dialogue, this interview, we find out that they’re going to cost us $1 million a day, an hour, I don’t know what the big number is but it’s a big number, when it comes to customer-facing transactions. Therefore, those transactions are the ones that we want to automate.
“Now if I find out, no offense to anybody in controllers, that at the end of the month they have a report that they need to generate, ‘I’ve got some time to fix that one, okay?’ ‘How often do you do this? Monthly? Okay, that’s a one. What happens if it doesn’t work?’ ‘Well, I just did last month’s report and … ‘ ‘That’s a one.’ Low priority, move on.
“We have been re-engineering all of our existing tests.
“This word crypt is an ugly word. I actually I know the person who named our existing automations. Critical Integrated Process Testing. We don’t say that word anymore, by the way.
“The middle child has risen. This is my team. When I joined the group in February of 2015, we had about 25 people. About 25 people. Within ExxonMobil, I think there might have been seven of us. Then, we had some offshore group that did all of those monthly executions of those Hewlett-Packard scripts.
“Today, I have Vulcar over automation, he’s based in Hamburg. I have Cathy in Houston for functional testing and Santa Juan for functional testing, basically split between the two regions. They’re what we call test leads. They do a lot of the coordination of our testing activities. I have Aaron in Houston for performance testing. I have Jay, he is Doctor Jay, he is the man that knows everything about testing and everybody loves Doctor Jay. We have Suzanne right there, one of my oldest children. I didn’t say that, did I? I meant oldest child. Anyway, the other thing that was really funny about these pictures, they kept saying, ‘Oh, I want mine to be skinny.’ I want mine to be really upfront. We had a whole dialog on how the pictures turned out. It was pretty fun. Guess what? There’s close to 150 people now. That’s a signal that over this past year and a half, two years of getting this team put together, the emphasis that ExxonMobil has put on quality. One of the ways to get to quality was through testing.
“We are everywhere. I’m going to go into a little bit more detail now about organization because it’s good to understand where you are located. I was talking about being on the six continents. We have now placed people in region, so that when a person has a question on, ‘I want to automate, should I automate?’ We’ve got people in those locations to help with that. You will see we are using architects.
“Within the Tricentis nomenclature, you have architects, automation specialists and automation engineers. It is a tiering of skillset.
“We have found that the architects become those go-to people, when it comes to, ‘Should I be automating this or should I not and how do I go about it?’ Yes, we are using some offshore resources in India. Yes, we have people in Auckland. Yes, Bangkok, and that’s actually one of our ExxonMobil business support centers in Bangkok. Then, you can see Houston, Vienna, New York and Curitiba, Brazil.
“Organizationally, what we have found is that when it comes to implementing automation and doing it in a … I’ll call it conscientious and disciplined way, you’ve got to have a structure. Don’t think that you can just, I’ll say, run into doing automation. There needs to be a structure upfront to help you manage this work, because once one customer comes, you’re going to get five more. The enthusiasm level that we have experienced is unbelievable. We’re almost on tilt. But through this concerted effort of having an organization to help support it through a governance and then program management. Again, you can see this partnership that we have with Tricentis. Highly recommended when it’s time to get started.
“Successful team composition. It’s typically one architect to one engineer to 10 specialists and then you ramp up or ramp down, based upon if you’re working on systems of engagement. The shiny coin fun stuff, mobile, gooey, or your systems of record. They’re a little bit of a worn coin, all right? I’m sure some of you have systems that are quite old. We have ones that are really old so our financial system is still in IMS in PO1. Yes, rock on! That’s what I grew up with. Yeah, give me some JCL, too.
“See, geek, we’re all geeks, I know. It’s like once you join the world of IT, you’re just like into it. Nobody really understands what you talk about, unless you’re with people that are like you. Uh-huh (affirmative), all in the family, totally all in the family.
“Okay. We also have an immersion program. We have, again, the process of getting somebody into you’re approved, we think you can handle the work, all right? Going through automation specialists and all the way up to graduation. Again, working with Tricentis, we have come up with again, a disciplined process. You need this when you get started because they will come in out of the woodwork, I want to automate. I talked to you two years ago about automating those processes at the refinery and you said that no machine could do what you could do. I don’t say that, but that’s what I’m thinking when they come to me two years later with, ‘I think I want to do automation.’ I’m like, ‘Cool, let’s get into it.’
“We have a quality manifesto. Yes, we do. We will create forward-looking, value for our customers and for our organizations through quality approaches. Quality engineering is a team responsibility. I’ve got a great team, but we’re just here. We’re the messengers, we’re the trainers, we’re the coaches. Everybody that we work to, we talk about quality.
“How did you get to work today? Did you think about a safe way to get to work today with all of the rain? Yes. That’s a quality decision right there. Little things that we teach when we’re starting to push the quality manifesto out. You have to understand business function risk. Again, what do you do all day? What happens if it doesn’t work? Oh, really bad things happen, or maybe not so bad and a month in close, we’ve got a whole month to worry about it. Yeah, we can always close the books this month of last months’ numbers. I actually have a degree in accounting, I can talk about them.
“Including quality checks throughout the delivery life cycle. Quality gates. That’s another item that we have built into our system delivery life cycle. Starting to talk about at the moment requirements are written down, let’s start talking about what you’re going to test, based upon those requirements. We have built these quality gates, these questions throughout system delivery life cycle.
“What has happened? Our organizational design has shifted.
“Tosca is strategic for ExxonMobil. Tosca is an enabler to speedy delivery.
“Test automation brings value. We’re actually connecting the automation into total cost of ownership, as a way to reduce application total cost of ownership. We have a quality philosophy now and we’re just pretty darn excited about what we did. It took the rise of the middle child in order to make it happen. Thank you so much.”
In a recent episode of the Continuous Discussions (#c9d9) podcast, a group of industry experts discussed why DevSecOps is officially more than just a buzzword, tips on how to get everyone in the organization to own security and some of their own challenges and experiences baking security into the software delivery pipeline.
The panel included: Alan Shimel, editor in chief at DevOps.com; Chenxi Wang, managing general partner at Rain Capital; Derek E. Weeks, VP and DevOps Advocate at Sonatype; Paula Thrasher, Chief Architect, National Security Division at GDIT; Geetika Tandon, director of technology at Federal Civilian CTO Group; and Electric Cloud’s Sam Fell and Anders Wallgren.
Continue reading for some of their top takeaways!
Why DevSecOps?
DevSecOps is really more about a cultural shift, said Shimel: “DevOps is DevOps. It’s not just about Dev and Ops, though. DevOps is about the whole organization adapting a common cultural way of interacting including security, and bringing security into that culture, into that software ordination assembly line.”
Integrating security into development practices takes a lot of collaboration and communication, per Wang: “It’s easy to say security should be part of the process. But in practice, it’s difficult to do. When you have a security testing process that takes days to complete from the input to the output, and then you have developer teams releasing 500 releases a day, those are two very different cultures. It’s usually an iteration process but it takes a lot of team communication, a lot of sitting down, figuring things out for security to be part of it, and so it’s a not an easy process.”
Why DevSecOps and not just DevOps? Well, security people weren’t necessarily invited to the party, explained Thrasher: “It’s a cultural challenge that security hasn’t necessarily been an integrative part of a lot of the DevOps transformations. I think ‘DevSecOps’ exists because either we’ve left security out or they’ve stayed outside this transformation that’s happening.”
There really shouldn’t be a trade-off between security and quality, according to Wallgren: “I long for the day when we don’t distinguish between security and quality because, yes, there are different disciplines to them in some extent, but as far as the end user is concerned a bug is a bug.”
Weeks wanted to let nonbelievers know that DevSecOps is real and more than just a buzzword: “DevOps and security can integrate with each other and integrate successfully at scale. There are organizations of a couple of hundred or even a couple of thousand people that have a DevSecOps practice in place that are continuously improving it, adding an automated security, so it’s definitely gone beyond buzzword status.”
Including security in the release process should be a no-brainer, according to Tandon: “I think of it as common sense. I don’t even understand how security has been left out. I don’t understand how we have these huge pipelines, we’re releasing a hundred code releases a day and then, boom, we’re stopped. We’re stopped for scanning.”
Common Barriers to DevSecOps and Patterns for DevSecOps Success
Progress takes time, but Shimel remains hopeful of the future: “We’re not going to boil the ocean or change the world in a day, but we will change the world and we will change the way software is delivered so that software and security is synonymous with quality.”
Wang explained how one company is utilizing automation to successfully bake security into their process: “A company that I have worked with is in the container world and they have what they call a Brown’s registry. So they have code that is assembled by the development team and then goes into a Brown’s registry. No other code can get in to the Brown’s registry unless it’s from their own team. Then from that Brown’s registry the security team automatically goes in and scans it and then only those that pass the scan gets put into what they called a Gold registry. And from there then they could deploy it to the server.”
Everyone in the organization should practice and be prepared to respond to security incidents, advised Thrasher: “The people that need to practice responding to vulnerabilities the most actually aren’t the security people. They’re used to responding to security issues. It’s the ops and the developers who aren’t used to responding to security incidents and they aren’t equipped, they aren’t trained on how to respond. So when your vulnerability happens it’s a bad new story.”
Many security issues have been around for a notorious amount of time, which is where the software delivery industry can improve, said Wallgren: “Where are most of the exploits coming from? The vast majority of them as I understand it are coming from well-known longstanding security issues that have been out there for a long time or, at least long by technology standards. And so a lot of it is just simple stuff that we need to be better at.”
Security as code is the next step in truly integrating security with DevOps, explained Weeks: “I was just talking to someone in Europe at a big telecom company, and they were saying, ‘I’m on the security team, we’re working with development, and we’re getting closer. We have all these rules and things that developers need to follow and we’ve been working with getting them to follow them more.’ And I said, ‘That’s really cool. Any of those rules or policies instrumented as code?’ And he responded cross-eyed like, ‘What? Code?’”
Tandon shared an eye-opening experience with a customer: “I’ve seen a situation where we were about to go to production—it was DevOps with security toward the end—and we got 4,000 hits and we had to go to production, most of them were ignored. So are we really securing our system or are we making our own system even more insecure by doing that?”
Watch the full episode:
Want to dive deeper into DevSecOps? Industry expert John Willis, co-author of “The DevOps Handbook” and “Beyond The Phoenix Project” and VP of DevOps and Digital Practices at SJ Technologies, is scheduled to lead a full day hands-on workshop immediately following DevOps Enterprise Summit in London and Las Vegas. Register soon—space is limited!
A bifurcation in types of developers will become more pronounced in the years ahead as more application code moves to the network edge. As that transition continues occurring, the pressure on IT operations to be able to be able to flexibly manage different classes of applications by embracing DevOps processes will correspondingly increase.
Mike Piech, vice president and general manager of middleware for Red Hat, said applications running on internet of things (IoT) devices and smartphones present developers with a finite amount of compute resources that require them to have advanced skills. Conversely, back-end applications can now take advantage of multiple layers of abstraction, including (soon) serverless computing frameworks based on event-driven architectures that make potentially unlimited compute resources available. The availability of all those compute resources, however, encourages developers of back-end services and applications to be less disciplined when writing applications.
The good news is that it’s getting easier to right-size applications using trial and error methods versus having to commit to a specific computing architecture up front, said Piech. If something doesn’t work as well as it should, modular approaches to microservices make it possible to more easily move application code to where it should best optimally reside. Rather than focusing on the rigor of the underlying architecture, developers are free to experiment with different tools and languages more now than any time in the history of IT, he said.
In fact, Piech speculated that many graphical user interfaces will continue to run locally while accessing a wide variety of back-end services via application programming interfaces (APIs). That approach essentially creates a new form of hybrid computing wherein developers no longer need to think much about the runtime environment, he noted.
However, all that freedom tends to put a lot more pressure on the IT operations team. As the types and classes of applications continues to evolve, Piech said it’s only a matter of time before more organizations move further down the DevOps path. Adoption of DevOps processes across organizations has been somewhat uneven. But as developers experiment with different computing models, the expectation is that operating teams will be flexible enough to dynamically make the appropriate adjustments, he said.
Longer term, middleware and databases will eventually be deconstructed into sets of microservices that developers can call when needed. Today, IT operations teams make available monolithic application servers and databases. Many of those databases and application servers are starting to be encapsulated in Docker containers as part of a first step toward lifting and shifting applications into the cloud before transitioning to more agile microservices-based frameworks, Piech said.
In some cases, those services will be accessed directly via a platform-as-a-service (PaaS) environment or a serverless computing framework. Piech said Red Hat expects serverless computing frameworks based on event-driven architectures will be used mainly to provide bursting capabilities as a complement to long-running stateful applications that typically consume a fixed amount of IT infrastructure resources. It’s probable stateless applications that run intermittently also will find their way on to serverless computing frameworks. Red Hat has thrown its support behind the open source OpenWhisk serverless computing framework.
There clearly will be many more types of distributed applications running across the enterprise. IT operations teams may not have much control over what types of applications run where and when. But they will be expected to be able to deploy and manage them all, regardless.
There may be nothing more difficult in a DevOps transformation than aligning the business with its technology organization—and vice versa. Dealing with skills gaps and information gaps, while addressing communication breakdowns and cultural hurdles can keep even the most seasoned practitioners and leaders up at night. How do we align disparate business units accordingly? What practical tips can people take and implement along their journey?
These are topics we immersed ourselves in during a live video podcast ahead of the DevOps Enterprise Summit (#DOES18) in London. I was able to sit down with Carmen DeArdo of Nationwide and Jayne Groll of DevOps Institute to take a deeper dive into the DevOps transformation patterns and practices. This discussion centered on a major theme of DOES18 this year: business and technology alignment for the modern enterprise.
We hit on many important issues that I think we can all learn from and apply in our organizations respectively. One of the things we’ve learned over the course of the years is that, in DevOps, the focus is heavily cultural, but it has to be squarely on nurturing and getting buy-in across the organization. But at the same time, we need to be orchestrating our strategic alignment initiatives between the disparate functions and organizations within the business. That’s a tall task.
Watch the full video replay below, and/or read through the full transcript to see what key points were made and how we can adopt best practices to move our businesses forward.
Carmen and Jayne will both be speaking at DOES18 London this year and I will be leading video panels that you can see exclusively on DevOps TV / DevOps.com. Hope to see you there!
Alan Shimel: Hey, good day, everyone. Thanks for joining us. This is Alan Shimel of DevOps.com. And I’m here to talk to you all today with a few friends of mine about the upcoming DevOps Enterprise Summit in London. This will be a live panel discussion ahead of the London DOES event, which, by the way, is June 25th to the 26th at the Intercontinental Hotel, sometimes called The O2. Our panel today, I’m happy to say, are two really good friends of mine. Well, one is one of my business partners at the DevOps Institute: Co-founder and CEO of DevOps Institute Jayne Groll, and a speaker at DevOps Enterprise Summit London. Jayne, welcome to our hangout today.
Jayne Groll: Hi, Alan. Hi, Carmen. Thanks a lot. Really looking forward to it.
Shimel: Okay. So Jayne has already sort of introduced our next guest there, another good friend of mine, a recognized DevOps leader for many years. He really kind of kicked off a lot of the kind of groundbreaking DevOps work that we’ve seen at Nationwide over the years. Also a great Pittsburgh Steelers and Pittsburgh sports fan, Carmen DeArdo. Carmen, welcome.
Carmen DeArdo: Welcome Alan. Welcome. Hi, Jayne. It’s always great talking to you.
Shimel: Thank you. So all kidding aside, both of you are kind of experts and leaders within the DevOps Enterprise community, and you’re both presenting at DOES London this year. You’ve actually both presented at previous DOES events as well. So thank you for coming here. You know what? When you guys first started kind of preaching in the wilderness about DevOps and so forth, it was a bit of the wilderness, but it’s becoming commonplace in businesses and organizations throughout the world now. Our goal on this chat, guys, is to learn about the patterns and practices that leaders have identified for business and technology alignment, and really what we’d hope is just to share your experience and thoughts with our audience so that maybe they can learn something and they could integrate or assimilate that into their own transformational journeys and escapades.
And just again a reminder, this is about DevOps Enterprise Summit London, June 25th to 26th, and you can get more information on the event at events.itrevolution.com. You’ll find on there, by the way, the entire conference agenda, and beyond Carmen and Jayne, it is a gangbuster lineup of speakers. If you’re interested in DevOps and any kind of related type of technologies and frameworks, etc., it’s really a can’t-miss one for those of us, you know, who follow this space, or even for those of us just getting involved in it.
That being said, though, I’ve spoken enough. Let’s get started. I’ve given a little bit of your background, but maybe I haven’t said everything that needs to be said. So, Carmen, since I see your lovely face on the screen ahead of me here, why don’t you go first? Give us your background, your company role, etc., professional accomplishments.
DeArdo: Okay. Thanks, Alan. So I actually spent most of my career in telecom working for Bell Labs. Thought I would retire from Bell Labs like everyone else at the time, and then the telecom bust happened, and we all had to figure out what we were gonna do with the rest of our careers. But it served me well, because I think a lot of those concepts, even though we didn’t call them DevOps, but, you know, certainly, creating high-availability, high-reliability capacity systems, you know, those concepts are still, I think, pertinent obviously today. I moved into Nationwide to really help with trying to get Agile set up about 12 years ago, and then, you know, as I think you see the case with a lot of companies, they start somewhere in their value stream. They may not even think of it at that point like an end-to-end value stream, but they’re starting somewhere. They start with Agile.
You know, we were very successful getting all our work done there, but then realized, okay, that’s only part of the story. The cards getting into the backlog through iterations is great, but we aren’t getting flow of work from the business into the teams, and we certainly aren’t releasing into production frequently. So about, I’d say, five years ago, you know, somewhat in affiliation, some of the work I was doing with, as you know, Rosalind Radcliffe, Hayden Lindsey, you know, started to talk about what’s this DevOps thing. Was lucky enough to end up at the first Gene (Kim) conference speaking with Hayden. Wasn’t sure what I was getting myself into, but it’s been a wonderful experience. You know, got to meet you, Jayne, and lots of great folks from Gene’s community.
In Nationwide, we’re just continuing to work, how do we make that end-to-end flow and lead time, you know, what do we do to make it better? What DevOps practices can we bring? But as you noted at the beginning, how do you get the business more online? How do you get work to flow? And how do you address culture, which we know is probably more important than technology?
Shimel: Excellent, dead on. Jayne, how about you? You have more hair than both Carmen and I. You’re the lovely face we should see on the screen here. Give us a little background and accomplishments.
Groll: So I call myself an accidental technologist. So I started my career, got out of school, ended up in the legal field for about 15 years working on the business side. So I wasn’t a lawyer, but I was one of the early paralegals for a big Wall Street law firm that adopted UNIX a long, long time ago. I don’t even want to say when, but screens were black and green back then. And so everybody had to learn UNIX. And so I was very fortunate to be given an opportunity to accidentally become a technologist, and I know even with my technical skills, I’m a little self-taught. Was able to move up into some senior management roles across several different verticals, leaving the legal space.
And then in 2004, kind of fell on ITIL and really started to see a paradigm shift in IT that technology is important, but your practices, your principles, your processes are important as well. And people, process and technology became a mantra. And so co-founded ITSM Academy with Lisa Schwartz and really saw the growth of maturity in IT. You know, I always say we love our frameworks. IT, we love our frameworks more than any other business unit, and really started to see that percolate. Met Gene and some others through that experience, but really, landing where we are today. In 2012, I was able to attend an early DevOps day, frankly, at Gene’s invitation and really saw something percolating there that was pretty much common sense, but not common practice.
So we started to see, you know, should Dev and Ops collaborate better? Should there be a faster flow? Should there be less silos? Yes. Should we use automation more? Yes. We automate the rest of the world, we don’t automate ourselves as much. And really started to see something grow there, and, Alan, as you know, in 2015, we launched the DevOps Institute primarily because the practices around DevOps unlike other frameworks were really emerging in the wild. You know, you said it before. They were emerging in the wild and making some sense out of that, you know, as it crossed over into the enterprise space was important and requires a lot of research.
So, you know, we don’t have a single body of knowledge in DevOps. Thankfully, we have a collective body of knowledge in DevOps. So for me, I think part of my, you know, pride is my journey has not been a linear path. Really blessed to be, you know, where I am today, and, you know, DevOps Institute right now is, what, 32 countries that are talking about in doing DevOps. I mean, that’s really kind of surprising and remarkable all at the same time.
Shimel: Yep. And just to excuse, I don’t know if you guys can hear. We have a train, the 11-08 Cannonball Express, going via the office here.
DeArdo: Just like my cousin Vinnie here, Alan? Is that like …?
Shimel: That’s “two yutes.” The way I talk, we’re not far off. Anyway, though, guys, let us move along here. You know, one of the things that I’ve learned in my own exploration of DevOps and digital transformation, etc., is that a lot of businesses, large enterprises, they may think they do, but on the whole, they don’t really understand technology. Yeah, some of the guys in IT understand some of it or what have you, but do you think most businesses understand technology? Carmen?
DeArdo: I think it’s … Obviously, people are exposed to technology. In fact, I talk about this all the time, that sometimes we’re still in the book writing business or book production that we still feel like we got to produce manuals of paper material … The disconnect comes as, How we can effectively utilize the technology that we have to meet the business needs? And I think that’s maybe where the disconnect goes. It’s not that, you know, people certainly are … they have Alexas. They have stuff in their home. They may be even, you know, great at programming their home devices, but how are we actually taking advantage of that to serve our customers? And I still think thinking about things kind of in this … you know, Jayne talked about silos, but I actually think the last frontier really of DevOps is in the left part of the value stream, not the right part.
I think they code the cloud. There’s patterns, maybe not everybody’s done it. I know it’s not easy to do. That’s pretty well-defined from a pattern. The front part is not, and I think understanding that, you know, you have to basically be an IT business. We hear this all the time. If you’re not an … consider yourself an IT business, someone who is good at IT is gonna disrupt you. So it’s not that I don’t think they understand technology. They’re certainly capable of it. It’s just, do we really understand how we can use it the best way in order to meet business needs? And, you know, things like the system of engagement. Well, the system of engagement is gonna change. We’ve already seen it change. It changed from web to mobile. It’s going to voice. It’s gonna go to different engagement models.
That’s fine, but you still have to have the back-end technology, and the microservices, and the APIs, and the ability to deliver more quickly, and I think that’s the part of it, the gap we still haven’t bridged. And how can we actually get …? You know, one of the things that Nicole and I are talking about at the conference is the project-to-product model. And I think large companies like ourselves are kind of flailing at what … you know, that makes sense. It makes sense to align around a value stream, but how do you really define what those value streams represent so that you can actually get that flow and then use technology to help get that quick deployment and, you know, feedback? So I think that’s where we see a lot of the gaps, you know, from that perspective.
Shimel: Sure. Jayne, before I have you answer on that, you know, I got an email of pitch. I get a lot of pitches as you can imagine, but I got one the other day that, to me, really nailed the issue. The issue was they pitched me about how great their solutions and their services are, and they said it makes IT costs less. Instead of … so they’re still thinking of the IT department as a cost center instead of a profit generator. And to me, when you say that, that means you really don’t understand technology, you don’t understand the role of IT in the modern enterprise, and you’re just not doing it right. Jayne, your thoughts.
Groll: So I think you’re right. I think, you know, the modern enterprise has a few challenges ahead of it. Carmen, I absolutely agree with you. I think every business knows they need technology and they need to accelerate their use of technology. I think they don’t understand how to apply that, and more importantly, I think the big paradigm shift, and, Carmen, you said it, is that most companies don’t realize that they are now technology companies. It isn’t even just IT does for you, is the core company as a technology company. And those that kind of recognize that, I think, are those that move ahead. Those that are still kind of hanging on to a legacy … Now, you know, I’ll add kind of something into that recipe that’s a little scary.
With the way technology is moving and how fast it’s going, I mean, the acceleration is remarkable, and, you know, Carmen, you said it. It was cloud. It was mobile. Now it’s voice. That it feels like, you know, this kind of moving target that if I’m a business leader, not a technology leader, although Lord knows that, I think, has a plan here, you know, where do you put the dart? You know, do you invest in mobile? Do you invest in voice? Do you invest …? So it’s such a moving target, and it’s going so fast that I think it can be very daunting to figure out, okay, where do we make our play, and, you know, where do we use our technology spend? And I would say something funny, Alan, because you’re right.
You know, in my years in senior IT management, I almost always reported to the CFO, because I was a call center. And so it was nobody understood quite what I was spending money on, but they figured if I reported to the finance guy, that somewhere along the way the cost would be controlled as opposed to let’s optimize the capability. Let’s look at this in investment. So I really think organizations are trying to figure out, first of all, are we a technology company? And I would say unequivocally the answer is yes regardless of what vertical you’re in. And then where do I invest? What technology do I start looking at? And if what I do today is gonna be obsolete tomorrow, how do I make that investment for my company further?
Shimel: Excellent. So, guys, you both have a little bit of a background, more than a little bit, in training and so forth. Do we train people outside of the IT organizations on what technology can do for you? Do we put up sort of Uncle Sam’s “We want you. What can I do for you?” posters? How do we get the business to understand the technology aspects, the technology side of things? Go ahead.
Groll: I mean, you know, the “we want you …” unless you can promote something that says what’s in it for me, me as an organization, me as an individual, me as a leader, I think that’s difficult. You know, doing road shows on here’s what technology can do for you isn’t something that you can tell. I think it’s something that you have to prove. And I think IT sometimes runs into an uphill battle trying to prove that, because it is costly and it does change a lot of the way people perform on a daily basis. I know what I did today, and tomorrow you want me to do something differently, and I may not be as successful as I was yesterday.
So there’s a little bit of a proof of concept usually, you know, done through piloting, whatever, but I think the, “How do we tell them?” I don’t think we tell them. I think we have to show them.
Shimel: Fair enough. Carmen?
DeArdo: Yeah. I mean, I think we’ve gotten a lot of appreciation. I mean, I’ve told … you know, and Cindy Payne and Jim Grafmeyer told the story at the last conference about Hurricane Harvey and how the business was able to respond within a day to what was a wind event and came a rain event with all our flooded cars and how we got the website optimized to handle those, to take the load off of our customer service center. They’re starting to get that, “Gee, having this capability to deliver things more frequently …” We now have some of us who deliver daily whereas before everything was going into a monthly release. So that’s a great accomplishment.
On the other hand, we’re running into the things that, you know, Topos talked about with compliance, with legal, with … and just, you know, how do we … Most of our time now on lead time is going from the idea to the backlog of the team, not from going from the … you know, the team getting the work to deployment. So I think going into, you know, that view of how do we start to think more about, you know, the other things that are getting in our way of lead time, because it’s not just the IT part of it. It’s now the business part of it, and we have a couple areas now who actually have true product owners sitting with their lines and are almost at the point now where they can deliver, you know, stories and storyboard them and move them all the way through production, you know, much more quickly.
So I think that’s an example, and I think that’s what you need. I always talk about you need stories, you need examples of how this can actually be done at your company, because reading about it, hearing about it is great, but then when you see the team next to you doing it, and even if it’s not perfect and even if you know it can be improved, that’s where really you start to change the culture of not just IT but of the business. You want something like that. And then, you know, I think you got something going that you can really build on top of. Jayne.
Groll: Yeah. I mean, I think that conferences like DevOps Enterprise London, Las Vegas, other events, local events, I think, also let individuals and organizations know they’re not alone in this journey. You know, part of the challenge in DevOps right now, I think, is we’re going from what is DevOps to how do we do this. And it’s not finite. It’s not, you know, you open a recipe book and it says, “Start here. Add these ingredients. You know, stir, bake for 350 degrees, and you have DevOps.” It requires a lot of higher-level thinking, requires some experimentation. And so I think that, you know, access to what have other people or organizations done … Carmen, I think you’re absolutely right, what have other teams done, really kind of making it a community of enablement is very, very important in this space, because everybody’s trying to figure out where do they start. And unfortunately or fortunately, the answer is it depends. It depends where you are, it depends what you’re trying to achieve. So I think that’s part of it as well.
Shimel: Yep. So, guys, let me try to tie a bow on this section before we move on. The reason I asked about the technology and technology education issue is I think it’s imperative for organizations to understand that in order for us to get what I call business and tech alignment, and what I mean by business and tech alignment is that we’re all pulling in the same directions for the same reasons with the same goals, and we’re following hopefully a road that we have agreed upon, and we all know the road we’re traveling, because, again, you get back to DevOps, guys. You know, when you guys are out there talking about it and we were covering it at DevOps.com, what do we always hear? What exactly is DevOps?
There’s no real definition. What does it mean? You know, how do we define DevOps and so forth? And I think one of the things we’ve learned over the course of the years is that, you know, in DevOps, the focus is, you know, heavily cultural, but it has to be squarely on nurturing, buying across the organization for this “DevOps initiatives,” you know, that we’re starting. But at the same time, we need to be orchestrating our strategic alignment initiatives between the disparate functions and organizations within the business. And that to me is business and tech alignment. If you guys don’t agree with that, speak up, but if you do, let’s move on to our next part, but can we wrap up, Carmen, Jayne? Are you fair to say that what we’re really talking about here is business and tech alignment?
Groll: So I’m gonna jump in just for a second, because … So you know I come from the ITIL space. And so in the ITIL space, they were coining the term BITA, business IT alignment. And so for years and years and years, there was a lot of conversation about BITA and its semantics. But alignment means that we’re kind of standing side by side in lanes next to each other. The reality is it isn’t alignment. It’s really unification. It’s really we don’t need to align with the business, we need to become the business. And, again, I know it’s semantics, but that word alignment gets tossed around a lot, and it implies something like, “Well, I’ll drive my car next to yours, and we’ll travel at the same pace, and we’ll get to the same journey,” as opposed to, “Let’s be in the same car.” And so you understand what I’m saying?
Shimel: Yeah.
Groll: And I said the term BITA is sort of falling away a little bit, but it was thrown around a lot during the years when ITIL in particular, you know, was kind of front and center.
Shimel: Yap. Carmen, any thoughts on that before we move on?
DeArdo: No, that was great points. And, you know, I refer back to Mark Schwartz’ last book, “The Seat at the Table,” and where he really talks about IT really should move beyond trying to get a seat at the table and move towards, you know, as Jayne said, you’re actually moving together, and you have to merge really these concepts of what the business wants, you know, has to be very much aligned, more than aligned. It’s part of what IT need to be doing. So that merging has to take a fact, and I think as much as we’ve seen the culture issues in Dev and Ops, that’s gonna be an interesting culture issue. But we’re seeing the companies who are successful are already doing it. They are working together. And, you know, if you walked up to some of the teams I just talked about, you wouldn’t be able to say, “Oh, well, such-and-such is a business person, and, you know, Jayne’s a business person, and John’s an IT person.” No, they’re a team working together to achieve something for the good of the company, and that’s the way it has to be.
Shimel: Yep. Guys, let me broach another subject then, because I think we’ve covered this one. You know, I read an article yesterday about AI, and automation, machine learning, and we’ve all seen these before, and, you know, how it’s changing. But automation really is changing the job market, especially for people younger than the three of us. I mean, we’ve both seen our … all three of us have seen our 21st birthday recently and …
Groll: Just recently.
Shimel: Recently, but really the folks coming up behind us, and we’re talking about our children and so forth, automation and the job market that they’ll grow into is paramount. And one of the reasons is that we have a tremendous skill gap. You know, my background is obviously in infosec, and now we call it cybersecurity. I hate the term. But there’s a tremendous gap in the security space. There’s also a tremendous gap in the DevOps space, obviously, and that’s one of the reasons we see such a high demand, even for people who may not be certified or classically trained in DevOps, I mean, take some sort of DevOps class. If they’ve learned to work Puppet or Chef or Jenkins, you know, they’re in high demand here.
And so with this skills gap, it doesn’t hinder our DevOps transformation. Is it a drag on the entire DevOps industry or community? Jayne, I’m gonna ask you to take first crack at that. I mean, you must see this with DOI stuff.
Groll: I do, and, you know, it’s kind of like good news, bad news. So the great news is, where else do you get to work in an industry or an environment where you are now encouraged to learn something other than what you’re essentially good at? I mean, you’re now encouraged to grow new skills to evolve. Organizations are funding skills modernization programs. So, you know, we groomed … and Carmen referenced that. You know, we groomed an IT based on silos. And in every silo, you were a developer, or you were a QA person, or you were a security person, and you lived in your silo with other people who had the same skills as you did. And now, cross-functional skills and, you know, what they call T-shaped people where you need to still have a specialty but you’re absolutely required to have very large breadth of different skills means that organizations are supporting and funding that.
So the continuous learner is more the norm today than it was before where you will be a developer … experience, and I think it’s a great day to be in technology truthfully, because you are encouraged to think, and learn, and practice, and train, and do, you know, all sorts of other things. And I think companies are starting to see that. I hear the term “skilling” a lot. We not only talk about training, we talk about skilling. We need to skill our force. We need to skill our teams. We need to reskill or upskill. You know, those are really terms that are starting to enter our common vocabulary. And the day of the generic, “I’m a master, I’m an expert, I’m a practitioner,” and I’ve got a few of those credentials as well, those are so ambiguous that now the skills are very, very tangible, and organizations are hiring, I think, hiring and funding individuals to develop not only technical skills but practice and principal skills, too. So we’re all gonna be upskilled, maybe outskilled.
Shimel: Perhaps both. Carmen, how about you on this one?
DeArdo: Yeah. I mean, automation is obviously nothing new. I mean, I’m really dating myself, but I can remember when people didn’t trust the output of compilers. Like, are we sure the compilers are producing the right machine language or assembly language? I don’t know. I can bring the assembly language better than this compiler. I mean, I think automation and learning is just … obviously, you know, if you spent, you know, four decades like I have, and, you know, none of us are spring chickens, but you’ve got to learn a lot and keep learning. Now, I think the great thing, and I agree with you and Jayne, the great thing about Nationwide is they have invested in learning and upskilling their people. We had this thing called Teaching Thursdays where twice a month we have two hours which is set aside for peer-to-peer learning, and it’s really not … and it’s peers teaching people, and people love to be the teacher.
I mean, at the last forum, Jason Cox was talking about … We had a subgroup focused on operations, and Jayne was there. And talk about … you know, he had some folks on his team, that they have to become engineers. They have to become coders. And one of the guys who initially maybe wasn’t so crazy about it, once they became kind of the expert in their area and they were teaching other people, well, you know, they were really engaged then, because, you know, they were involved and they were the one now from a peer-to-peer learning perspective.
So companies, I think, like Nationwide, Disney, other companies that are leaders in this space need obviously to invest in that. And then, you know, it also means that the associate themselves, you know, needs to take advantage of that. I mean, we can offer those capabilities. We know the jobs are gonna change. We can make those skills available, and then folks need to take advantage of them, but yes, I mean, people have to become more broadly skilled and be able to become more of what I would call problem-solving engineering mindset as we move forward.
Shimel: Sure. So, guys, though, you know, so we have skills over here. And on this side, there’s another thing that creates a tension with the skills, and that’s what I call the information gap. So we have a skills gap, and you guys have both addressed that really well, but we also have an information gap. And I alluded to it when I talked about training. If an employer or an organization thinks that someone who can write a recipe in Chef or has heard of Jenkins is a DevOps engineer, if there’s such a thing as a DevOps engineer, or is someone who, you know, is gonna really help build their DevOps team, to me, that’s not a skills gap, that’s an information gap. How do we understand … How do we make organizations … How do we fill that gap on information so that organizations recognize, A, that there may, in fact, be a skills gap, B, how to fill that skills gap and bring the organization up to it?
I mean, this tension, this chicken-and-egg, if you don’t know it, you don’t know. How are you supposed to know? And so I’ll put it to you. What do you think is more important? The information gap, the skills gap? Are they equally important? Do we tackle them concurrently, individually? Carmen, how do you guys do it at Nationwide?
DeArdo: I mean, I think one of the things that IT hasn’t been good at is treating IT as … we’ve been a very bad customer to ourselves. So if you look at our pipeline, we don’t treat it as a product. If you actually want to get faster at delivering a capability from idea through deployment, you need to consider that as a product, not a bunch of different tools. It’s not Jenkins, and New Relic, and UrbanCode or, you know, the AWS. It’s not that. It’s a pipeline that’s gonna enable you, and it doesn’t start at code, it starts at idea to get you all the way through that deployment. So then what … do you have a team? Are you doing thinking across the pipeline? Are you doing architecture? We don’t do things like value stream architecture or architecting the pipeline as a product. We want to buy a bunch of technologies. We can hire people good in those technologies, but as you said, that’s not really gonna get us where we need to go.
So I think the first thing is, do you really treat your product … One of the first things, do you treat your pipeline as a product? Do you have a team that can actually support that product all the way? Do you have a product owner who can drive continuous improvement across that with funding and prioritization? And do you have one of your best architects, if not the best, to actually create the architecture and do value stream architecture, which is something, you know, that Mik Kersten and I have been talking a lot about. I think that’s one side of it that we don’t get, is teams could … if you ask a team, they can kind of figure out, tell you like a Deming quality circle, how they can go faster. And you can experiment, but then if you’re gonna operationalize that, you don’t want 200 teams doing things in 200 different ways. You can then take that and say, “How do I address that as part of my delivery capability?”
And then I think what you need your leadership to do is apply systems thinking, and that’s a whole another subject we could get into, but I don’t think executives and leaders know how to really address the, “How do I optimize this as a whole?” They want to get into the weeds. They want to get into the tool selections. There’s people that can figure that out. You need to really focus on the strategy and how you can optimize all this great information you’re getting to make sure you’re removing barriers and allowing teams, you know, doing the learning, making sure they can be upskilled, investing in their ability, and allowing them, you know, to be able to innovate and improve your delivery capability. And I think that’s what I just said quickly. It’s easier said than done. And I think that’s where I see a lot of the struggles, and I think where we’re trying to address this is how do we start to put some of those pieces in place so it’s not just a bunch of, you know, divergent disparate technologies we’re trying to bring in and hope that that’s gonna make us get better. Because those things and having skills just around those things is not gonna actually get us to where we need to go.
Shimel: Fair enough. Jayne, I’m gonna give you a chance to comment on the DevOps skills versus information gap.
Groll: So I think the two are really related, and I think Carmen brings up some good points. You know, I think it all comes down to silos. So let’s look at our silos. Well, within our silos, we speak the same language. We use the same acronyms. We use the same tools. We live in a common community where everybody kind of understands what’s going on and what’s happening. And then when we leave our silo and we try to talk to a different silo, we don’t speak the same language. We don’t use the same tools. We don’t have the same processes. So I think the skills gap doesn’t fully overcome the issues with the information gap, but, certainly, there’s help there that if we’re looking to share information, we have to share it in a way that everybody kind of has some type of uncommon understanding.
The other thing with information is, you know, information is really interesting to me, because everyone thinks they communicate a lot. People generally think that they communicate well. You know, teams generally think they communicate … like I told you that. You told it to me maybe in language I don’t understand or it wasn’t clear. It was some report that, you know, I don’t even know where it is. So I think, you know, kind of the flow of information and from a vocabulary perspective, from an understanding perspective, from a “Did you understand what I shared?” is something we all need to overcome. And I think, you know, again, they’re so interrelated to each other that if we start to create a broader set of skills, hopefully, that helps to improve the communication. I think the other side of it is we talk a lot about collaboration, and there’s a really fundamental difference between communication and collaboration.
And so information flows through communication, but communication is pretty much one way. I tell you, you tell me, and somewhere along the way. Collaboration is I start to ask your opinion, and that’s a really big, big difference. And so when we’re starting to share information, there’s a respect that comes across the skills gap. There’s a lot of things that go across that that, again, there’s separate gaps, but hopefully, the long term, and as Carmen said, not easy to fix, is by overcoming some of the skills gaps as well.
Shimel: Fair enough. I think we’ve covered that one well. Let’s move on to another area if we can. We have 20 minutes or so. You know, one of the things that really does my heart good is this concept of DevOps everywhere as far as the eye can see, like the meme with Buzz and Woody. We have seen DevOps move from just Dev and Ops to QA, DevSecOps exploding on the scene where, you know, a couple days, the 28th, a day or two after DOES London, John Willis and team are rolling out our second DevSecOps Days, which I’m involved in, and devsecopsdays.com for those interested. But even beyond IT, we’re seeing what I call DevBizOps, where DevOps’ Agile sort of practices are starting to take hold in HR, in sales, in finance, in, you know, accounting, etc.
And let’s be fair. This is not unique. The fact of the matter is a lot of the lean terminology, a lot of the lean stuff that was built into DevOps didn’t come from IT, it came from manufacturing. And so, you know, it’s like pay it forward. Now we’re seeing these kinds of DevOps principles move into other areas of the business. First of all, is that a good thing? I think it is. I can’t imagine it’s not, but I’ll defer to the both of you. And then, secondly, you know, hey, at the end of the day, remember, it’s all about the culture, stupid. And so what human factors, cultural factors should we be thinking of when we talk about a broader DevBizOps opportunity? And, Jayne, you’re on my screen, so I’m gonna tell you to go first on this one if you can.
Groll: You know, it’s business transformation. You know, it’s not really DevOps transformation. Digital transformation is not limited to IT. It’s really business transformation. And so it goes back to, you know, we talked about a little while ago, where, you know, every organization has to recognize that technology is their business, and so some of the practices from the business start to assimilate into IT and beyond. I saw this in service management where there’s now movement towards enterprise service management, where you’re gonna make changes, you do it the same way.
I think when we start to look … I mean, I can tell you from the content perspective, you know, we use tools in content development, even though it’s not software development, that are very, very similar to that. So I think it’s fantastic. I was talking to a gentleman from an organization on the Netherlands recently, and they actually took the Spotify squad model and applied it to their entire business, and not a small company, by the way. And so they applied it to their entire business where their business is now broken into squads. And, you know, some think that that’s only for a tech company, and by the way, this is not a tech company. It’s a large financial organization.
So I think that’s great. I do think there’s some … I think there’s good news, bad news. I think there’s a little bit of risk that, you know, with DevOps and some of DevOps practice is still emerging that it becomes, “This week, we do it this way, and this week, we do it another way.” And, you know, “Oh, that didn’t work because …” You know, fill in the blanks, but I do think that kind of having a one business and getting away from this we love our frameworks and IT, and every year, we have a new framework that supposedly supersedes the other framework, you know, that’s a maturity issue. And I think if the business embraces it … you’re right, Alan. Lean Six Sigma. That was a business initiative that a lot of organizations underwent that weren’t necessarily manufacturing organizations. So, you know, as we get closer to that, I think the unification becomes stronger.
Shimel: Great. Carmen, how about you?
DeArdo: Yeah. So I think of things like … It is a great thing. So I think of things like lean concepts. We have leaders across the company that we have gone through lean education and lean concepts. So when you talk about value streams, when you talk about lead time, when you talk about wait states, people understand those concepts, whether they be on business or IT. I mean, this whole idea that Dominica, I’m sure, would have talked about, about making work visible, Kanbans. I mean, we actually had an explosion of people who wanted to use some of our tools that initially were brought in for work out of management because they wanted … you know, it’s like, “Gee, who knew, you know, this Kanban stuff would work for us and WIP.” WIP limits. And so people certainly want to apply those kind of concepts.
So I do agree that, you know, the basic concepts, you know, really can spread everywhere, and the whole concepts of even the things, like we talked about it, the three ways. The three ways don’t just apply to IT. The three ways apply to anything that you’re trying to do and get better at. I think when it comes to information sometimes, though, you know, it’s kind of like the robot story and the goal. We have lots of KPIs, but are we actually focused on the goal? So the goal is not around keeping this person utilized or that team utilized. The goal is about, can I get faster from one end of my value stream to the other? And do we have data that shows flow? And that’s why I’m excited with some of the things that I’ve been … you know, that Mik’s talking about. You know, he has a book coming out that is gonna show actual flow of work, because up until now, we have not been able to actually see work flow all the way through that delivery pipeline. And so we guess a lot of times. We think, “Oh, well, if we just add more developers. Oh, we’ll add more testers.”
Well, how do we know? We don’t really know, because we’re making maybe educated guesses, but we’re really not all focused on understanding where the work is and those concepts that can shine a light on it. So I think it all does come back to a lot of those concepts that, growing up in Bell Labs, I was lucky enough to experience with Deming, even going back to Shewhart, around doing those experiments, getting better, seeing the entire end-to-end value stream, but these concepts apply. I mean, “The Phoenix Project” is really not a book about IT even though I teach part of it. It’s a book about how do you get better at doing whatever business you’re in and trying to, you know, stay in business and improve what you’re doing for your customer. So I think those concepts will continue to go, and I think, again, as we’ve talked about today, there’s a thread here of everybody needs to be on board with these concepts: legal, compliance, the business, everybody, because they all play a role in allowing you to do a better job, delivering more quickly to your customers.
Shimel: Fair enough. Guys, let me … You know, Carmen, you hit on some great things there. The Kanban board or even here at DevOps.com, frankly, we run our entire customer engagement, not practice, but now into farming, on Trello cards, which is sort of an electronic Kanban. And it’s a great project manager. I’ll give you another thing. You know, there’s the cultural aspect of this of taking that sort of DevOps culture and expanding it throughout the organization, but there’s also the bits and bytes and tools aspect of it. Kanban’s a great example. I’ll throw another one out at you. Get up. Why does it have to be source code for programs? Why can’t it be source code for contracts or any kind of digital document or, you know, things that we work on?
So I think we’re just starting to scratch the surface on how we can take that DevOps can do or Kanban and, you know, move it out across the entire organization. And we’ll all … I think we’ll have better organizations as a result, but let me now turn it back around. How can we take some of the business culture positives, some of the things that have worked in other parts of our business, and reflect it back into our DevOps and IT processes? How can we enhance our DevOps by adopting broader business principles and practices that seem to work? Carmen, I’ll let you take first crack at that.
DeArdo: So I think, actually, a lot of our test and learn activities started with the business who had actually hypotheses around, “Well, if we offered a policy in a different way or if we made a tweak to something, would we be able to sell more of those things?” So we had people who … and there’s a lot of statistical rigor that you have to bring to that. So we actually had people on the business side who had these ideas about, “I have a hypothesis. You know, is there a way you can allow me to test that hypothesis to see if it, in fact, will yield some positive result in terms of actual policies being sold?” So it was a very scientific-type experiment that they actually wanted to run, and at the time, we really didn’t have a way to support that from an IT perspective. And it kind of forced us to do some things to support, you know, how could we actually get this out and offer it to only a subset of people, which we now might call canary launch or some other terms that are DevOp-ish terms, but really nobody was reading a DevOps book at this point. They were just trying to figure out, “How can we support the business’s need here to try to adapt more quickly and have a high plan around?”
And that started, I think, some of the things that actually were successful that then led into once we started to get into DevOps, like, “Okay, what we actually did there was DevOps.” We didn’t call it DevOps, but those are principles that really we need to support not just in this one’s case, but for all kinds of business hypotheses that we want to get out and turn around real quickly. And doing that really does bring everybody together, because it’s exciting. It’s exciting if you can sit at a business and you can think up, “I have ideas for this product or that capability. You know, how can we more quickly try to see if that would be a good idea?” rather than writing a 12-month business case and trying to do, you know, a protracted study.
So the whole idea of experiments and gain things out quickly, I mean, everybody will resonate and get behind that if you talk about it in that way. If you start talking about Puppet and Chef and things like that, that’s not gonna cut it. But if you talk about it in terms of business and what it can mean for the business, then that resonates and gets a lot of people excited.
Shimel: Excellent. Jayne, how about you?
Groll: I mean, I think Carmen is right on point. And, Carmen, you said something I think that was … you’ve said a lot of very insightful things, but I think in particular that the experiment came down to, “How can we sell more policies? How can we make our business more successful?” And I think if you look at kind of the past in IT, sometimes, we didn’t know how we were contributing to the bottom line. So we were introducing a lot of technology, and we were doing all this cool stuff. But how did that convert into an increase in revenue, an increase in market share, increase in competitive advantage, things like that? And so I think that this unification with the business becomes more significant when IT sees themselves as an enabler of the goal. The goal of every business is to make money. And so sees themselves, and it goes back to information and communication as well, that they understand that what they’re doing is actually helping to sell more policies or increase market share or stand above the herd or anything along those lines.
Now, again, if you’re the CIO, you probably know that. If you’re in senior management, you probably know that. If you’re a developer and you’re one of a few hundred developers, you may not know that. And so I think that it is that kind of bottom-up top-down approach where understanding that what you do every day is contributing in a very, very substantial and tangible way. And then I think the other part of it, and, Carmen, I think you said it really well, is encouraging those experiments where the business and IT work together. And maybe it is that 1 in 100 coder that’s working as part of that team to say, “You know what? I have an idea.” Everything starts with a great idea. It could be from the business. Hopefully, from the business, that says, “I have an idea. Let’s see if we can, you know, bring that to life and what it will do to our bottom line.” And then there’s really a great collaboration between IT and the business.
Shimel: Agreed. Agreed. So, guys, we’re coming up to the top of the hour here, and I guess we need to wrap up. Otherwise, we’ll just be on air all day, and I know Jayne has a show in Vegas and so forth, and Carmen has stuff to do. But first of all, Carmen DeArdo, Jayne Groll, thanks for being our guests here on this hangout as we, you know, continue to build up to DevOps Enterprise London, DevOps Enterprise Summit London. Just a reminder for anyone watching, it is June 25th and 26th in London. There are still some tickets left, I think. I don’t know how many. I wouldn’t wait. You can get updates about the event as well as get tickets at events.itrevolution.com. It’s obviously on the web.
By the way, this year, as in past DevOps summits, DevOps.com is a media sponsor. We’re partners. We’ve partnered with Gene from the beginning. We will be doing videos again this year at DevOps Enterprise Summit. I think it’s gonna be a little different this year. We’re doing some interesting roundtables. You’ll see them on our YouTube channel, DevOps TV. So stay tuned for that. All right. Guys and lady, or guy and lady, I had a great time today. I’ve learned a whole bunch. I hope our folks listening and watching this have learned as well. You know, you guys have done, not just today, but, Carmen and Jayne, you’ve been instrumental in helping to bring the gospel of DevOps to the world, if you may, or at least maybe not the gospel, but at least some really good learning and understanding.
So thanks again, guys, for being with us. We’re gonna call it a wrap. Carmen, any last word? Three words.
DeArdo: Three words. Project to product.
Shimel: Okay. Jayne? Last words from you.
Groll: Pittsburgh Steelers. I don’t know.
Shimel: We finally got her. Carmen, we converted her.
DeArdo: I should have said, “Let’s go Tens.” That’s three words.
Shimel: Okay. Hey, everyone. I will see you both in London June 25th and 26th. Hope to see many of you listening or watching this in London as well. If you’re there, stop by, say hello. I always like to meet people who have watched us online. But until then, this is Alan Shimel for DevOps.com and IT Revolution. Thanks for watching, everyone, and hope to see you soon. Bye-bye.
Red Hat is making a strategic bet that IT organizations increasingly will rely on platform-as-a-service (PaaS) environments to make a broad range of distributed services available to developers, which would eliminate friction without enabling IT operations teams to maintain control.
Paul Cormier, president of products and technologies for Red Hat, said IT organizations are realizing PaaS environments such as OpenShift have evolved into a platform through which a broad range of curated distributed services can be exposed to developers by IT operations teams. That capability is critical because, otherwise, developers simply go around the IT operations team to download libraries. IT operations teams before too long are asked to support applications of uncertain provenance that in many cases wind up duplicating functions.
As appreciation of the role PaaS environments play in enterprise IT, Cormier said Red Hat is increasingly having conversations with CIOs and other senior IT leaders who are trying to bridge the historic divide between developers and IT operations teams by exposing a broad range of curated distributed services via well-defined application programming interfaces (APIs).
Red Hat is expanding the types of applications that can run on OpenShift alongside traditional 12-factor applications. In fact, by embracing Kubernetes container orchestration software, Red Hat is effectively obliterating the line between a PaaS and container-as-a-service (CaaS) environment, Cormier said, adding OpenShift makes it possible to build and deploy those applications on-premises or in the cloud as IT organizations deem best, based on cost, data gravity and regulatory requirements. Serverless computing frameworks will make it possible to bridge those environments by invoking an event-driven framework to take advantage of, for example, cloud bursting in a heterogeneous hybrid cloud, he said.
In fact, Cormier noted more IT organizations will mix and match internal and external services, as each platform tends to run different classes of application workloads better. For example, IT organizations may want to be able to invoke one type of artificial intelligence (AI) service on one cloud over another based on their performance attributes.
Traditional enterprise IT organizations are not looking to eliminate the need for IT operations teams, he said. Rather than have developers take over IT operations, most organizations would prefer to maximize the amount of time developers spend building and testing applications. The need to build more applications faster than ever will force IT leaders to separate those concerns to eliminate much of the internal friction that results when developers throw application code over the proverbial IT operations wall, Cormier said.
Most IT organizations have yet to embrace a PaaS environment broadly. There tend to be pockets of adoption. But most applications today are still built and deployed without employing a PaaS environment, mostly because PaaS environments are viewed by many as being too prescriptive in terms of dictating application development workflow. Red Hat is betting that with the rise of microservices-based applications based on containers, many IT organizations will finally decide to trade a little less control in exchange or for much less IT infrastructure complexity.
A global survey of 353 application development professionals conducted by Dimensional Research on behalf of LightStep, a provider of application performance management (APM) tools, finds a full 86 percent of respondents expect microservices to be the default architecture within five years.
A total of 60 percent of respondents said they already have microservices in production or deployed as part of a pilot, and 92 percent said they expect to grow their use in the coming year.
Top reasons cited for employing microservices are agility (82 percent) and scalability (78 percent).
The downside of increased reliance of microservices is the need to manage them all. The survey finds 99 percent report challenges in using them. More than half (56 percent) reported that each additional microservice increases operational challenges, and 87 percent of those that have deployed microservices in a production environment reported they are generating more application data.
To make matters more challenging, a full 98 percent of respondents said they have trouble identifying the root cause of performance issues associated with microservices. More than three-quarters (76 percent) said it also takes longer to resolve issues. In fact, 73 percent said it is more difficult to troubleshoot application performance problems compared to a traditional monolithic application.
Not surprisingly, just less than three-quarters (74 percent) said they plan to increase investments in microservice performance management tools in the next year.
LightStep CEO Ben Sigelman said that while microservices come in many forms, advances in how software is developed is clearly coming at a cost. Not only is software becoming more challenging to manage, APM tools are now a prerequisite. Historically, IT organizations only monitored their most mission-critical applications. But the level of dependency each microservice has on others requires a more holistic approach to APM, Sigelman said, noting that IT teams now need to be able to see well beyond the border of any one microservice.
Challenges associated with managing microservices are not likely to deter organizations from eventually adopting them. The level of agility and resilience inherently built into in a microservices-based application are too high to ignore. But the rate at which they are employed might be impacted. There’s usually a direct relationship between the availability of tooling and expertise and the rate at which any emerging technology gets deployed in a production environment. The good news is advances in machine learning algorithms and other forms of artificial intelligence (AI) should make it easier to manage microservices in the years ahead.
In the meantime, IT organizations can take some comfort in the fact that developers are being held more accountable for the microservices they build. That level of accountability tends to vary widely by organization. But the end result is generally higher levels of software quality because developers increasingly realize they can’t throw code over the proverbial IT operations wall without it coming back to them. The challenge now is to bridge the historic divide between developers and IT operations teams in a way that works best for everyone concerned.
