The Assessment of DevOps Capabilities (ADOC) is crowdsourced, vendor-neutral and designed for individuals, teams and organizations who want to baseline their current DevOps state, identify the next target state, gain insights into how to improve their organization and team capabilities and measure and accelerate continuous improvement during their DevOps journey.
The assessment models five DevOps dimensions – the human aspects, process and frameworks, functional composition, intelligent automation and technology ecosystems – to help you gauge your DevOps maturity and identify areas for improvement. In this TechStrongTV interview, chief content officer Charlene O’Hanlon talks with Feisal Mohammed, Niladri Choudhuri and Don White about the evolution of the ADOC and how it can assist organizations at any stage on their DevOps journey. The video is below, followed by a complete transcript.
Announcer: This is Digital Anarchist.
Charlene O’Hanlon: Hey, everybody. Thanks so much for joining us today. This is Charlene O’Hanlon, I’m the Chief Content Officer at MediaOps, and I am very, very excited to be having a conversation with three very, very smart people from the DevOps Institute Ambassadors. We have Feisal Mohammed, Niladri Choudhuri, and Don White. Gentlemen, thank you so much for joining me today. I’m very, very excited to have a conversation with you all about the new assessment of DevOps capabilities.
But first, I wonder if we can start by having you guys introduce yourselves. Please tell us your name, your title, your company, and what you think we all should know about you.
Feisal Mohammed: I’ll start first—Charlene, is that okay?
O’Hanlon: That’s perfect.
Mohammed: Alright. Good morning, everyone, and good afternoon, good evening, where you are listening in. First of all, thank you for having me here, it’s a tremendous honor to speak today. My name is Feisal, I am with Sapience Consulting, I’m the Principal Consultant at Sapience Consulting, which is a consultancy and training firm that is headquartered in Singapore, but we deliver our services regionally in Southeast Asia and in Asia, per se. We’ve been in business since 2008 and our core expertise is actually in the area of service management, governance, risk, compliance, security management, and of course, in the last five or six years, the natural evolution of DevOps.
And I’m proud and happy to be here. I’m a DevOps Institute Ambassador. I am leading the practice when it comes to DevOps for Sapience Consulting.
O’Hanlon: Alright, great. Thank you, Feisal. Niladri, tell us a little bit about yourself.
Niladri Choudhuri: Thanks, Charlene. I’m Niladri Choudhuri, CEO of Xellentro, India and Singapore. I’m also the Chief Organizer for the largest DevOps event that happens out of Asia, and coordinated the DevOps India Summit. And quite excited to be part of this panel because the ADOC, the Assessment DevOps Assessment Capability is going to be a great one, and we will further talk about it.
O’Hanlon: Alright, great. Thank you, Niladri. And Don, please tell us about yourself.
Don White: Well, I’m honored to be in the presence of some pretty brilliant folks that have been in the DevOps community for a long time. I really appreciate, Charlene, you inviting me. I come in to DevOps from a 15-year career in the software quality assurance space, and started with extreme programming and it evolved into the lean agile space, and then eventually got into DevOps.
Worked with many small startups, got them transitioned into lean agile ways of working, extreme programming ways of working, and that led me into coaching and being a DevOps Dojo Sensei for the Fortune 1 company in the world. That catapulted me, launching my business, Agility Science, and that’s where I do consulting and coaching for people throughout the world, quite frankly, but mostly United States-based.
O’Hanlon: Alright, great. Such an honor to have all three of you on the panel with me today. I love the breadth of experience and depth of experience that you all have and each one of you brings something different to the table, so I do appreciate all three of you being here.
I want to dive right into the Assessment of DevOps Capabilities tool, I guess, for lack of a better word, it’s an assessment tool, that the DevOps Institute has recently released, and it is now available for organizations to use it to basically assess their DevOps capabilities. It’s one of those titles that needs no explanation, basically. [Laughter] But I think it’s a great tool, and I think it’s going to be very well received in the market.
But I do wanna talk a little bit about the background of it. So, Feisal, maybe we can start with you. What are some of the challenges that you’ve seen in assessing DevOps capabilities in the enterprise? It seems to be, obviously, there is a need or this assessment tool, so what do you see?
Mohammed: Well, for me, an assessment—any assessment worth its salt would need to have a good sample size. I think that’s essential. You need to have a really good sample size and not just depend on the responses of a select ________. Let me represent the—misrepresent the actual conditions, if you like, that the organization is actually at, right?
So, what I do like about the ADOC assessment tool was that it allows for you to scale up the assessment. It allows hundreds of people to participate in this exercise. You know, if you do not have such a tool available, what happens is that this will be largely a manual, face to face process, right? And when it comes to a manual, face to face process of getting assessments, getting people to chime in on their views, chime in on their feedback—essentially, what you will get is tyranny of the loudest, right? The person with the loudest voice will be heard. And then you’ll get the messes that just shrink into the background, right, just disappear into the background.
With this assessment tool, it allows people to, in some ways, some anonymity, and they’re braver to give their true assessment of where they are in terms of DevOps capabilities. I think that’s the really cool part about the ADOC tool.
O’Hanlon: Alright. Niladri, what about you? What challenges have you seen with assessing DevOps capabilities in the enterprise? Beyond Feisal noted with having enough people to do a good sample, what do you see?
Choudhuri: So, a couple of points. One point is, if you look at the assessment—who is doing what, is the point. So, there are multiple organizations or consultants who goes and does assessments. And one of the challenges which I see is that many of them thinks in terms of maturity. And like what I have discussed with John Willis and some of the people who have started the momentum of this, I mean, how can you have a majority of an outcome?
It’s a continuous improvement process. So, we have to look at it from that point of view. Not that these 25 points have been ticked, so we are done, we don’t have to do any more.
O’Hanlon: Right.
Choudhuri: However good we are, we can still improve. So, that’s the first thing. And the second point, which I also see is that the senior, top people in the organization, they have, as people mentioned, they have a rosier picture of what is actually happening.
And I have seen that, when I go to any of the conferences or even in the DevOps Enterprise ________, when you see them on the podium, the top management talking on the podium, I call it—I mean, it’s not that they are not doing good work, they are definitely doing good work, but it’s a more sanitized version of what is actually happening. When you talk to their team members during the breakfast or the lunch, you get to know a lot of the other challenges which comes up.
So, as Feisal said, another very important aspect is to get the right information. So, one is the sample size. The other is, I would call it the psychological safety. The people should be able to come out and say what they really feel, and not always say everything is nice.
O’Hanlon: Interesting.
Choudhuri: Now, that, getting that right picture is very important, because ultimately, we all have to understand that we are doing it for the improvement.
O’Hanlon: Yeah, that’s very well said—very well said. Thank you, Niladri. And what about you, Don? What are you sing? Because, obviously, you are working with so many different companies in so many different areas. I imagine you’ve seen these challenges kinda run the gamut, so what are you seeing out there?
White: Everything that Niladri and Feisal said is spot-on. The one thing that I would like to mention is what to assess. Our assessment tool is so comprehensive and informed by all the expertise. And knowing what to assess is a huge part of it, but it’s really important to get a true, good assessment of the whole area. The psychological safety is absolutely spot-on. People need to be able to be true and honest about what’s going on, because you’re absolutely right, Ni—
O’Hanlon: Niladri.
White: – Niladri. The starting point, you can always improve. It’s all about that continuous improvement journey, but knowing where you’re at is critical. And most people try to think of the technical stuff—there’s so much people, the people part is the biggest part. And being able to assess the people part—because culture is my big thing. Culture is my huge thing, because if you don’t have the right culture, that mindset will not take root and it won’t grow, it won’t blossom, and you can’t grow it. you gotta get that culture right. The people part is the critical component.
So, that’s—what I love about this assessment, it digs into that. It’s not just all the technological know-how, it’s the people pieces.
O’Hanlon: I love that.
Mohammed: And that’s spot on. If I just may add on a little bit and ________ on what Don said, I fully agree with you. The culture is the most important ingredient necessary for any movement to take place, right? Any maturing, any improvement—it has to be the right culture. And the thing about DevOps is, the organization is so—it’s so diverse, right, that DevOps may mean different things to different people.
O’Hanlon: Mm-hmm.
White: Yes.
Mohammed: You ask a group of persons and they say, “Oh, we’ve got this set of tools, we are already DevOps.” And you ask another set of persons and they say, “Look, our processes are really lean. I think we’re DevOps.” And so, you know, people do not have a holistic, complete view of what DevOps is. And that’s fine. They’re coming from different perspectives. And when assessment comes in, it binds everything together, and it gives the true capabilities of what an organization is really capable of, right, from all aspects. And that’s really important.
White: I mean, I learned some things just by taking the assessment of some things that I wasn’t even aware of, practices and things of that nature. And you know, quite frankly, we’ll take it next year and we’ll learn more, because DevOps is continuously evolving, right? That’s the big thing, especially from the technological perspective. But the people pieces—that’s the huge thing, the mindset and where you’re at from psychological safety, those are the huge pieces.
O’Hanlon: That’s a really, really interesting point, because, you know, you think about an assessment, a lot of times, companies will take an assessment, and it’s kind of—in their mind, it’s kind of a one and done thing, right?
White: Mm-hmm.
O’Hanlon: And it’s kind of the same with DevOps, you never really—to your point, you guys, you never really achieve DevOps maturity, because the industry is changing so much.
So, with that in mind, though, a company will take an assessment, they have the information—how do they use that information, and is it something that they treat as maybe, say, a living document that is constantly updated and improved upon? Niladri, I’d like to start with you on this question, because I think it’s very, very important that organizations understand exactly what we’re talking about when we’re talking about this assessment tool and DevOps in general.
Choudhuri: Absolutely. Very, very important point, because that also has a lot of impact on what we get as the data. The point here is, we have to be very, very clear as a consultant when we are going with ADOC is that this is not a tool to fire your people. This is not a tool to evaluate your people, whether they are doing something good or bad. It is about how to make them more efficient, effective, and better. It is about achieving the organization’s goal, what needs to be done.
That is definitely a very important aspect that I feel. Because if there is that fear, if there is, that psychological safety is not there and people think, “Oh, at the end of this, my management will look at it and see whether Niladri has done better or Feisal has done better and give a promotion or a hike to Feisal and not to Niladri, then Niladri is not going to deliver—I mean, give the right information.” Then the whole purpose is lost. So, we have to be very, very categorical in stating, as a consultant when we go, that this is for the improvement journey and not to punish somebody.
White: I couldn’t agree more with that.
O’Hanlon: Go ahead, yeah. So, Don, do you have maybe some thoughts on how organizations can use the assessment information after it’s completed? I mean, you know, we’ve made it clear that organizations really need to consider this as kind of an ongoing thing, and I think it’s super important that organizations do understand that. But, you know, maybe after the first assessment, what can they do with that information?
White: Well, the big thing is, getting everybody having a collective understanding of what your current state is. And this is just one current state tool to get everybody collectively understood on that. Understanding your value streams and all that, those are all current state tools, but it’s a starting point to help you figure out a roadmap to move forward. It’s your starting point, and you look, and as a consultant, we can guide that, because we know kind of the sequential order in which you wanna attack these things, right? And we can help them prioritize their roadmap and their backlog of how they wanna proceed, is how I view it.
O’Hanlon: Feisal, any thoughts on that?
Mohammed: Yep. You know, assessments—Niladri’s right, Don’s absolutely right as well. Assessments shouldn’t be used for the wrong purposes. It should be guided by the real desire to improve, to move to the next level. That’s the first.
The second thing is that an assessment shouldn’t be a one-time exercise. Look, I’m at an age where I’m doing health screening every few years, I really am. And every health screening, I get a snapshot of where I’m really at in terms of my health. And then I figure out, where are the areas I need to improve upon? Where are the areas I need to watch out? I need to look at just little bits of maybe change in lifestyle habits so that I get the benefits of the health assessment results, right?
And it’s not a one-time exercise. I need to do a checkpoint assessment at some point in time in the future to determine whether what I’m doing is yielding the right results.
O’Hanlon: Mm-hmm.
Mohammed: So, it should be an organized, periodic set of assessments with predetermined, agreed plans to move from one state to a future desired state, and then those assessments take place over and over again, heading towards that general direction of improvement, right?
So, one thing we need to debunk is that we do it once, then we figure out what to do, and then that’s it. No, we need to constantly evaluate. We need to constantly assess ourselves in regards to our capabilities.
O’Hanlon: So, it’s not just about understanding or taking this assessment and understanding where you are, though, but you also have to have kind of that guidance, I you will, from folks who are experienced in the field. You know, it’s one thing for you to—Feisal, if you went to the doctor and you got the tests done and thy explained to you exactly what the results were.
Mohammed: Yeah.
O’Hanlon: But if you went out and did all the testing yourself, you’d look at all these numbers and say, “I have no idea what this means.”
Mohammed: I would not only have no idea, I would probably be alarmed, you know?
O’Hanlon: [Laughter] Right.
Mohammed: I’d think, “I should be dead by now,” right? I’m looking at all these test results—but you do need that expertise to help break these pieces of information down for you, calm you down, and say, “Look, these are the steps that you need to take,” right?
O’Hanlon: Yeah.
Mohammed: There are a lot of things that you can do, there’s a lot of things that you should do, right, but these are the things that you should focus on immediately in that journey, right? There’s an order is what Don said. There’s a sequence, there’s a realistic sequence which you can undertake, right, to gradually improve and hit towards the general direction of excellence.
O’Hanlon: Exactly, exactly.
Choudhuri: If I may, Charlene—so, this is a tool which is helping us in moving to improve the way of working. And taking what Feisal said, that it’s not a one-time activity—so, we have to look at that evolutionary process of improvement. So, you look at an assessment, you find the weaknesses in the different, the five dimensions, then you decide, “Okay, let’s look at some of the value streams where the weakness is more,” decide on what you need to do, whether you need training, whether you need automation, whether you need whatever change in the process, you do it, you take it forward, you go for another set of assessments. And you keep improving on your journey.
Because today, what is important to understand is, we have to be adapting to the changes that is coming and more to come.
White: What I would like to also—
Choudhuri: And this will help you with that.
White: What I would also like to point out is, you also need to keep the outcomes. Why are you doing all of this in the first place? Make sure that that is what you’re driving all of this toward.
The other thing I wanted to point out is, there’s the enterprise assessment and then you have, I think, a team assessment. And so, you use enterprise to kinda get a holistic picture of it, and then once you kinda figure out where you’re at holistically, you can figure out where within the organization you want to do it. Then you can assess those areas, and then you continue to assess those areas as you’re continuing that journey, and it’s a continuous process.
And Feisal, I love your metaphor with the health. I think that really resonates well, because it’s really what—it’s very similar to that. We assess all the different metrics in our body—well, just look at an organization as an organism, right? An organism that has health concerns, and we need to tweak that with continuously assessing it and coming up with a roadmap and a plan and then executing on it, and then testing our hypotheses that we’re moving toward our outcomes that we’re trying to achieve.
Mohammed: Absolutely. And it should not be a project, it should be a program.
White: Yes, yes.
Mohammed: Right? It should not be a one-time effort, it should be a program effort—weight management, health management is lifetime. It should be that way, and organizations should embrace that.
White: And the wider it’s within the organization, the better, because the folks that—if you’re just dealing with a partial part of the organization, well, those people that are going through the transformation are not working in a vacuum, they’re working within an organization that needs to kind of be on the same page, because they need to interact and collaborate with one another. If they’re not on the same page and wanna keep the old ways of working and are not open to being able to collaborate and work with these people that are trying to do more DevOps stuff, they’re gonna have limited success.
So, the broader the understanding of this journey is, the more successful it will be.
Mohammed: Agreed.
O’Hanlon: That’s a great point, that’s a great point. And, you know, Don, just to kind of expand a little bit on that and the need for having somebody who’s kind of an experienced person who’s been in the business for a while to kinda help them through the assessments—I would imagine that it not only helps them understand where they are, but also kinda help them build a roadmap for where they need to go, and also show them the areas where they’re in need of most improvement and, from a culture perspective, I imagine that that is super important for organizations.
White: Absolutely, yeah. Being able to prioritize your plan of attack and know where you really need to focus your energies, but being able to assess your actions is just as important. Because you have a hypothesis that you’re doing these things that are going to move you closer to your goal, but after you do those things, you want to validate your hypothesis by assessing.
O’Hanlon: Right, right, right. Alright, and offering up areas where maybe the organization hadn’t thought—they would not have thought on their own. So, interesting stuff. Alright.
Hey, guys, I would love to have this conversation for hours, and I know we could, but we are running up against our time. But I do want to thank Feisal, Niladri, and Don for such a great conversation. I wish you guys the best of luck with ADOC, and I think it’s gonna be a great success. You guys are awesome, you’re so well versed in the DevOps space and you really know your stuff, so considering the fact that you guys were involved in creating it, I know the industry is definitely gonna get a lot of value out of ADOC.
So, thank you, again, and thanks so much for getting on the Zoom with me and talking about it and about the market in general. So, thanks again—it was great to see you guys.
White: Thank you. Pleasure meeting you, too.
Mohammed: My pleasure. Bye, guys.
Choudhuri: Bye.
O’Hanlon: Alright, everybody. Please stick around. We’ve got lots more coming up, so stay tuned.
[End of Audio]
Suresh GP is the Managing Director of TaUB solutions and Regional leader of BRMI (APAC). He has more than 15 years of experience in IT Service Management, IT Governance, Program Management and Organizational Change Management. He is regular blogger and International speaker at itSMF UK, itSMF Australia, itSMF USA, itSMF Finland and itSMF Norway. He is a Certified DevOps, Agile Service Manager, Business Relationship Management Professional, PMP, CGEIT, ITIL V2 Service Manager, ITIL V3 Expert, ISO 20000 practitioner. He was awarded the ITSM contributor of the year by itSMF Singapore in 2013.
