I had a chance to sit down with Michael Azoff, principal analyst for Ovum. Ovum just released a set of matrices that Michael was the author of on ALM, DevOps release management and one on Agile project management. Taken together they provide some great insight into the entire software lifecycle management space.

I had a chance to view the reports before speaking with Michael and you can too courtesy of IBM. Here are the links:

1. Ovum Decision Matrix: Selecting a DevOps Release Management Solution, 2016-17
2. Ovum Decision Matrix: Selecting an Application Lifecycle Management Solution, 2016-17
3. Ovum Decision Matrix: Selecting an Agile Project Management Solution, 2016-17

Here is my conversation with Michael. Below is a transcript so you can follow along if you like as well.

Here is the transcript of our conversation:

Alan Shimel:                Hi, this is Alan Shimel of devopsy.kinsta.cloud.  Very happy to be joined today by Michael Azoff, principal analyst at Ovum.  Michael, welcome to DevOps Chats.

Michael Azoff:             Thank you very much, Alan.  Pleased to be here.

Alan Shimel:                Thank you.  Michael, you – well, Ovum, but you as the author, just published several Ovum decision matrices – I guess plural of “matrix” – in the last day or two, one of which was on DevOps, Agile, et cetera.  For our audience who may not be familiar with the Ovum decision matrix, can you explain kind of what they are, who the audience is, what they’re about?

Michael Azoff:             Sure.  Yeah, absolutely.  So the Ovum decision matrix is Ovum’s product comparison methodology, where we compare products side by side, across three dimensions – one is the technology; the other is enterprise fit; and then we use the matrixes – the result is displayed as a bubble chart, so the size of the bubble is the marketing, so those are the three dimensions.  And I cover the software lifecycle management at Ovum, and we published three Ovum decision matrices covering the whole life cycle, so one was on application lifecycle management, one was on Agile project management, and the other, DevOps release management.

Alan Shimel:                Fantastic –

Michael Azoff:             The ALM – sorry, do you wanna ask a question?

Alan Shimel:                No, no, I said that’s fantastic.

Michael Azoff:             Yeah.  It’s not common for us to – for one analyst to do three at the same time – I think I’m the first to do that – but it made logical sense because I’m covering the lifecycle from end to end.  The ALM one we’ve been – we’ve had a cycle of these decision matrices for many years, but this is the first one that spans not just enterprise IT but also engineering manufacturing, where, because of the exponential rise of software content in advanced products, there is a very, very clear need for managing that complexity and a role for ALM.  So this covered, for the first time, the engineering side as well.

The Agile project management was then a subset of the ALM one and focused purely on enterprise IT, so that’s the original scope for ALM.  Calling it “Agile project management” because it’s very much about the management at a high level, and then using – typically, organizations use a lot of open source tools for code development, so that made logical sense to name it in that fashion, with Agile being now so typical and common, but many of these solutions are generic and allow any kind of process to, from traditional stage-gates through to Agile, allow that coverage.

And then DevOps release management, which has been around for some time – I think the first report where we looked at products was in 2010, but, this year, we’re seeing the topic has become really, really hot, and it was timely to do an Ovum decision matrix – in fact, the first proper decision matrix on solutions in that sphere.

Alan Shimel:                Got it.  Let’s talk about some of the companies and what – well, before we do that, Michael, can you give our listeners a little insight into how this was conducted?

Michael Azoff:             Sure.  So the original matrix terminology, I think, stemmed from the fact that we send the vendors – we invite vendors to participate.  We send them project criteria.  If they fit the criteria, then they’re in.  It’s, essentially, we’re looking at the top players in the market.  I also tend to invite some of the more interesting and innovative smaller companies to participate as well because I think that that’s of interest to the readers.  But, generally, it is the top market leaders who are participants.  So we send them a spreadsheet and that’s their sort of matrix.  It’s – has the 200 to 400, which we’re actually trying to reduce the number of questions, rows, that we have in the matrix, but the vendors fill these out.  We then have a briefing with the vendors.  We have a demo of their product, and we also delve into their marketing approach and we look at their financial numbers.  We look at their approach to customer service.  So it’s very in depth, it’s multi-dimensional, as I mentioned, and, on that basis, we then produce the decision matrix bubble chart.

Alan Shimel:                Got it.  I don’t know if we wanna name names, Michael, but let’s take the DevOps matrix, to start with.  What were – who – according to the matrix, what were some of the let’s call it “top performers” – is that a good term to use?

Michael Azoff:             Yeah, it certainly is.  And I think of all the – I’ve been an analyst since 2003 and conducted these type of vendor product comparisons, and I would say this has been the most contentious and the most difficult to, of all of these exercises, in drawing vendors apart.  So I ended up with quite a number of leaders who looked very sort of in that leading category.  Oh, I can – I’ll try and do this alphabetically, so, let’s see, so CA, CloudBees, Automic, Clarive, Microsoft Electric Cloud, IBM, Serena – did I mention CA? –  XebiaLabs, so, yeah.

Alan Shimel:                Sounds pretty comprehensive.

Michael Azoff:             [Chuckles] There’re a lot of vendors.  They all did extraordinarily well.  It was very difficult to draw them apart, but they all have rather different approaches to the market, and this is what is sort of delved into in the actual ODM report, so I do encourage people to actually look at the report and read carefully as what – so that’s a strength, weaknesses, opportunities, and threats analysis, which goes into quite a lot of detail and really gives the flavor of each of these solutions, in that leading category.  And then we have some challenger solutions as well, yeah.

Alan Shimel:                Got it.  And, Michael, I mean, certainly DevOps is on top of everyone’s list of hot topics or near the top of everyone’s list of hot topics.  You know, separating the hype from the reality here, just where’s the ceiling for these things?  I mean, do you see – what do you see down the road in short term – 12 months, 18 months, 24 months?

Michael Azoff:             Well, we had the first wave of the innovation in this market with support for managing continuous delivery, so, at an open source level, there was activity.  There were vendor startups that provided their own continuous delivery solutions and then, on top of that, management and orchestration.  So all of that was happening around 2011, and the larger vendors made acquisitions.  I mean, IBM acquired UrbanCode; CA acquired Nolio; BMC, StreamStep.  So very much a sort of rationalization and an anticipation of the market.  But we’re still seeing new players come onto the scene, like Clarive, for example, is an interesting one and CloudBees, the open source project – owners of Jenkins have sort of stepped into the enterprise side and expanded their offering with an enterprise-ready release management solution – so still a lot of activity.  And I would say the next step in this market would be ALM and release management tying a lot closer, and I think, from a customer point of view, it would be a useful benefit to have a single ALM solution that included these advanced release management technology in there as well.

Alan Shimel:                Yeah.  Absolutely.  Michael, where do you see – now one of the questions I always get is, with all due respect, right, these analysts do great reports, but they’re behind firewalls, they’re behind reg-walls.  One of the nice things about these Ovum matrices is I see IBM and some of the other vendors are making the matrices themselves available if you’re willing to fill out their registration form, and what I like about that is it gets this intelligence into the hands of people, whether you wanna take it slanted or whatever because you had to fill out a registration form – that’s up to you – but for people, Michael, who are saying, “How independent – how much stock do we put in these things?”  You’ve already given us a little background into the methodology behind the research, the wide spectrum of companies you’re covering, but talk about independence of Ovum and of your research here, so – I want people to understand that.

Michael Azoff:             Yeah, absolutely.  So, as analysts, our credibility is our independence, so we’re not aligned with any vendor.  As an analyst firm, we face both ways – we do work for vendors, we work also with end-user customers, and we also provider solutions for vendors and for enterprise customers.  So we face both ways – we work with all parties – but our independence is our credibility in the market, so we take a completely objective view of the responses from the vendor.  And, sometimes, the vendors don’t like what they see, but they actually find it very useful to get a completely unjaundiced and independent view because it helps them understand exactly where they are in the market.

I think the ALM one, in particular, was a bit of a shock for a number of vendors because of our re-scoping of how we look at ALM.  Some vendors completely ignore the engineering market, and we see that as one of the growth areas in ALM, so I think that caught out a number of vendors by surprise, despite the fact that we’ve been telling them they ought to look at that part of the market, for ALM.

Alan Shimel:                Absolutely.  Michael, can you give our readers what’s next?  What other matrices related to these subjects or other kind of research might you guys be working on next?

Michael Azoff:             Sure, sure.  So, personally, so my brief is the software development lifecycle management, and I cycle through three areas – ALM, mobile development platforms, and application performance management – despite the fact that the ALM one was a triplet, but it is, logically, the lifecycle, if you like.  So the next one in the cycle will be Q4 this year, and that’ll be on application performance management.

Alan Shimel:                Great.  All right.  Michael, we’re just about out of time, unfortunately.  I apologize; maybe we can talk again another time.  It was great chatting with you and I hope our readers have learned something here that can help them.  And thank you for all the research you do – can never have enough information on this stuff.  Micheal Azoff, principal analyst at Ovum, thank you very much.

Michael Azoff:             Thanks, Alan, for inviting me.

Alan Shimel:                Thank you, and this is Alan Shimel for devopsy.kinsta.cloud.