This DevOps Chat features Scott Willson of CA Automic talking about CD pipelines and the modern software factory. The concept of a software factory may be new to some, while others may have heard the term but not really thought about what it means and still others may be very familiar with the term and practice it at their own job. But the idea of a software factory utilizing a pipeline for continuous delivery is at the heart of DevOps.
This is a really great chat that I enjoyed participating in with Scott. I hope you find it as such too.
As usual, the streaming audio of our conversation is below, followed by the transcript of our conversation.
Audio
Transcript
Alan Shimel: Hey, everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another episode of DevOps Chat. Today’s chat features my friend, Scott Willson, of CA Automic. Scott has been a frequent guest on DevOps Chat and the reason is he actually has real, usually, good stuff to say and he’s an interesting fellow to speak with. Scott, welcome again to DevOps Chats.
Scott Willson: Thank you, Alan. As always, glad to be here.
Shimel: Pleasure to have you here, my friend. So, Scott, we are going to delve into something today that maybe a lot of people in our audience are really familiar with and will like the drill-down, but there may be people who hear the terms but they’re not quite sure what it means. And so what I wanna talk about today are pipelines and especially when in terms of continuous delivery pipelines and how this fits into a concept – really, CA has coined it and led with it, but it’s the modern software factory, right, for enterprises. It’s a factory methodology, almost like an assembly line, if you will, of developing and deploying software and all that that entails.
So why don’t we start with basics, Scott, if it’s okay. What is – when we talk about the CI/CD pipeline, how would you explain that to someone?
Willson: Excellent. So the CI/CD pipeline is basically just the concept or the metaphor of taking any change, right, and this is any change that a developer has coded – a configuration change, a change to the data, and, currently, any change to an environment configuration – needs to go back to a starting point and then kind of run the gauntlet, as it were, where it is vetted, it’s tested, warranted, and then made available in production environments. And there’s a whole sequence of things that need to occur, and that sequence is referred to as the “pipeline.” And pipelines are often broken up into stages, from a logical perspective, but in those stages are specific tasks that have to occur in sequence, to have your software, ultimately, be ready to be safely deployed and delivered to production environments.
Shimel: Yep. It sounds great. Now, Scott, one of the things we’ve seen is, for instance, with Jenkins 2.0, the concept of pipelines and CD pipelines has really, I think, taken on a new gravitas, if you will, within the market. People are now getting serious about it. What about at Automic? And, beyond Automic, the rest of CA even? Where does this whole pipeline thing fit in?
Willson: Excellent. So, you know, Jenkins really is a continuous integration tool. It’s meant to run a sequential set of all-or-nothing tasks – I mean, “all-or-nothing,” that if step three of five were to fail, then the idea is the whole build would be marked as having failed. And they realized that, “Well, you know, maybe we ought to extend this to a continuous delivery thing, ie. move ourselves out of just building software and trying to push it to other environments. Now, of course, there are some challenges with this, so lots of vendors, including CA Automic’s ONE automation platform – and there’re several other vendors – have jumped in the space to help orchestrate the continuous delivery pipeline, continuous delivery kinda picking up where CI ends.
And that’s really where the CA Automic release automation comes in, where it is set to orchestrate and automate that pipeline from where CI stops, where the build basically ends and deposits thing in a secure repository, and then it helps move it all the way through to production. And the thing that CA, as well as other vendors who have entered this space and provide tooling for release automation, is that you’re trying to focus on having the same automation mechanics run in every environment, from dev all the way to production.
And, indeed, that is one of the formal tenets of continuous delivery – to use the same automation mechanics. If you don’t do that, you’re still throwing things over the wall, right? And you’re missing an opportunity to test and QA and vet the automation mechanics themselves so that you can have high level of confidence that they will work in production because they themselves have been tested out dozens of times prior to that new release or that new change that’s being published.
Shimel: Fair enough. Fair enough. Now, Scott, I guess this sort of begs the question, in order to understand, right, how powerful this concept is, can you give us kind of a little historical perspective? You know, what happens before pipelines? Before this concept of a software factory?
Willson: Well, right. Things were kind of done manually. You would use Jenkins to run your builds, not just to compile your little change, but to recompile all the other pieces that it integrates with. You know, integration. The integration part of continuous integration. And make sure that changes that have been made aren’t going to break the aggregate app. And so, once that was done, it was really handed off to QA, and QA would do their thing. There were all these manual handoffs. Spreadsheets were being used to track the progress or charts. Microsoft Project and other PM tools were used to help migrate and manage, rather, the path to production, with a lot of manual instruction sets.
I, previously, worked in a previous life, where production environments or staging environments would be handed a set of instructions that operations team had to follow. As you can imagine, there were always problems. There was always some little step, some little configuration thing, that was missed in those instructions. And so there would inevitably be errors. And so it was just very error-prone and failure-prone, actually, I would say, so CD is the evolution of this, of trying to move things in an automated, repeatable, predictable way, and a safe and secure way as well. And that’s kind of what brings us into this market.
And I would say, you know, I was going through some of my old notes. A lot of people don’t know this, but, prior to Automic being acquired by CA, Automic was the one who really started this ARA market. And we had approached Gartner and Forrester, trying to explain to ’em that there was actually a need for this market, and I won’t reveal which analyst, but, you know, a conversation like, “Well, what is this thing you’re talking about? This need to automate?” And we said, “Well, we like to call it ‘application release automation,’ ” and the term stuck with the analysts. So what the analyst call “application release automation,” funny enough, actually came from us, as we were starting down this journey. And, in fact, when I started, I had – with Automic, and I was part of the group that helped build this whole thing out, is that – it was funny – you’d go to do meetings and I heard, again and again, all over the world, literally globally, Alan, I’d hear this comment – “Wow, we didn’t know anything like that existed.” So this is kinda where we started.
Now, within two years, that changed. Within two years, there were several competitors, Nolio, UrbanCode, which both, like Automic, were acquired. And the world just kind of understands this whole concept nowadays, of continuous delivery automation. But, when we started out, this was new. And, you know, Automic actually was at the forefront and named this market, so I thought that was – that’s kind of cool to know.
Shimel: Absolutely. Absolutely. You know, Scott, I was just sitting here, thinking, you know, we’re both around in this business a long time, and so, in the late ’90s, dot-com days, I helped start a company called “Interliant,” and we were one of the early ASPs. Right? And we did Lotus Notes hosting and PeopleSoft hosting, Oracle apps, some other defunct software that’s not around anymore, but this was before cloud. And we had to customize every installation, every – you know, ’cause the apps needed to be customized. And I’m just thinking back to how we were developing and deploying back then. And, you know, it seems like 100 years ago, not 20 years ago, first of all, but it’s amazing when you think about the changes we’ve seen in such a short time.
Willson: Yeah, but you know what? It’s funny, Alan. So I remember those days as well, the pain of working weekends and doing massive deployments, but what’s funny now – we see all the things that have changed, but a lot of things are still remaining the same. In a lot of the large enterprises I visit, Alan, they’re doing this DevOps thing, the continuous delivery thing, with their new cloud-native-based apps or even they’re taking a big app, like their online banking app for a bank I won’t name, and they’re rewriting it in Node.js. Okay? Great. They do this since it’s in the cloud. They’re able to do all these new things – Agile, DevOps, continuous delivery – but they still have all this infrastructure in apps in the background, what I call “classically architected,” right? Your IIS, JBoss, WebSphere, WebLogic, so forth, client servers. You then also have your legacy – your mainframes and maybe large, packaged systems, like Sable and so forth.
So, even though they’re doing a lot of things – we’re hearing a lot of messaging towards the new ways to do stuff – a lot of enterprises I visit still haven’t gone on that journey ’cause they haven’t quite moved so many things in the cloud and they’re still doing the way things, Alan, you and I saw done in the ’90s.
Shimel: Mm-hmm.
Willson: In some ways, a lot of it’s kind of this panic. This fear of like, “Well, you just never know, man. If this thing goes down, there’s so much money at risk and at stake,” that they’re slow to change. And I would tell you that, in the years since I first started my journey during all the stuff, some eight or so years ago, that it’s – even just last year, it was kind of – I don’t wanna say refreshing, I don’t wanna say surprising, but it was certainly eye-opening to see how many large enterprises fundamentally still do things the same way they did in the late ’90s.
Shimel: Yep.
Willson: And they’re asking, “How do we go forward? How do we do this?”
Shimel: Agreed.
Willson: So it’s not a given, all this new stuff, Alan, is what, I guess, what I’m trying to say.
Shimel: No, it’s not a given. It’s not a given. But you also have to ask yourselves, Scott, right, fundamentally, is it a survival threat or threat to the survival of these companies who don’t kinda get with it, right? And what you see with that, Scott, is, look, there may be some industries and some companies and some businesses who, frankly, it’s just not that important. Right? Important. And they could do what they want and if they don’t get with it, they don’t get with it – that’s okay. And you need to – you know, and they may never adopt these ways of doing things and it may not really be a huge effect, but I think, for the overwhelming majority of these organizations, I mean, they really start falling behind the eight ball here, if they don’t get with it, and I think that’s what we’re seeing play itself out in the market today.
Willson: Absolutely. It’s a danger if you don’t adapt. If they don’t improve their customer experiences, they could absolutely find themselves going out of business. I remember reading a lot about Bill Gates in the mid-90s and how he was just really maniacally focused and concerned with the concept that he could be out of business within just a couple to three years. And, when I was much younger and reading this, I thought, “Wow, that’s a good lesson to learn, one, but, two, I think he’s a bit over-exaggerating things.” Well, he wasn’t. Turns out, big companies, as we’ve seen since the year 2000, have vanished, and it’s very important that organizations who haven’t begun this move begin to modernize their way of delivering software. They just have to.
Shimel: Got it. Got it. And I think we’re seeing that play out, right? Where it’s kind of “DevOps or die” or whatever you wanna call it. Anyway, Scott, we’re just about out of time. Before we close up, anything news coming out of Automic you can share with us?
Willson: Well, obviously, we’ve got new stuff coming out, but I really think the new thing – it’s not that new, but one of the things I’m very excited about in joining CA is just this whole concept of a modern software factory, where – and joining CA, the Automic team is an integral part of this strategy and what they’re trying to help these large enterprises do, right? ‘Cause the enterprises, as I said, it’s just not the cloud-native stuff. They have all the classical and legacy stuff that they want to onboard for continuous delivery too and they’re struggling.
And this whole thing about this modern software factory, I think it’s a bit more of a focus on the tooling, right? It’s another layer to DevOps and to continuous delivery, is looking at assembling and organizing the tooling to get all of your platforms moving at continuous delivery speeds. And it’s definitely possible, but I’m very excited and passionate about this modern software factory and where we can help CA take it. And I think that’s really a good mindset to have, is, yeah, this whole idea of you just plug in your inputs in one end, where your destination’s on the other end, and have things just happen and get you to the promise land.
Shimel: Yep. I hear ya. All right. Well, Scott Willson, thanks for being this episode’s guest on DevOps Chat. Hope to have you back soon. I know you’re working on some stuff, Scott, and, at the right time, we’ll have you on here to brief the world on it, but, until then, it’s always great catching up with you. Lots of success over at CA and Automic and we’ll talk to you soon.
Willson: Sounds good, Alan, and, at this time of the year, I will just say, good luck on your bracket. [Laughs]
Shimel: Yes to everyone out there. Okay. Hey, Scott Willson, CA Automic Software, your guest today on DevOps Chat. This is Alan Shimel and you’ve just listened to DevOps Chat. Have a great day, everyone. We’ll see you soon.
— Alan Shimel
