Tag: CI/CD

The software development tool needs of a two to three person team can be much different than those of a large enterprise. CI/CD is a common approach nearly all software teams establish. But what needs of a larger software organization do you adopt early on and which simply add more complexity than benefit?

Rob Zuber, CTO of CircleCI, began tackling this problem in the mobile space and then moved to CircleCI to help create what is now a well established SaaS-based CI/CD offering to software teams of all sizes. Join us on this episode of DevOps Chat to take away some valuable CI/CD lessons for your organization.

Transcript

Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today, I’m joined by Rob Zuber, CTO of CircleCI, and our topic is software innovation at scale. Rob, welcome to DevOps Chat.

Rob Zuber: Thanks so much, Mitch, it’s great to be here. Thanks for having me.

Ashley: Would you start by introducing yourself, tell us a little bit about what you do and tell us a little bit about CircleCI.

Zuber: So, I’m Rob, as you said. I’m the CTO here at CircleCI. I’ve been with the company almost five years now, and I’ll talk a little bit about how I got here once I talk a little bit about what CircleCI does. I think that’s probably the right order.

So, CircleCI, our big focus is helping every software engineering team be better at what they do. So, be better at getting software from their sort of concept and idea to value delivery in the hands of their customers. So, we sit between the collaboration element and the production deployment element by really being the delivery pipeline. So, we help you take your software from the point that you’ve, again, had an idea, started to write, to making sure that works well, making sure everything everybody’s doing is working well in concert and getting that into a production environment where you can start to realize customer or business value confidently, quickly and being ready to move onto the next thing and to drive value for your customers.

Ashley: So, this is a pretty big space, a lot of folks in it. I mean, people can download Chef or Puppet or Jenkins and—you know, we could go on and on about different tools within sort of that CI/CD pipeline chain.

But tell us a little bit about where you uniquely fit into the market.

Zuber: Yeah, so, a lot has changed in the market over the time that I’ve been here, unsurprisingly—not just at CircleCI, but in the market in general. We’ve seen a lot of people come and go, a lot of companies either snapped up into larger organizations, but maybe they weren’t going to be focused, necessarily, on CI/CD standalone or just not reach sort of critical mass as an organization.

And the market has also shifted, and what we see now is a lot of people trying to maybe add CI and CD as an extension of something that they’re doing, but not treating it as the core business that they focus on. So, that’s the real thing that separates CircleCI is our focus on CI and CD as a core of what we do. Every day when we get up and get to work, that’s what we’re focusing on, and not tying you into—you know, you’re using this CI and CD capability because it’s part of this other package that you maybe want or don’t want, and really being the choice in this space and giving developers the opportunity to pick the best tool that they want in every different part of their flow, if that makes sense.

Ashley: Let’s peel back the onion a little bit more. I know that—I mean, it’s a very complex process and CI/CD and all the tools you might use, if I understand correctly, you don’t take a “let’s try to be everything to everybody,” but more of a platform approach with workflow and then you can plug in various tools as part of that workflow package and you pack things into orbs for configuration management, et cetera.

Can you give us a little bit more detail and meat on the bone about that?

Zuber: Yeah, that’s exactly right. Well, we believe it’s kind of hard to be wrong about this, I guess, that our customers are software developers, right? And so, they are more than happy to create the capabilities they need if we give them the tools to do that. And orbs is one of those such tools.

So, it’s basically about realizing that many of our customers are very capable, they’re solving very similar problems, and giving the opportunity to share that, to reuse pieces and then to build. Some of those problems that they’re solving are building integrations with best in breed components of things that we don’t necessarily provide, right? So, that might be something like, I guess, anything from vulnerability scanning or connecting into security systems through to how you do integration with your eventual deployment target, right?

Ashley: Mm-hmm.

Zuber: And rather than trying to solve all of those problems and say, “Well, you need to deploy,” like, we’ll also build a cloud and you can deploy to that cloud, or we’re also gonna build all of the tools for all of these parts of your system. We basically give you a really central and easy to use pipeline, effectively, or workflow is the term that we use for it, and the ability to plug into that, those capabilities, right? So, by leveraging orbs which our partners build, you know, to get access to their systems, community members have built, and we’ve built, basically to give you access to the richness of the ecosystem, if you will, using that to really tap into those additional systems, but the ease of central management of all that. So, you have great clarity about what’s happening in your delivery process.

Ashley: And I understand congratulations are in order. Recently, you had your series D, 56 million led by Owl Rock Capital and NextEquity Partners—congratulations.

Zuber: Thank you very much. Yes, it’s very exciting.

Ashley: So, with great power comes great responsibility. What are you gonna do with that money? [Laughter]

Zuber: Yeah, that’s a great question. I mean, honestly, a lot of it is very much continuing on this trajectory, right? Like, we believe in what we’re doing, we believe we’re doing the right things and we have the success to show for it, but in every one of those areas, we can do more, right? So, we’re continuing to invest in engineering and product development, clearly to add capabilities. I mean, part of that is really building around those types of integration models like orbs that we talked about.

Also, taking one of the key areas that we’ve really had—well, I guess just through our success over the years, because we’ve been around for about eight years as a company—we’re sitting on a lot of deep understanding of how software engineers are building. We have thousands and thousands of customers building tens or hundreds of thousands of projects. And so, we see that, right? We see the flow, we see what works for them, what doesn’t work.

We see whether they—basically, the rate at which they’re able to deliver software, the kinds of tools they use, how the ecosystem is supporting them or not, and being able to take all of that rich data and turn it back to our customers as value for them to really help them be better. Like, moving away from just a tool—I mean, a great tool, but really, a tool that not only moves your code through, but helps you understand how you could be doing better and where your opportunities are for improvement. Or, honestly, at the other end, it’s always worth mentioning, if you’re doing great and you can invest your time and energy in something else, because things are really stable and working well, right?

So, that’s a place that we’re making some investment for our product with that perspective, and then again from a, I guess, the extension that we talked about and just investing everywhere that we’re already investing in core CI and CD. I mean, over the time that I’ve been with CircleCI, the entire process of software delivery and how we package and build product has fundamentally changed, and I expect that to continue to happen. I mean, the pace at which we move in this market is a little crazy sometimes.

And so, we need to continue to evolve to support how software development is getting done in order to be, really, the best tool, as we are today, for software delivery.

Ashley: And I should know this, probably—are you running the jobs that you’re seeing in the cloud or is a lot of this on prem for customers? How do you get access to that [Cross talk]?

Zuber: Yes, great question. So, we offer both. We have a cloud platform, which is, the majority of our customers use that, and the majority of the builds that we’re running are on that platform. We do have an on-prem solution as well for very large customers with complex cases or specific needs where they’re interested in deploying the software and operating it themselves.

Yeah, and so, one thing I was gonna say about that change, and tying it back to kinda orbs and being able to integrate with other systems that we’ve talked about, we’ve been evolving to support the change in the market, but also evolving to have a more flexible platform because we see this constant change, and not wanting to play catch up or try to stay ahead of it all the time, but rather, create more flexible tools that continue to evolve with changes and how people do software development and give our users and our customers the ability to use the platform in ways that they see fit or best meet their needs as we continue to evolve.

Ashley: That certainly makes sense why you focused around the orb strategy and investing in CLI and APIs. Are you also focusing on onboarding?

Zuber: Absolutely, yes. So, I think the key to this for everyone is time to value, right? Like, CI and CD can be a complex system—

Ashley: Mm-hmm, very.

Zuber: And it depends a little bit on the size of your org. But, as I said, when we talk about those small little companies starting out, I mean, they just want something that works that will get out of their way. And so, it’s that sort of gradual growth with you as a team to meet you where you are.

So, if you have a simple application and you’re really focused on finding product market fit, then it should take as little effort as possible to get up and running and just make sure that you’re delivering with confidence so you can focus on those iterations of your early—I always talk about it in terms of companies like startups, but it might just be an early project, right, in a larger organization. Those small new projects are a great place to try out new capabilities, new tools, new processes.

And so, supporting you to get up and going and realizing that value very quickly, but again, as you get to large organizations, you need to be able to roll in more and more capabilities. But getting up and going quickly and really getting those moments where you recognize, “Okay, this is actually changing how I’m working and this is really valuable,” that needs to be very simple, and so, we’re constantly investing in making that as simple for as many different kinds of teams, of projects, code bases, tech stacks, again, as they change, we want to cover as many of those as possible.

Ashley: You’ve talked about startup companies and startup projects, but I know, also, you focus on enterprises. Those can be very, very different—drastically different in terms of customer requirements, issues, scale that you have to solve. What are some of the challenges that you’ve both run into, encountered, and also feel like you’ve addressed pretty well in those two different communities?

Zuber: Yeah, that’s exactly right. I mean, I like to think of it as a continuum. I don’t know, maybe that just makes me happier than two really distinct communities, right? And I think that one of the things you see in enterprises that are succeeding is seeing the types of approaches that smaller companies take, right? But, of course, you have to take that and figure out how to make it work at your scale.

Ashley: Mm-hmm.

Zuber: To your point, we absolutely have a very large number of very large customers, about a third of the Forbes Cloud 100. So, really large, influential, but I would still say—that particular group, tech forward companies. Meaning, coming from a background where their software development was always central. You mentioned transformations previously, so we also play in a lot of those. But it’s a different space, right, of companies that are trying to reimagine how they build versus companies that have grown to scale, sort of always thinking about software development.

But many of the challenges are similar, right? Just the pure scale. And so, that scale comes in multiple different forms. It might just be the total volume of jobs that you’re running, and the growth of our cloud platform over the years has been remarkable, and we’re constantly working to figure out what’s the next order of magnitude, the next order of magnitude, how do we continue to support that as we just continue to go through this growth, which is fantastic, but also exciting, I would say, from an engineering perspective.

Ashley: Mm-hmm.

Zuber: So, there’s kinda pure scale on just a job perspective, but the things that get more interesting and that we’ve really invested in are, for example, the complexity of workflows, right? So, if you have a lot of different systems that all need to be either validated together or have dependencies that need to be understood as they flow through the CI and CD process, or maybe a large set of validations that you do beyond just testing. So, that might be—again, I mentioned security scans earlier, linting. You know, all of the kinds of gates and processes that get put in place as companies get larger and being able to manage those independently against a very large code base with a large number of contributors.

Again, that might not be total number of jobs, for example, but the complexity of those definitions scales itself, the number of developers contributing—that scales. And so, orienting the product around those kinds of scaling points has been something that we’ve had to do as our customers have grown, as we’ve brought on large customers where we’ve really invested a lot of time to make sure, particularly in a cloud environment, which is where—there’s an additional layer of value, for sure, for our customers, because they don’t have to think about any of the management, right? They come in with very, very large workflows, and the capacity is all available. And when they’re quiet, they don’t have to worry about additional machines being online or wastage or anything like that, because we’re managing all that for them.

Ashley: Mm-hmm.

Zuber: And so, doing that for this large blend of customers, some of whom are quite big and driving a lot of work, is something that we’ve really figured out.

Ashley: Well, thinking about, you mentioned continuum, which is a great way to think about it, too—why don’t you share with us a couple lessons that you’ve seen customers learn, maybe you’ve learned yourself, running your own product development and software development process. As people go from that continuum of, “I’m just getting involved in CI/CD,” using a product like yours, obviously, there’s great features and stuff, we all know that, that you all have—but what are some of the lessons that they learn as someone goes from, “I’ve got a development team, we’ve just implemented CI/CD,” kinda getting up to this point? The next scale issue that we tend to run into is X and then kinda what happens after that.

What’s that maturity lifecycle look like?

Zuber: Yeah, I really like the word maturity that you used there. Because I think that a lot of times, when we think about this, we think about the growth of companies at scale, we spend a lot of time talking about tools. And of course, you know, it’s easy to look at CircleCI and what we do and think about it as a tool. But ultimately, the biggest challenges are gonna be shifts in culture, right? So, process how you think about getting work done, how people interact, definition of roles and responsibilities.

I mean, everybody starts out in a—well, everybody—when you start a company in this space as a tiny little, maybe you’re like three people, five people, whatever, roles tend to be pretty loose, because everybody’s just trying to get done whatever needs to get done, right? And one day, you’re the DBA and the next day you’re the barista making sure everyone else has enough coffee to get work done, right?

And as you grow, there’s parts of that that you really wanna keep—absolutely, right? Like, the focus on the mission and being really clear about where you’re trying to get to. But, it becomes more challenging for everybody to do everything, right? You actually need to have some clear boundaries and understanding.

So, that shift in culture that is like—what are the parts that we keep? Or one of the ways that I often describe it, and we, you know, for our own part, as I said, have gone from about 20 when I joined to 250 and still growing. One of the things that I always look for is what was the intent, and then how do we implement that now at this next level of scale?

Ashley: Mm-hmm.

Zuber: And I think that applies to your tooling and your process and everything else as well, right? So, we have always been a very transparent company. And so, transparency when you’re five, 10, 20 people is usually about just kind of making sure everyone knows everything that’s happening. But when you get to 250 people, it’s about really being clear about what’s happening, but being able to define that message and convey it in a way that’s not just this firehose of information or an overload for everyone, right?

And I’m trying to shift that back to sort of technology and where you were going with this—even though I believe it’s ultimately, it’s these underlying cultural pieces. When I look at people building software, I see, for example, people looking to large organizations—you know, Google, Facebook, Netflix—and saying, “Well, they build software like this, so that must be right.” And again, it’s important to look at the intent, right?

So, yes, you probably wanna have a CI and CD pipeline, because it’s gonna get a lot of stuff out of your way, but I would guess that when you’re starting out, you probably wanna have a very simple application, because you don’t know yet what your business is, whether you have a business, what the fit is.

Ashley: [Laughter]

Zuber: So, this is like, this argues for the monolith, right? Everybody, ultimately—not everybody—many people ultimately say, “You know what? The monolith isn’t scaling with our organization. It doesn’t match our ability to follow our intent. Our intent might be autonomy or ability to work on disparate areas of a product separately, stuff like that.” That works, actually, really well in a monolith at a small size, because you have so much shared understanding.

Ashley: Right, right.

Zuber: And at some point, maybe you say, “Okay, this isn’t working any more, we’re gonna start breaking out services”—kind of a natural evolution that many companies go through. But, again, people look at that at the beginning and say, “Oh, well, that’s where we’re gonna end up. Let’s just do it out of the gate” and bring in a lot of overhead and complexity that’s unnecessary for their size, and ultimately end up, I would go so far as to say failing, because they create sort of a technical overhead or burden that’s not in line with where they’re at as a business.

Ashley: Mm-hmm.

Zuber: And so, if I were to pull out the underlying lesson in all of that, it’s like—find the intent in what you’re doing and then determine the implementation that’s right for your scale. And then, I guess the other big lesson if you’re going through scale at a high enough rate, recognize where you are on that curve and recognize that it’s changing, right? Because it’s easy to think that things are gonna stay the same and the system that you have is gonna be a great system for a long time.

But if you’re growing quickly, this applies to your software, it applies to your team structure, to how you manage your organization. You need to be aware of that change.

Ashley: Yeah, I believe that when we talked about developing software, what organizations are really doing is learning how to create software, and that’s what we—or the team, the organization is in the process of doing, and then there’s that evolution on the maturity curve.

So, you, whether you’re the QA person, the DevOps person, the developer, the architect, et cetera—it’s about all that function so that you can think of it as running a football play or a soccer play or a baseball play. You’re learning to function and operate as a team to create software. Do you agree, disagree?

Zuber: Yeah, I think it’s quite on point. And the reason that I think this is really fascinating, I know this is very—speaking of side tangents, my earliest days before I got really into software development, I actually spent in a factory. I was a process engineer. And the great thing about a factory is, we would build tens of thousands of the same thing, every day. And so, refining the process was about refining the process for building the same thing.

But in software development, you basically never build the same thing twice, right? I mean, you might get hired into another company and someone’s like, “It’s great that you are here, because we need you to build basically the same thing that you already built.” But it’s this constant discovery, right? You’re constantly in a design phase of trying to figure out how do we go about building this thing that we’ve basically never done before? And of course, there’s prior art out there and all those things, but we have to map that then into our specific domain, our specific team and constraints.

And so, a lot of that is figuring out—okay, we know that we need to do a thing and we have some basic sense of that, but how do we map that into our process? And you certainly described it as our model for building the software as opposed to just thinking about building a thing.

Ashley: Well, we’ve run out of time. Rob, we should just grab a beer or a coffee and finish this conversation in about three hours.

Zuber: I’d love to. Perfect.

Ashley: [Laughter] Sounds great. Well, thank you for being a guest on the podcast. This is Rob Zuber, CTO of CircleCI. Appreciate you joining us.

Zuber: Yeah, thanks so much for having me. It was great fun.

Ashley: It’s been a pleasure. And also, thank you to you, our listeners, for joining us. This is Mitch Ashley with devopsy.kinsta.cloud, and you’ve listened to another DevOps Chat. Be careful out there.

— Mitchell Ashley