devopsy.kinsta.cloud

Category: Enterprise DevOps

  • TechStrong TV: How to Build and Refine Your DevOps Skills with OpsCompass

    TechStrong TV: How to Build and Refine Your DevOps Skills with OpsCompass

    The role of DevOps is constantly evolving as new technologies and trends keep emerging and reshaping the business. It is critical for DevOps engineers to continuously learn about the latest tools, best practices and technology, and be able to adapt to change.

    There are many ways for DevOps teams to refine their skills in order to become high performers. In this TechStrong TV interview, OpsCompass Co-Founder and CTO John Grange and Engineer Amy Wall join Mitch Ashley to discuss upskilling and how to build and maintain your software, development, cloud and technology skills in the fast-paced world of cloud. John provides a look into the future and where our skills need to move next, while Amy graciously shares her journey into software development and some valuable lessons learned along the way.

    OpsCompass is a leader in Cloud Security Posture Management (CSPM) offering an enterprise SaaS product that provides security and compliance management in cloud platforms.

    The video interview is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Mitch Ashley: I have the pleasure of being joined by two really interesting folks. We’re gonna have a great conversation, I’m looking forward to this today—Amy Wall, who is an engineer with OpsCompass, and then John Grange, CTO with OpsCompass. Welcome to you both. Good to be chatting today.

    Amy Wall: Thanks. It’s great to be here.

    John Grange: Yeah, thanks for having us, Mitch.

    Ashley: Absolutely. You bet. Let’s do this—how about if we have you both introduce yourselves. Maybe, John, if you go first, introduce yourself—tell us a little bit about OpsCompass and what the company does, too, and then hand it over to Amy and you can tell a little bit of your background.

    Grange: Excellent. Again, I’m John Grange, I’m co-founder and CTO of OpsCompass. OpsCompass is a really powerful SaaS product that does security and compliance management in the cloud. And what that really means is, you know, we’re really helping DevOps teams, CloudOps teams get the visibility, have the intelligence and the control they need to run a successful cloud operation.

    You know, my background really plays a lot into what OpsCompass does and kind of the cloud security space. I’m a career entrepreneur. I’ve co-founded a number of companies, from top five global Microsoft ASP.NET hosting provider to a health and wellness SaaS product. So, I have a lot of experience, and our team has a lot of experience in kind of the infrastructure security compliance space. The cloud, obviously, was a big game changer, because a lot of the things that we used to deliver through a variety of different software products or hosted solutions inside data centers, we can now build software to easily do it in the cloud.

    Ashley: Mm-hmm.

    Grange: So, that’s kinda how I got into the business through kind of that hosting and infrastructure space, and now here we have OpsCompass, which is leading the charge with helping customers with their security posture, their compliance—all these different aspects of running or having a well-run cloud.

    Ashley: Sure is a fun and interesting space to be in and I think we’ll get into that, too. Amy, how about you? Just a little bit of introduction, and I think we’re gonna get into more of your story, too.

    Wall: Sure, yeah. I’m Amy Wall, I’m a DevOps engineer at OpsCompass. I have the pleasure of working with John Grange, working under John Grange, and kind of forming this exciting new product, OpsCompass. And so, I’m kind of—where I’m at in my career and on my cloud journey is really in a space where I’ve kind of come to it from a software engineering perspective. I’ve been a software engineer for years now, and you know, been going through the steps and kind of getting the skills under my tool belt, upskilling on the cloud path so that I can become a full DevOps engineer for OpsCompass. So, really been learning with OpsCompass and growing with OpsCompass as they sort of kind of figure out and carve out their space in the cloud infrastructure.

    Ashley: Mm-hmm. Isn’t that nice when you work with, you find a company that’s really compatible with what your goals are and what you wanna learn with where you are in your career? That’s awesome.

    Wall: Huge. Absolutely.

    Ashley: Well, that’s a little bit of what we’re gonna talk about, the upskilling is kind of a general, I guess, tag, label for it. We’re gonna explore a little bit, and you volunteered to share your story of how you got into software and how you’ve evolved into software in the cloud and DevOps, and then maybe take a look forward, you know? We’re here in December thinking about getting through the holidays and everything else going on and thinking about the new year. Maybe John can share with us some ideas on that, too, and we’ll explore some of those things about how do we prepare ourselves for the next things that are coming.

    So, let’s start with your story. I’ll let you tell it however you’d like, but you know, maybe how did you get into software?

    Wall: Sure, yeah. I always like to joke that if I told my 12-year-old or 14-year-old self that I was a software engineer or DevOps engineer, I would not believe it, at that age.

    Ashley: [Laughter] 

    Wall: But yeah, I really didn’t have a super traditional path. You know, I was a Humanities major in college. I left college and I became a content editor and then later a project manager in the marketing space. And I really enjoyed it, but kind of along the way, I got exposure to various technologies, I got a little exposure to programming, I got some exposure to IT, and over a few years, I realized that I just loved it, and that’s what I really wanted to do. And my favorite days at work were the days where I was able to kinda solve, you know, software problems or technology problems, whatever.

    So, I did, like, a total career 180 about five years ago and took a U-turn, learned how to code, and my first job after that was at OpsCompass or a subsidiary of OpsCompass that OpsCompass then later purchased. And so, I’ve been working in software ever since as a software engineer. And it wasn’t until OpsCompass a couple of years ago, I think two or three years ago, when I was able to join that amazing team of engineers where I started to really ramp up my cloud journey. You know, I had become pretty strong in the fundamentals of software engineering, and you know, I’d become really used to kind of the experience of learning on the job. You know, that’s kinda the nature of software engineering itself, and it’s the nature of DevOps, and it’s really the nature, I think, of cloud, because there’s new technologies coming out all the time and everybody’s kind of learning a way to navigate the space a little bit better all the time.

    And so, yeah, I just kind of figured out how to learn on the job and continued learning on the job, and now I’m surrounded by world class cloud engineers and John Grange, our CTO, who have excellent mastery in this space, and it’s been a blast to kind of learn together with them and figure out how this entire ecosystem works.

    Ashley: Wow, very complimentary of the company. Be careful, John, she may have an ask.

    Wall: [Laughter] 

    Grange: Yeah, I know. This is—this is great. No, but in all seriousness, you know, just from our perspective, even, having somebody who’s a rock star at this point, like Amy, it’s hard to find cloud expertise. And you have to be able to find people that are interested in growing, willing to grow, have that interest—like, it sparks that fire, so that then you can enable that and you can grow a lot of your own expertise, and it’s a really powerful thing, but I think it’s something that a lot of companies, particularly like us, have to do today to stay around.

    Ashley: Mm-hmm. You really do have to invest in people and give them the right, whatever ways that they learn, the right opportunities. You know, there’s also the person side of it, too. One of the best things I learned in college from a Computer Sci professor was, he said, “Everything you know now is gonna be obsolete in a year, so you need to keep learning. You have to have a thirst, just a hunt, a desire for learning just to keep—keep going.” 

    And it seems like that’s consistent with what your journey’s been, Amy. Maybe talk about what the evolution from some of the technologies you started developing software in to getting into cloud—what kind of things are you doing today?

    Wall: Sure, yeah. So, I started where I think a lot of new coders start is, I started on Ruby on Rails, and I used it for years, and I actually still use it a lot of the time. But yeah, I started there, and then I worked in the consulting space for a few years where I was able to kind of bring to life a lot of customers’ ideas, clients’ ideas, and I loved that, that was really exciting.

    And so, from there, I kind of have just begun dipping my toe a little bit in all the cloud has to offer. I still have a ton to learn. But our infrastructure has a number of different resources in various clouds. You know, I get to work with AWS, I get to work with Azure, I get to work with Google Cloud Platform. So, you know, any given day, I could be working on any one of those technologies. I could be spinning up Lambda, I could be debugging something in Typescript or JavaScript, whatever it is.

    But yeah, it was definitely a journey along the process, but we have a team, like I was kinda mentioning before, we have a team that we all, I think, feel that we’re in it together a little bit. And so, there’s a great energy where I think we kind of all learn together, even the master practitioners will continue to kind of get new certifications as they come out, will continue to offer advice, and will continue to learn.

    And so, I’ve never really felt like I was the baby in the room, so to speak, [Laughter] or that I was less skilled than anybody else, per se. I kinda just felt like—okay, we’re all making ourselves better, sharpening our skills, getting better all the time, so.

    Ashley: Especially, it can be an advantage in a technology area that’s relatively new, because everybody’s learning it, right? Nobody’s been doing it for 20 years or whatever makes you an expert.

    I’m curious, too, did you find mentoring, kind of learning from peers, things like that were helpful, since everybody’s learning at such an accelerated pace moving into the cloud, whether it’s on the UI side or on the software infrastructure side?

    Wall: Hugely, absolutely, yeah. Our lead architect, Nick Allmaker, is an excellent mentor for us. You know, his knowledge never ceases to amaze me, and he’s really, really helpful. He’ll always—he’s always willing to take some time out of his day to spend a couple extra minutes on a pull request or help you figure something out.

    But, you know, even earlier on in my software engineering education, I remember having a teacher who told me, “Always T.S., try it and see.” [Laughter] And I think I mentioned this in a blog recently, but that’s been hugely helpful to me as I’ve moved into this space, because you know, for me at least, in learning cloud and in understanding the ecosystem, tutorials online can really only take you so far. Sometimes, you just have to, you know, go in there and spin up a Lambda, you know, spin up an EC2, and figure out what’s going on, play with it, move it around a little bit.

    And so, I definitely—that’s a big insight that I’ve learned is that just get in there, try it. A lot of the clouds, I think all of them, in fact, have a free trial where they give you a little bit of free canister to play with at the beginning. I used that for all of them, absolutely, and you just, you know, spin up resources. 

    And I think in addition to that, having a third party like OpsCompass where you have kind of that visibility into every single resource that you have going on in every single cloud can be super helpful to someone. Obviously, helpful for experts and helpful for people who are in the cloud security posture management space, but also helpful for people like me who are learning and who don’t really understand or who are kind of putting the pieces together of what changes do what and what resources are like others, how similar is an Azure function to an AWS Lambda, and having that kind of visibility into all of your clouds and all of your resources can be hugely helpful.

    So, you know, I kind of had that advantage at OpsCompass where, you know, I’ve definitely had a leg up, for sure, in being able to kinda learn this stuff.

    Ashley: Mm-hmm. Opportunity is everything. Sometimes it’s self-created, and sometimes it’s the situational company, et cetera. I’m curious, John, what’s your perspective on, because you’ve, you know, you’re obviously probably in a little different place in your career, but as yourself and folks you’ve worked with, what are some of the mentoring and kinda helping others advance in their careers have you seen to be really successful—either for yourself or people you’ve worked with.

    Grange: Sure. Well, you know, there’s a couple things that come to mind, and one of them Amy just mentioned, being able to go and mess with something, being able to go out into the cloud and play with the resource and have kind of both the curiosity, the time, the capability to be able to go do that is a really powerful thing. And in the cloud, when all of these services are changing and being—you know, there’s a couple hundred different ones on AWS—there’s literally something new every day.

    Ashley: Mm-hmm.

    Grange: So, I think that that’s a big part of it. The other thing that I learned early on from mentors of mine that I also try and share with the team is that when there’s these big trends—you know, cloud is an example, but then also, you know, some of what you see with big data and AI, et cetera, there’s the sort of application top layer trend that’s all of the robust sort of user applications for those technologies. 

    But for those to actually come to fruition and for that trend to really be powerful, there’s gotta be all this plumbing underneath that isn’t as—it’s not as glamorous and not everybody thinks about it, but as those spaces burgeon, it’s not just the mobile app that’s using AI to really change kind of directions in navigation, it’s the whole industry of sort of the infrastructure beneath that needs to serve those sorts of trends and applications. And those end up being great places to go and learn about. And there’s tons and tons of opportunities, sort of, in the wake of those big trends and the leading kind of applications and companies.

    So, I always like to, even in our product, you see a little bit of that, but I even try and tell, you know, folks on our team that we need to be looking at things that way, as these trends are big. What kind of plumbing do the customers need? What are the dots that need to be connected? What’s the kinda unglamorous thing underneath that everybody’s gonna need that we can learn a lot about and solve the problems around?

    Ashley: I’m curious, Amy, have you seen that? Has that been consistent with your experience, too?

    Wall: Absolutely, yeah. I think—yeah, per what John was saying, I do really think that there is kind of an ecosystem to cloud and we’re trying to always stay on top of it, in front of it, you know, figure out what’s going on with it and view it from kind of like that 30,000 foot view, so we can really understand it as a whole picture.

    So, yeah, I think there’s a great knowledge share, for sure, and I think that’s hugely important to figure out where is this going, what is going to be happening in 5 years, in 10 years? Where do we need to start working now so that we’ll still be relevant, we’ll still be making sense at that time? And it’s not always easy to guess, but definitely, there’s a lot out there in the industry that you can kind of get a feel for that.

    Ashley: I think we’re in violent agreement. [Laughter] The best developers I’ve worked with really knew it from the operating system, the whole software stack, the network—you know, not an expert in every piece of it, but knew how it all fit together and how to make it all work and kinda hum and get to the performance that you need or make the right decisions on how you want to, the architecture you want to use and what parts you do and don’t want to use.

    You know, another thought—you may have mentioned this in your article, too, Amy, and I’ll put a link to the article in the description—open source is another fantastic resource. You talked about getting free time on cloud services and using sampling software—it’s a great place to watch how software’s built, in a little bit different way, in an open way, open community, but using that as well can be super fantastic. And by the way, everybody’s using open source in their products and their IT systems, their applications. And so, the more you know about it, the more you can leverage it, right?

    Wall: Absolutely, yeah. I love—I just, I love the idea of open source. You know, I think it’s—I love the ideology of it. I think it’s fantastic, and certainly, we do use it. And I think open source is the place where a lot of people kind of go to exchange ideas in the cloud space right now. You know, we have a lot of—Twitter, for example, Tech Cloud Twitter is always really exciting and people…yeah, I think people are making some really interesting things right now with open source, and it’s an exciting place to be.

    Ashley: So, let’s kinda turn the lens from, you were talking a little bit about your background, John sharing a little bit of his as well—turn the lens to kinda thinking forward, right? Things have changed. It’s a different world in the last nine months, for sure, and a lot of organizations have accelerated digital transformation projects. We’ve got data in my analyst firm to back this up very strongly. 

    The move to the cloud is accelerated, and you know, not stopping. It’s gonna continue to move even quicker. And I know you think about a lot where you’re going as a CTO with your company and your product and your technology, John. What thoughts do you have as you think over the next 6, 12, 24 months kinda into the where we’re going and how that might influence what people might learn or maybe how they learn?

    Grange: Mm-hmm. I actually—that’s a great question, and I’d actually go back to the comment I made before about some of the different trends. And, you know, just like we’ve had the DevOps trend over the last handful of years, there’s a whole number of kind of portmanteaus, right, the CloudOps and DataOps and AIOps.

    And I think that as some of these, you know, Fortune 1000, Fortune 2000s start to become more cloud mature, you’re gonna start to see more of the need for, okay, not just general DevOps where you really understand the cloud platform and orchestrating and automating workloads and things of that nature, but you understand the details of AIOps, for instance, which is a much different challenge, it has lots of other issues with data issues and labeling and things of that nature. Similar issues with sort of DataOps engineering—lots of new problems, really important business solutions to be built there for organizations that are already pivoting their organizations to prioritize these things, particularly after the pandemic. 

    DevOps is all about efficiency, and now what companies are saying is, “Hey, you know, coming out of the pandemic, we’re gonna prioritize our most innovative projects higher and we’ve got to push to be more efficient. We can’t have the cloud spending overages, we can’t have these big breaches that are costly reputationally and financially. We’ve gotta get more efficient.

    So, I think that those are the big things that, coming out of the pandemic, I’d be thinking about. If I were kind of a DevOps practitioner out in the field but also what I’m thinking about at OpsCompass is we’re, you know, looking out, we’re looking into how can we provide more value to our users for some of these really specific use cases that are driving very significant value for companies that are implementing them.

    Ashley: Mm-hmm. I’m curious, too, then, without getting into customer details, but as you’ve worked with your customers through the pandemic, through economic challenges, recovery, that kind of thing—what changes are you seeing in the market with the customers that you’re working with? Are they emphasizing different things, stepping back and taking new directions or trying to get better at just the core things? What’s happening?

    Grange: Yeah. You know what? It’s interesting. From my perspective, it’s very much almost no new phenomenon. It’s almost all just an acceleration of things that we were already seeing.

    Ashley: Mm-hmm.

    Grange: So, in other words, a lot of our kinda strategic plans that we thought were maybe a year out all of a sudden became, you know, “We need them, like, yesterday.” So, I think that a lot of those trends around remote work were already in place. I think a lot of—you know, many of the projects that we’re seeing companies, you know, prioritize, put a higher priority on, those projects already existed. Now, they’ve just become more important.

    Unfortunately, a lot of these IT teams and cloud teams shrunk during the pandemic. So, they now have the requirement that they’re more efficient and everything else, but they have, in many situations, fewer resources to work with.

    Ashley: Mm-hmm. Well, though that’s hard, it’s almost always that growth comes through investing again, so at some point, that starts to rebuild on that efficiency.

    Grange: It’s the story of IT, right? The story of IT—being understaffed and overly important.

    Ashley: Yeah, I don’t think I’ve met an IT team that said, “You know, we just have too many people. I’m not sure what we do with all these folks.” [Laughter] 

    Grange: Yeah.

    Ashley: You know, it’s interesting—I agree with you about, it’s very much accelerating. Another trend I would share—let me see if you concur with this—is, you know, you have the fortunate, if you wanna think of it that way, situation of being in a company where your software is your product, right? So, you can connect the dots between work that Amy’s doing or someone’s doing a test or someone’s doing customer service or architecture or what you’re doing in your role, John. You can connect that to the business and why you want to invest in AI and machine learning or whatever the next thing is.

    That can be really hard to do when you’re in a bank or an insurance company or something else where there’s a lot of distance between what I’m doing in my job—especially in a Fortune 1000 company or above. With that acceleration, though, and I think because of DevOps and AsteriskOps being adopted across different functions, a lot of that is about driving that connect the dots to why we’re going and where we’re going there, and how to accelerate, right? So, the efficiency can be for financial purposes, but recovery can also require agility, very fast response. You know, markets change, our supply chain is gone. We gotta rebuild that in a totally different way—so, guess what? Software is critical. It’s not just a back office thing. Thoughts?

    Grange: Hey, I think that’s exactly right. And, you know, using the financials as an example, you know, many of these financial companies were spending an extraordinary amount of money on reducing their risk, their technology risk, their physical security data centers, really expensive software to secure it all. And you’re exactly right, one of the accelerating trends is that efficiency. That’s not efficient at all. Capital One now has moved—they’ve shut down, now, all of their data centers around the world. They’re completely in the cloud.

    Ashley: Mm-hmm.

    Grange: That’s kind of an outlier example now, they’re like, the only one. But they’re all—I can tell you, just on experience with our own customer base, the mid-market is moving very quickly that way, too. Because they just realized that—hey, we don’t have to, we now don’t have to build data centers, we don’t have to have these huge capital budgets. We can be more efficient, we can be more operational. We can—to your point, the agility that’s required with how quickly this technology’s changing, you almost don’t have a hope unless you’re in the cloud. You know, you can stand up your own container stack in your data center and set up a rack and buy hardware and do all that, or you can go turn on, you know, Amazon’s Elastic Container Service and go with it. I mean, it’s just, it’s a whole different ballgame. And, yeah, I think that that’s a good example, Mitch.

    Ashley: Excellent. I’m curious, Amy, your perspective on it, too. Will you—in your role when, kinda, you think about your career and where you’re moving as well as the company, how is the current situation influencing how you’re thinking about what you do next, what you learn, and how you can maybe help others accelerate their learning?

    Wall: Hmm. Yeah, I agree with both of you, that I think the pandemic has had a real—there’s a renewed sense of urgency because of it. And so, I think a lot of us have had a moment to kind of take a step back and think a little bit from a broader perspective about what our company is, what our company is doing, what we are doing, where our careers are going—all those kinds of things. 

    And yeah, I think people are having an opportunity to figure out—like John was kind of saying, John mentioned the infrastructure for working remotely was largely in place before the pandemic, but that wasn’t always true, and that wasn’t true for every company. And so, I think there’s a lot of catching up that’s happening right now, and yeah, there’s definitely this added urgency behind all of it because of the state that the world is in right now.

    Ashley: Yeah, it’s interesting. I was running IT for a while at a prior job, and we were at this beginning of—stopped thinking about it as we were working remotely or working remote and think of it as work anywhere. 

    And it seems like that’s totally apropos now, that’s—you know, if you think about forward, it’s not like we’re all gonna show up back in the office after we vet the vaccine or whatever, even if things are in good shape, you know, public health wise, there’s a lot of things that, you know, we’ve been doing this for a year, two years or whatever the time’s gonna be. People are gonna build on it and leverage that, at least I think. I don’t know if you all agree, but I think this is the work anywhere that we’re in now.

    Grange: Well, CFOs got a little taste of really, really small travel and entertainment budgets, and I think that, as those start to come up, I don’t think that the tolerance levels will be quite where they were before the pandemic.

    Ashley: Yeah, we’ve proven, right, what we can do.

    Grange: Yeah.

    Ashley: Also interesting, you know, folks have talked about, we’ve made decisions a lot faster, too, in some cases. You know, we were debating—do we use Team or Slack or whatever, you know, for a year and suddenly in a week, two days, we decided it. Like, why can’t we do that all the time? Well, maybe not always you do that, but we’ve shown we can accelerate a lot of things.

    Grange: Well, I think that another thing that happened is, I think that American companies in particular, I can’t comment as much internationally, but American companies, I think, got very good in this, in the 21st Century at buying software—maybe even before that. 

    But I don’t think that companies are very good at actually adopting and using software, and I think one of the things that the pandemic did is, like, everybody bought teams and bought all this stuff from these, you know, the big tech companies, and nobody really fully adopted it, fully implemented it. It was always just kind of another thing on the shelf, and I think that that’s another thing that’s happening right now.

    Ashley: Mm-hmm. Getting greater adoption. 

    Grange: Lots of the collaboration technology, lots of the things that we all kinda use are now becoming highly mission critical business processes that we care about. So, we’re, like, fully adopting them.

    Ashley: Just the essential tools in the toolkit versus, “Yeah, I’ll use it if I have to talk to that team” or something, [Laughter] that group. As you’re thinking about the technology stack, Amy, you know, John was talking about adopting software. I remember a time in my career where folks felt like they had to write all of it, right? I mean, yeah, they used a database system, or—you like to build software, so build as much of it as you can.

    Today—yeah, you write software, but you want to, is there a better container management system than Kubernetes? Well, let me go check it out. You know, I don’t necessarily want to write my own. Maybe I do wanna do it as an open source, but you’re really a software integrator, software builder as part of the whole process, not just a software developer. Is that how you view things?

    Wall: Absolutely, yeah. Yeah, I’m a big proponent—totally, yeah. I always say, “If you can avoid it, don’t reinvent the wheel.” And everyone says that, but not everyone always takes that to heart.

    Ashley: Mm-hmm.

    Wall: And I think a lot of the times, especially when you’re in a software team, you’re in a development team and you’ve kind of got your head down and you’re really working hard on code, I think it’s really easy to kind of lose sight of the broader picture.

    And, you know, just last week for example, we had this problem we were trying to solve where some customers weren’t—you know, there was a miscommunication between our software and some customers. And, you know, I kind of just was able to take a step back and—

    Wall: – you know, customers weren’t, you know, there was miscommunication between our software and some customers, and you know, I kind of just was able to take a step back and say, “You know, if we just think about this and maybe document it, write about it, you know, find some documentation online, you know, find a good article, perhaps, that we could recommend, I think we could solve this problem without writing any code at all.” And we did. And I think those kinds of solutions are happening a lot, for sure.

    I mean, and you mentioned open source earlier as well, and I think that plays into it to a huge degree as well.

    Grange: That’s what I was about to say Amy, is that that goes right back to what you were saying, Mitch, with the open source. And I think that’s why open source sort of is kind of supreme right now. Go find a library, go find a package that already does this, go find a service that we can just integrate APIs with, you know—let’s not reinvent the wheel.

    Ashley: I remember being on some panels of, you know, “Should you use open source?” and my answer was, “I can guarantee you your competitors are.” Talk about a source of innovation that you can get for little or no cost—I mean, yeah, there’s a cost with it, but you better leverage it, you know, [Laughter] in the right way for your company as well.

    Let’s talk a little bit about OpsCompass. I’d love to hear, maybe, some of the future directions you’re heading. You’re not announcing anything here today, but what are the kind of things you’re thinking about since we’ve talked about this evolution of where the cloud and technology and skills need to go? What’s happening at OpsCompass as you evolve and change and play a leadership role in the industry?

    Grange: Sure. Well, for us, the way we like to think about the product and the product strategy is, what’s going on in the cloud space, what are customers doing, and then how can we provide kind of visibility and intelligence across those different dimensions?

    So, I talked a little bit about kind of some different AI products and services that we were starting to see companies use more containers. We obviously—that’s kind of a number one use case for our customers today. So, we’re fairly strong there, but we’re gonna get much, much stronger.

    But another big part of it for us is the DevOps pipeline. We provide visibility into what the pipelines are doing today, but it’s becoming so much more critical to the full cloud processing companies. They’re becoming more DevOps mature that we are going to, you know, kind of completely blow out that kind of set of functionality in our product over the next year. So, we’re excited about that. And you know, our customers are really looking—you know, you can parse through a lot of the technical explanations for problems, but what they’re really trying to do is get control, and the definition of control is a little bit different than it was in the data center. And there’s some more subtleties and things to it, it’s got a little bit more texture, I like to say.

    And so, it requires a different approach and when we can combine, you know, the deep visibility into what they have with the intelligence and context around, you know, what’s happening and the state of things, that’s kinda how you end up getting control. And there’s processes and people involved and everything else, but kind of everything we do—and we kind of, you know, eat, breathe, and sleep that kind of mantra, and we’re constantly poking and prodding at the clouds, working with the clouds. We work with all three at the corporate partnership level. So, we’re just really—we’re having a lot of fun, and it’s sort of the key problem of the age right now is just the different teams, all these different people, different resources, different services. It’s just sort of the impact of this sort of scale and kinda chaos a little bit. So, there’s just so many great problems to solve around making sense of it all.

    Ashley: Mm-hmm. Excellent. Amy, anything from your perspective you would like to add or talk about?

    Wall: Yeah, I think John is just spot on. And I also think, from an internal company perspective, you know, what’s happening within OpsCompass is being mirrored by companies across the globe right now, which is that we are really kind of getting a feel for how to operate as a totally distributed team, like you were mentioning earlier. We have employees now all across the United States and internationally. We also work with customers and _____ abroad.

    So, we have this—I think we’re kind of learning along with our customers how to kind of interact with a more global ecosystem than we had before and how to kind of bring all of these, all of the water cooler conversations or all of the idea sessions or brainstorming sessions and whatever onto Slack, onto Teams online so that we can all engage.

    So, that’s definitely a change that I’m seeing internally and that I’m not, I don’t see going away when the pandemic is over, and I see it becoming something that goes on to the future.

    Ashley: Yeah, that’s very wise. I think it’s very sticky kinds of behavior changes and work patterns that will be with us for a long time to come.

    Well, I’ve really enjoyed, this has been a fun conversation talking with you both and it’s just—it’s so much fun to share real experiences that we’re having in our jobs and our careers and also doing it in a way that hopefully this conversation can benefit some other folks, you know, along the way who may have similar or maybe different experiences that they might pick up from what we’ve talked about.

    So, before we wrap up, to check out things at OpsCompass, John—OpsCompass.com, is that correct?

    Grange: Yep, you can go to OpsCompass.com, check out more about us. We have a free trial, really encourage people to go jump in, check out the product. Again, it’s built by a bunch of DevOps engineer cloud nerds for our sort of brethren, if you will. So, love getting feedback and let us know if there’s any, if there’s anything that we can help you with.

    Ashley: Built by DevOps people, for DevOps people, right? [Laughter] 

    Wall: Exactly.

    Ashley: Exactly. Well, Amy Wall, it’s been fantastic talking with you. Thanks for sharing your story and your insights. 

    Wall: My pleasure.

    Ashley: And John Grange—similarly, your insights as well as kinda where we’re headed and a fun conversation with you. Both the best, have a great holiday, and we’ll see you into the new year.

    Wall: Thanks a lot, Mitch.

    Grange: Thank you, Mitch. Be safe.

    Ashley: You, too.

  • Operations Model: DevOps, NoOps, AIOps or None of the Above Ops?

    Operations Model: DevOps, NoOps, AIOps or None of the Above Ops?

    Within an organization, an operations model must be fit-for-purpose and be able to adapt as business needs change. At the same time, changing technology necessitates an advancement in how an organization operates. Small and medium-sized enterprises are faced with the difficult choice of how much to invest in operations and which approach to choose. In light of the numerous possibilities, companies need to identify the right upgrades for their needs. For instance, should a company now using a traditional operations model invest significantly to reach DevOps? Or would AIOps better fit customers’ needs? Is NoOps an option?

    There is no “correct” answer. Each option comes with pros and cons and important budget considerations. That is why every business needs to evaluate the appropriate system for its products and its customers with three concepts in mind: cost, goals and experimentation.

    Operations Models

    Traditional IT operations (ITOps) is the process of designing, developing, deploying and maintaining the infrastructure and software components of a specific product or application. It also ensures the customer experience is delivered through traditional means of support such as tickets and escalated paths for resolutions.

    DevOps is a process for accelerating the delivery of features to the application or product infrastructure in a consistent model with less human intervention, while also providing a better quality product through automation in the software development life cycle.

    An AIOps process introduces data science into the operating model by learning the behavior of the systems, scaling according to the needs of a platform (both infrastructure and customer usage of the platform). It expands the horizon of DevOps through the introduction of machine learning, focusing on data generated from the hardware and software systems, and allows organizations to grow organically based on demand.

    NoOps, although the name suggests otherwise, is an advanced approach to managing IT operations through the mindset that everything is derived as development. This model is best applied with startups and companies with high technological maturity.

    Typically, these companies have advanced skill levels to deliver the best experience by adapting to the new industry trends of both software and hardware maintenance practices. NoOps is at the opposite end of the spectrum from traditional operations, and it’s largely a theoretical concept at this point.

    NoOps is a fully automated, cloud-based approach that eliminates the operations team, aside from an integrated team focused on life cycle. The self-managed system requires resiliency and a high level of maturity with fail points that have already been addressed. NoOps, or high maturity structures, will also provide quick learning on drawbacks, customer and product needs. However, moving toward a self-maintained system requires upkeep for advances and innovation.

    Invest in an Operations Model to Reach Higher Levels of Maturity

    The first factor to consider in evaluating an operational approach is cost. Each level of IT maturation is gradual, incremental and expensive. Costs vary depending on the size of the company and the type and number of products. A small company that seeks to move from a level 2 to 4 maturity would need to make a significant financial investment.

    Some questions to consider when looking at operations model upgrades include:

    • Does this make sense for the business?
    • Will the investment allow for scaling?
    • Will it increase customer loyalty and trust in the brand?

    Next, it’s important to consider overarching goals and brand identity. The appropriate level of maturity might depend on the company’s value proposition. If, in the company’s vision, technology is at the heart of its journey, the company needs to invest, at a minimum, in a DevOps operating model to have the competitive edge to grow the business and meet its needs. If the company’s vision is to retain existing business and gain customers through continued investment in innovation based on the trend analysis of their core business, then AIOps will have the best ROI. If the company’s vision, culture and mission are intertwined with technology, adapting to NoOps is the right move.

    Know the Product, Customer and When to Halt

    Still, it’s important not to overreach with an operations model that might not fit the customer. To make a sound decision for the organization, it is important to understand the model, the product, customer base and when to halt an operational model and invest in a new one. The leadership team needs to ask how adaptable its end users are before implementing changes. Take, for instance, the poor consumer response to replacing live customer service representatives with automated features. Some end users were not prepared to adapt to the massive change in the platform or service offerings this move entailed.

    Product-driven businesses often don’t see investing in new operations models as worthwhile. This mindset can result in continual churn that creates a backlog and increased operating costs for maintaining systems and services. Another important consideration in deciding which operating model meets the company’s needs comes through experimentation. That is because finding the right operating model for an enterprise can’t be done with a cookie-cutter approach. The point is to maintain the existing quality and productivity while investing in outcomes that will garner more trust and confidence in the company. If you do this right, the end user will appreciate the changes, and the company will outperform the competition.

    Allow for Operations Failure

    Enterprises should expect failures during experimentation with different models and plan to pressure-test an advancing model for the cracks that will emerge. It is important to invest in failure to understand why and how the model did not perform to expectations. Those failures provide valuable insight and important data, result in significant learning about products and customers and help you know what to improve or modify going forward.

    An operations model serves as the backbone for any business that wants to grow. Smart operations investments will ensure adaptability, competitive advantage and customer loyalty. But there’s no one-size-fits-all, easy-to-follow recipe for each enterprise. Whether the investment results in AIOps, NoOps, DevOps or traditional operations is unique to each organization and needs to be calculated with the brand, customer and end goal in mind.

  • Enterprise Software Development Will Break the Speed Limit in 2021

    Enterprise Software Development Will Break the Speed Limit in 2021

    For some, 2020 was the year for software development speed … but not for everyone. Startups and SMEs climbed on board the CI/CD, shift-left and agile development train en masse – and reaped the benefits of faster iterations and tighter release schedules.

    But the “big kids” got left behind. Core enterprise applications – the legacy systems with 1+ million lines of code that do the heavy lifting in many corporations – have not yet made the switch to the world of rapid release cycles. And this is natural; enterprises are rightfully more cautious by nature, and because the technology driving this evolution was not mature by enterprise standards.

    But in 2021, that’s going to change.

    Here are four speedy predictions for pushing enterprises over the speed limit in 2021, and some of the companies making it happen.

    Four Enterprise Software Development Speed Predictions

    #1: AI-based test generation will make enterprise development faster and better

    Testing automation is crucial for any software development organization trying to transition to CI/CD. But enterprise stakes are arguably high, and enterprise code is by nature inflated, often carrying significant technical debt. Preparing suitable test coverage manually is resource-heavy, to say the least. But a new generation of AI-powered testing automation tools solves this problem for you. Solutions from companies like Ponicode and Diffblue allow enterprises with large code base apps to embrace CI/CD by bridging the technical debt gap in testing while still meeting release cycles.

    #2: Test avoidance tech will make a dent in unnecessary testing

    Running tests on an entire build for one small code change is overkill – no one’s arguing that. But until recently, there wasn’t much of an alternative to ensure the ironclad quality that enterprises require. In 2021, solutions from companies like Sealights and Launchable (which counts among its investors Jenkins creator Kohsuke Kawaguchi) will help enterprise software development teams deliver quality at speed. The key is the effective use of machine learning to reduce test cycle times and run only relevant tests, not the entire test suite, to ensure faster iteration cycles.

    #3: High Performance Computing will move toward distributed computing

    Compute power will remain a bottleneck to development in 2021 – but the processing arms race will likely change. As enterprises get used to virtualizing their infrastructure, they will think twice about investing heavily in dedicated multicore build/render machines powered by next-gen processors. Instead, they will look to virtualized processing power like distributed computing, cloud bursting, and spot instances – which provide the massive scalability enterprises require at build time, while still maintaining a more reasonable price-performance ratio.

    #4: Managed CI/CD in the cloud will automate enterprise release pipelines

    Managed CI/CD, like Amazon’s CodePipeline and Azure Pipelines, help enterprises save time setting up and maintaining CI/CD infrastructure, scaling up during peak times and maintaining security. These services, while not for everyone, will continue to gain popularity in 2021. A key reason for increased adoption is that they allow companies to scale much faster: without hardware or software, development teams can push a button and suddenly gain 20 more build servers. Moreover, managed CI/CD enables faster onboarding of new teams, simpler creation of repositories and more streamlined work with multiple geolocations. Finally, pay-as-you-go models, no upfront fees and no commitments are increasingly attractive to enterprises in tough times.

    Put the Pedal to the Metal

    Newly released AI-powered products and mature hosted services from big-name cloud providers are changing the face of enterprise-level development. In 2021, we’re going to see teams supporting backbone enterprise applications increasingly moving to CI/CD, shift left and agile development methodologies. Because everyone – even enterprises – deserves to break the speed limit.

  • Measuring the Business Benefits of AIOps

    Measuring the Business Benefits of AIOps

    Staffing levels within IT operations (ITOps) departments are flat or declining, enterprise IT environments get more complex by the day and the transition to the cloud is accelerating. Meanwhile, the volume of data generated by monitoring and alerting systems is skyrocketing, and Ops teams are under pressure to respond to incidents more quickly.

    Faced with these challenges, companies are increasingly turning to AIOps – the use of machine learning and artificial intelligence to analyze large volumes of IT operations data – to help automate and optimize IT operations. But before investing in a new technology, leaders want assurances that it will bring value to end users, customers and the business at large.

    Leaders looking to measure the benefits of AIOps and build KPIs for both IT and business audiences should focus on key factors such as uptime, incident response and remediation time and predictive maintenance to prevent outages that could affect employees and customers.

    AIOps KPIs include employee productivity, customer satisfaction and web site metrics such as conversion rate or lead generation. AIOps can help companies cut IT operations costs through automation and rapid analysis; it can support revenue growth by enabling business processes to run smoothly and deliver excellent user experiences.

    Specific IT and Business Benefits

    AIOps can digest operational data and spit out actionable recommendations for keeping important systems running at peak efficiency. These are the most-cited benefits of the technology:

    • Alert management: In most cases, the first problem AIOps systems will address is reducing the volume of noise: the torrent of alerts that inundate IT operations. AIOps uses clustering and pattern matching algorithms to eliminate as much as 90% of false alarms and other redundant or irrelevant alerts, making it far easier for staffers to focus on what really matters.
    • Incident prioritization and routing: AIOps systems can learn, over time, which types of alerts should be sent to which teams, reducing redundancy and confusion when, say, networking and database teams both get the same alert related to an incident.
    • Event correlation: AIOps can correlate alerts and event data to identify the root cause of an outage or application slowdown so that IT teams can respond faster.
    • Advanced anomaly detection: AIOps systems can identify anomalies to proactively detect abnormal conditions and relate them to business impact. For example, AIOps can predict if a system will run out of disk space based on projected growth or seasonal patterns, even if the growth is non-linear. Or, if there’s a sudden increase in the number of failed server requests, the technology can determine whether the server in question is handling a mission critical task or simply performing routine backups.
    • Automation: AIOps can be used to handle routine tasks such as backups, server restarts and other low-risk maintenance activities which otherwise involve heavy manual effort.
    • Predictive analytics: A more advanced use case could involve predicting events before they happen – such as detecting when network bandwidth is reaching its limit or storage capacity is nearing threshold.

    Seven KPIs for AIOps

    These common KPIs can measure the impact of AIOps on business processes:

    1. Mean time to detect (MTTD): This KPI refers to how quickly it takes for an issue to be identified. AIOps can help companies drive down MTTD through the use of machine learning to detect patterns, block out noise and identify outages. Amid an avalanche of alerts, ITOps can better understand the importance and scope of an issue, which leads to faster identification of an incident, reduced downtime and better performance of business processes.
    2. Mean time to acknowledge (MTTA): Once an issue has been detected, IT teams need to acknowledge the issue and determine who will address it. AIOps can use machine learning to automate that decision-making process and quickly make sure that the right teams are working on the problem.
    3. Mean time to restore/resolve (MTTR): When a key business process or application goes down, speedy restoration of service is key. ITOps plays an important role in using machine learning to understand if the issue has been seen previously and, based on past experiences, to recommend the most effective way to get the service back up and running.
    4. Service Availability: Often expressed in terms of percentage of uptime over a period of time or outage minutes per period of time, AIOps can help boost service availability through the application of predictive maintenance.
    5. Percentage of automated versus manual resolution: Increasingly, organizations are leveraging intelligent automation to resolve issues without manual intervention. Machine learning systems can be trained to identify patterns, apply solutions, such as previous scripts that were executed to remedy a problem, and take the place of a human operator.
    6. User Reported versus Monitoring Detected: IT operations should be able to detect and remediate a problem before the end user is even aware of it. For example, if application performance or web site performance is slowing down, ITOps should get an alert when the issue can be measured in milliseconds; they can then fix the issue before the slowdown worsens and affects users. AIOps enables the use of dynamic thresholds to ensure that alerts are generated automatically and routed to the correct team for investigation, or automatically remediated when policies dictate.
    7. Time savings and associated cost savings: The use of AIOps, whether it’s to perform automation or more quickly identify and resolve issues, will result in both operator time savings and business time-to-value improvements. These have a direct impact on the bottom line.

    These seven AIOps KPIs can be correlated to business KPIs around user experience, application performance, customer satisfaction, improved e-commerce sales, employee productivity and increased revenue. ITOps teams need the ability to quickly connect the dots between infrastructure and business metrics so that IT is prioritizing spend and effort on real business needs. In the future, as machine learning matures, AIOps tools may even recommend ways to improve business outcomes or provide insights as to why digital programs did or did not succeed.

  • Red Hat Expands RHEL Subscription Program for Developers

    Red Hat Expands RHEL Subscription Program for Developers

    Red Hat is extending its developer program to make it simpler for both individual developers and small teams to access a free Red Hat Enterprise Linux (RHEL) operating system subscription.

    These programs are launched in the wake of Red Hat’s decision to no longer invest in an open source CentOS Linux distribution project. Many developers employ CentOS to build applications that are later deployed on RHEL in a production environment.

    Starting in February, an existing no-cost Individual Developer subscription for RHEL can be used in production for up to 16 systems in an on-premises environment or in the cloud. Previously, developers were limited to using the subscription on a single machine. Red Hat is also making it easier for teams of developers to join this program at no additional cost via an existing subscription their organization may already have.

    In place of CentOS, Red Hat is positioning the CentOS Stream as a hub through which development teams will gain access to a continuously updated Linux distribution that will become the next minor update to RHEL. The Fedora project, managed by Red Hat, will continue to exist as the vehicle through which the RHEL community experiments with major innovations.

    Brian Exelbierd, head of RHEL community projects, said these changes should ultimately prove to be a boon to DevOps teams, as they will better align the instances of applications being developed with existing instances of RHEL running in production environments. Historically, there was a significant amount of effort required on the part of DevOps teams to move an application developed on CentOS to RHEL, said Exelbierd.

    At least two other entities have rushed to fill the perceived CentOS gap. Gregory Kurtzer, one of the co-founders of CentOS, along with the support of more than 650 contributors, has launched Rocky Linux to make sure IT organizations have access to a fork of RHEL without a subscription. At the same time, CloudLinux is pledging to spend $1 million annually to create a distribution of Linux, dubbed Project Lenix, that is also based on a fork of RHEL. CloudLinux previously provided a hardened edition of CentOS that has been installed more than 200,000 times by 4,000-plus customers.

    Exelbierd said Red Hat is not trying to subvert those efforts, but it does have a vested interest in making it simpler for developers to build applications that eventually get deployed on production systems. Red Hat only generates revenue when organizations pay to subscribe to an instance of RHEL that gets deployed in a production environment.

    Regardless of what operating system developers employ, unless organizations decide to deploy CentOS in production environments, all paths eventually lead to RHEL. It will be interesting to see how far organizations will go in supporting a fork of RHEL that, in many cases, was employed simply because a developer didn’t want to pay for full subscription.

  • DevSecOps Implementation: Intrusion Detection

    DevSecOps Implementation: Intrusion Detection

    Originally, this series was just going to be four articles on the DevSec side of DevSecOps. There are many reasons for this, but primarily because that side is cleaner. The other reason is that these topics are beyond the work we were doing at Accelerated Strategies Group.

    But we’ve had a number of requests to continue, and I’ve used many tools in the space, so on we go.

    Intrusion Detection

    Intrusion detection (IDS) started because attackers were getting through firewalls, and, once inside, seemed to have free rein. Intrusion detection wanted to offer another option for detecting and stopping attackers by watching things inside the firewall for indications your organization had been compromised. Relatively quickly, intrusion detection was supplemented by intrusion prevention (IPS). The idea of prevention was to proactively make changes based upon known issues to keep intruders out. Intrusion detection offered information about what paths attackers were exploiting, and intrusion prevention would plug them. Note that IDS and IPS are often further subdivided into host-based (HIDS) or network-based (NIDS) systems. Since their role – detection of intruders – is the same, for this overview we will discuss them together. A serious deep dive into IDS/IPS should look at HIDS and NIDS separately.

    In the DevSecOps space, some have folded these into security monitoring, but they are still performing the same basic tasks – just much better. In fact, if you found intrusion detection to be limited and clunky to work with because of rules maintenance and fingerprinting issues, it’s probably time to take another look. Here’s why:

    Problems With Traditional Intrusion Detection

    The problems with pure rules-based intrusion detection were two-fold.

    First, rules were hard and fast.

    “Look for X and Y and Z. If all are present, raise an incident.” This tended to result in only two types of rules – ones that were too narrow, and thus brittle, or ones that were too broad and that buried IT/security staff in alerts.

    Second, those rules were outdated before they hit the network.

    The old adage “You are always fighting the last war,” (meaning you are doing what worked the last time you won) is more true in IT security than almost anywhere else. For a rule or a fingerprint to be assembled, it has to have been seen. That means we were perpetually protecting against previously successful attacks and doing little against emerging threats. Security vendors took a lot of steps before coming to our current solution, which is a great step in the right direction.

    Attacks against IT infrastructure share a lot of commonalities. Purely rule-based systems counted on tribal knowledge to code those commonalities into rules. It was clunky and imperfect. Templates helped, by adding common items based upon the type of rule you were creating, but, like rules, they tended to be static and old news.

    Enter Machine Learning

    Bright people with an eye on the ever-changing environment knew that we needed something that could adapt to multiple attack forms/vectors and draw the type of conclusions that security analysts were tasked with. The ability to reduce the number of alerts that get raised is pretty close to miraculous. Anyone with a web presence sees thousands of security events a day, as attack bots and scanners automate chunks of the attack matrix. Some kind of solution to weed out the false information is nearly mandatory, as no organization has the manpower to monitor at those levels and reasonably follow up on threats.

    Machine learning (ML) makes connections between alerts – that might not be a big deal in isolation – and other events that, together, make a case for human intervention. Added to fingerprinting and rules, it is a powerful addition. While a successful attack would require going through rules and making certain you covered the behavior/changes found in the new attack, ML can take that information in and adapt automatically.

    This is cranking through volumes of data we couldn’t even imagine a decade ago, and finding anomalous behavior, correlating it and alerting where necessary. How much it reports up the chain is generally configurable, allowing an organization to tighten the flow of alerts that analysts have to deal with.

    Go With the Flow

    More recently, flow evaluation has become a positive step in the monitoring process. If this connection/user moves from A to B to C, it might be an attack. This allows for another way to rate suspicious activity and raise it to an analyst’s awareness.

    An increasing number of vendors are moving the analysis portion of these activities into their hosted environment. This is a “great strength/great weakness” solution – putting the data points gathered from your system into their cloud allows them to correlate events and improve all customers’ responses. The other side of the coin is that you are paying them operational costs each month, and for most vendors, you’re charged based on how much of the service you use. This has budget implications. It also assumes you have connectivity to their service. If you don’t, for any reason, no analysis will be done on collected data. A careful rollout and contingency planning are in order for this scenario to be successful.

    Integration with security information and event management (SIEM) systems is ubiquitous. We’ll talk about SIEM next; I just wanted to include here that IDS/IPS vendors are feeding SIEM.

    IDS isn’t perfect, and there are a ton of other things out there to help, but it’s a step in the process, one you should consider as you gear up your DevSecOps to professional levels. Some of you are using the tools, but don’t have them integrated into your DevOps systems. This is a nudge to do so. If resolution of an attack automatically creates tickets for Dev to make the code (or ops to make the config) more secure, it absolutely is a plus to the organization responding proactively.

    Keep kicking it. These are just tools, and they’re useless without you. Figure out what your organization needs, and update those needs regularly as the environment (both attackers and architecture) continually changes around you.

  • How MLOps Can Help Get AI Projects to Deployment

    How MLOps Can Help Get AI Projects to Deployment

    Did you know that most AI projects never get fully deployed? In fact, a recent survey by NewVantage Partners revealed that only 15% of leading enterprises have gotten any AI into production at all. Unfortunately, many models get built and trained, but never make it to business scenarios where they can provide insights and value. This gap – deemed the production gap – leaves models unable to be used, wastes resources and stops AI ROI in its tracks. But it’s not the technology that is holding things back. In most cases, the barriers to businesses and organizations becoming data-driven can be reduced to three things: people, process and culture. So, the question is, how can we overcome these challenges and start getting real value from AI? To overcome this production gap and finally get ROI from their AI, enterprises must consider formalizing an MLOps strategy.

    MLOps, or machine learning operations, refers to the culmination of people, processes, practices and underpinning technologies that automate the deployment, monitoring and management of machine learning models into production in a scalable, fully governed way. Laying an MLOps foundation allows data, development and production teams to work collaboratively and leverage automation to deploy, monitor, manage and govern machine learning services and initiatives within an organization. In short, MLOps can help enterprises see AI projects through to deployment at scale by addressing the three previously mentioned barriers: people, process and culture.

    People

    There’s a skill, motivation and incentive gap between teams developing machine learning models (data scientists), operators of those models (DevOps, software developers, IT, etc.), and the business leaders who look for the results of that work. By providing actionable visibility into the different steps of the machine learning model life cycle for these different roles, MLOps can increase collaboration across departments so that enterprises can streamline the process of getting models operational and finally meet strategic goals.

    Process

    Operations teams are geared towards ensuring overall quality of service, optimizing runtime environments on their cloud, resource management and role-based access, among many other tasks. Data science teams are often unaware of the considerations these dependencies require, and, hence, models they create do not take these tasks into consideration. Additionally, lack of a proper native governance structure as it pertains to machine learning models – with system, performance, life cycle, and user logs – stifles troubleshooting, as well as hinders legal and regulatory reporting. Organizations that don’t properly monitor their models end up introducing potential risk to their organizations. When production models don’t reflect the ever-changing patterns in data and user behavior, these factors may affect the accuracy and overall performance of the model. Machine learning-based applications are far more sensitive to production environments than typical software applications. Therefore, machine learning models need to be validated and updated over time to ensure that they’re performing properly and leveraging the most accurate and recent data. As part of implementing an MLOps strategy, by having the right monitoring processes in place to detect drift – and many other machine learning-specific metrics – enterprises can ensure their models are delivering the most accurate results, and that their projects are more likely to move forward.

    Culture

    Fear of failure (or worse) can delay organizations from collaborating to operationalize machine learning models, or prevent this entirely. Leadership needs to be clear about their commitment to the process and fully support teams in enduring the hardships involved. This is only compounded by the fact that machine learning applications typically tend to involve processes that might introduce material risk to the company. To stay on top of such transformations while still producing trustworthy value, enterprises must keep careful tabs on all model deployment processes and the updates they entail. Many operations professionals are not aware of the unique characteristics and sensitivities of machine learning. As their role is focused on managing mission-critical production environments, they are often solely concerned with the implications on their work that deploying machine learning into production might have. While these are justifiable concerns, in the final analysis, they serve as trivial reasons for significant delays in progress and deployment. The way to mitigate all these risks and get all teams aligned around these common goals is to implement a solid MLOps strategy that matches the goals and unique position of the organization and its various teams’ responsibilities.

    MLOps: All Systems Go

    MLOps allows organizations to alleviate the people, process and culture issues posed by machine learning. MLOps provides a strategy and a set of processes and best practices, a technological backbone, for managing and scaling machine learning-based services through automation. It also provides for seamless collaboration between the data science teams responsible for generating models with the teams that are accountable for running the services in production environments. This combination of process automation with role-based collaboration and governance capabilities helps pave the way toward strategic AI goals. Specifically, MLOps can help organizations move from anticipating the management and scaling of machine learning services into production and turn that into reality.

  • Rapid Strategic SRE Assessments Accelerate IT Transformations

    Rapid Strategic SRE Assessments Accelerate IT Transformations

    A Strategic SRE Assessment is critical in helping organizations successfully transform traditional IT operations to SRE

    Site reliability engineering (SRE) is rapidly becoming more important to organizations that need to scale IT operations while maintaining or improving reliability and security, and to modernize operations to match the growth of DevOps in the organization. Without sufficiently thorough consideration given to best practices for people, processes and technologies, the transformation from traditional IT operations to SRE is unlikely to succeed. Despite challenges, the transformation to SRE is compelling because it provides an opportunity to reduce total operations costs and improve security and reliability while scaling services.

    To be successful, IT leaders need a comprehensive understanding of SRE best practices and need a well-engineered, yet affordable strategy and road map that effectively implements best practices and balances the many aspects needed to transform people, proces and technology. A Strategic SRE Assessment approach, based on best practices, is a key tool to help leaders understand, obtain the strategy and create a road map that will successfully transform traditional IT operations to SRE.

    There is no “standard” SRE assessment approach in the industry. This article enumerates a practical and proven Strategic SRE Assessment approach. The heart of the Strategic SRE Assessment approach is a survey of SRE best practices categorized under nine pillars: Culture, Work Sharing, Toil Reduction, SLAs/SLOs/SLIs, Measurements, Anti-Fragility, Deployments, Performance Management and Incidents Management. The survey is used to collect importance and capability scores for each of the SRE practices. As part of the assessment, a survey determines the current state and preferences of SRE practices for an organization for each of the nine SRE pillars.

    The following are examples of best practices for each pillar that are part of the Strategic SRE assessment survey:

    Culture – Leaders need to understand and sponsor a clear vision for SRE.

    • The culture embraces working practices to proactively shifts left the “Wisdom of Production.”
    • The culture encourages learning from failure and continuous improvement.
    • The SRE role is clearly defined and understood and SREs are considered a vital part of the organization.
    • Rules of engagement and principles for how SRE teams interact with their environment are codified—not only the production environment but also the product development teams, testing teams, users and so on. Those rules and work practices help to maintain SRE focus on engineering work as opposed to operations work.

    Work Sharing – Successful SRE requires that work be shared between SRE teams and development.

    • SRE teams are composed of software developers with strong operations knowledge or IT operations people with strong software development skills.
    • SRE workloads are managed to a budget with typically 50% or less spent on operations work.
    • SRE on-call work budgets are typically 25% or less. Developers share on-call work.
    • Technical debt is worked in small increments.

    Toil Reduction – SREs work to reduce non-value-added work through automation and standards.

    • Work standardization and automation—of both tools and processes—is employed to improve scalability, repeatability and other important goals.
    • Work standardization is used to reduce time spent on non-value-added work.
    • Automation is used to reduce time spent on non-value-added work.

    SLAs/SLOs/SLIs – Service level metrics, based on user perspectives, combined with error budget policies, are critical to managing flow and reliability.

    • SLOs are selected to measure availability, performance and other metrics in terms that matter to an end user.
    • An error budget (= 1-availability) internal service level objective (SLO) is established for each service. SLOs are set to be more stringent than external service level agreements (SLAs) promised to customers. The SLO budget considers what level users will accept, alternatives for users who are dissatisfied, and usage patterns of the product at different availability levels.
    • An error budget policy prescribes consequences if an error budget is spent. For example, the service freezes changes (except urgent security and bug fixes addressing any cause of the increased errors) until either the service has earned back room in the budget or the count period resets.

    Measurements – Smart monitoring implements observability to accelerate response times to incidents.

    • Monitoring solutions are designed to produce service level indicators (SLIs) from which SLOs can be calculated automatically.
    • SLIs do not require a human to interpret an alerting domain. Instead, software observability algorithms do the interpreting, and humans are notified only when they need to act.
    • Monitoring systems use telemetry and instrumentation to ensure the data being monitored is accurate, relevant and available in real-time.

    Anti-Fragility – Systems of people, processes and technologies are continuously tested and improved to assure they are resilient enough to serve applications as they scale.

    • Anti-fragility strategies are practiced to proactively assure the resilience of applications, infrastructures and pipeline services.
    • Fire drills are periodically conducted to proactively determine weaknesses in systems of people, processes and technologies that could potentially impact SLOs.
    • Chaos monkey or equivalent infrastructure testing strategies are used to proactively determine weaknesses in infrastructure systems that could potentially impact SLOs.
    • Security testing and DevSecOps practices reduce the risk of security vulnerabilities.

    Deployments – Deployment strategies reduce the risk of deployment failures.

    • Non-emergency deployments use progressive rollouts in which changes are applied to small fractions of traffic and capacity at one time. If unexpected behavior is detected, the changes are rolled back first and diagnosed afterward to minimize mean time to recovery.
    • Automation is used to implementing progressive rollouts, detect problems and implement rollback changes safely when problems arise, because outages are primarily caused by changes in a live system.
    • Deployment strategies such as blue-green, feature flag rollouts, A/B testing and canary rollouts are used to reduce the blast radius of failed deployments.

    Performance Monitoring – Proactive testing and monitoring ensure applications and infrastructures have the capacity necessary to flex and scale when needed.

    • The SRE team is responsible for provisioning and monitoring utilization because utilization is a function of how a given service works and how it is provisioned.
    • The SRE team is responsible for capacity planning and provisioning. Regular load testing of the system is used to correlate raw capacity (servers, disks and so on) to service capacity.
    • Provision to handle a simultaneous planned and unplanned outage without making the user experience unacceptable. This results in an “N + 2” configuration, in which peak traffic can be handled by N instances (possibly in degraded mode) while two instances are unavailable.

    Incident Management – Incidents are managed in an efficient, blameless manner with a goal to use incidents to improve responses.

    • Playbooks record the best practices for human response actions because playbooks produce roughly a 3x improvement in mean time to recovery.
    • Emergency response performance is measured using mean time to repair (MTTR) metrics instead of mean time to failure (MTTF) because the most relevant metric in evaluating the effectiveness of emergency response is how quickly the response team can bring the system back to health—that is, the MTTR.
    • Postmortems are blameless and focus on process and technology, not people.
    • Postmortems are conducted for significant incidents regardless of whether they resulted in a page; postmortems that did not trigger a page are even more valuable, as they likely point to clear monitoring gaps. Find all root causes of the event and assign actions to correct the problem or improve how it is addressed next time.

    SRE Assessment Surveys Are Not Enough

    The survey alone is an essential tool to collect information about the current state and preferences for each of the SRE practices. However, this is only part of the comprehensive Strategic SRE Assessment approach.

    The Strategic SRE Assessment approach starts with workshops to get alignment between leaders and teams regarding big-picture goals prior to conducting the survey. This is necessary so that everyone who answers the survey has a common perspective and can better respond with considered views of importance and capability scores for each of the practices. The results of the survey include an analysis that points out specific pillars and individual practices that have a high gap score in which the practices are scored as important but not yet practiced well. The high-priority gaps are then used to organize a value-stream map working session, that is designed to find the highest priority bottlenecks in people, processes and technology bottlenecks in the value steam, relative to the organization’s goals and gaps. With the goals, gaps, and data from the value stream map in hand, the next step is to formulate an implementation road map that directly addresses the goals, gaps and bottlenecks in the value stream.

    The final step is to obtain strategic alignment of the big-picture plan by reviewing the findings and road map with the leadership and implementation teams. This entire Strategic SRE Assessment approach, from goal alignment through surveys, value stream mapping, road map creation and strategic alignment, varies depending on several factors such as the size of the organization, number of applications and size of the deployment domains to be included in the assessment.

    A Rapid Strategic SRE Assessment for a sample set of model applications typically is conducted in 21 days. More complex Strategic SRE assessments involving many applications may take up to 90 days.

    In most cases the Rapid Strategic SRE Assessment is the best place to start. Most often, it is not a good idea and is too expensive to “boil the ocean” and make changes across all applications and deployment domains at once. By starting with a small number of model applications, it is easier to implement, measure and demonstrate success and use the results at each step to justify ongoing investments needed to complete the transformation across the organization. Another advantage of the Rapid Strategic SRE Assessment is that it is a quick and affordable approach to check-point progress as SRE is implemented progressively across applications and deployment domains.

    Key Takeaways for SRE Transformation Leaders

    SRE offers immense value for modernizing and scaling IT operations reliably and securely and to complement DevOps transformation. Adherence to SRE best practices is essential to reduce risk and assure security as the transformation is implemented.

    Each organization is different and has different requirements and priorities for its applications and deployment domains. The specific SRE practices that are the highest priority vary according to the priorities of applications and deployment domains. The Rapid Strategic SRE assessment approach, based on nine pillars of SRE best practices, is quick, effective and affordable.

    The Rapid Strategic SRE Assessment is a preferred approach to kick-start the SRE transformation. This approach is also a preferred and affordable tool to reassess the direction as SRE is implemented progressively across the organization.

    Summary

    There is no “standard” SRE assessment approach in the industry. The Rapid Strategic SRE Assessment approach described in this article has been tried and proven with large and smaller organizations. The author invites your comments regarding this approach and the SRE best practices.

  • Legacy Systems: Should They Stay or Should They Go?

    Legacy Systems: Should They Stay or Should They Go?

    Digital transformation requires new technologies. But does that mean legacy systems should be replaced?

    Survey after survey has shown that IT and business leaders have had to drastically rethink priorities, strategies and spending in the wake of the global pandemic. A survey our company conducted last October showed that technology spending hasn’t taken a terrible hit—in fact, 60% of IT leaders reported they had either significantly or moderately increased their IT budgets in the last six months. Other research from 2020 mirrors those findings.

    Yet, there are significant challenges. Three top ones reported in the OpsRamp survey include business and IT alignment, available technology solutions and economic uncertainty. Executives want to move forward, yet they need to be cautious with budgets and develop strategies that can flex with the unpredictability of 2021.

    And this is how we arrive at the topic of legacy systems. As part of the reprioritization effort, IT leaders are wise to take a hard look at their technology portfolio and determine which systems should go and which should stay—factoring in economics, business needs, customer demands and change management requirements. Progressive IT leaders know that if they don’t adopt the right technology at the right time, continued business success (and even survival) is at risk.

    Defining Legacy Systems

    We tend to associate a legacy system with a technology that’s been around a long time, yet it could also be a three-year-old system. Legacy systems can be a burden when the UI and the UX look dated, are slow or difficult to upgrade and integrate with other applications, and costly to maintain. Many legacy systems were built for on-premises environments and don’t migrate easily to the cloud. Finally, because some systems are built on languages and technologies that are 10 to 20 years old, IT organizations struggle to find people with the requisite skills to maintain them. These are all red flags.

    On the other hand, there are valid reasons why enterprises maintain legacy systems:

    • The technology technically isn’t broken and satisfies a core business process, industry requirement or customer need.
    • The system integrates with many other applications and core processes, making it difficult to replace.
    • Longstanding vendor relationships make it politically or culturally challenging to abandon.
    • Influential stakeholders love it, despite any negative impact on customers and revenues.

    How to Evaluate a Change

    Even if a system still currently meets your business need, it’s important to understand what your competition is doing and what your customers are asking for to ensure you’ve got a future-proof strategy. Consider Slack, Amazon, Google and Netflix: These innovators have redefined what experiences software can provide and thereby have changed expectations for all users regardless of the sector.

    In IT operations, if a customer is receiving fewer IT alerts due to filtering and correlation capabilities, that’s a good thing. But does the software proactively manage those alerts, filtering out the high-priority items to the right person and suggesting the best next steps for a quick resolution before users are affected?  From the executive and business view, you need to demonstrate efficiencies and cost savings from moving to a new solution as well as revenue-generating impacts.

    Mitigating Snafus With Legacy System Replacements

    When you decide it’s time to make the big move and replace a major system, consider the top factors that can ruin these projects or needlessly delay the time to value:

    • Gaps in knowledge and mindset. Look at what happened when every CIO got a mandate from the CEO to move operations to the cloud. Understanding how the cloud works and how to manage cloud-based assets was a struggle, so most early cloud migrations were simply lift-and-shift. That led to a common scenario in which not only did companies not achieve the savings they expected, but also the environment became even more expensive to support than their on-premises infrastructure.
    • New skills. In lockstep with the gap in strategic understanding of new technologies is the fact that it’s difficult to acquire or build the associated new skills fast enough to keep pace with changes in the market. This is why still today, more than 10 years after the advent of public cloud, there are significant talent gaps in the areas of cloud architecture, cloud optimization and cloud-native development.
    • Cultural changes. People don’t like change, especially if they’re not unhappy with the status quo. IT leaders are improving here, but there’s still ample work in helping convey the reason for change and the benefits to designing a frictionless path to adoption.

    The 3-step Plan to Modernization

    There’s nothing especially new about how to avoid a marred enterprise software implementation, except for the fact that tolerance levels are at an all-time low. Executives don’t want to see one-year rollouts and two-year time-to-value cycles anymore. Here’s how to simplify and speed up the benefits when moving to new platforms:

    1. Start with a small, powerful punch. You can gain momentum with a small project that doesn’t commit the organization to a gargantuan, tangled overhaul. Pick a project that’s not going to stress out your team and has a specific, predictable ROI. Truth is, people love innovation and want to be part of the “next new thing” as long as it doesn’t require too much pain all at once. Make sure to socialize the outcomes across the organization and explain how you did it.
    2. Make the leaders your champions. If your executive team doesn’t feel motivated to promote a new system internally, you might want to investigate whether it’s the right solution. Be sure to get that buy-in before you sign any vendor contract. Communication style matters. Who on the executive team can really create excitement around the future and effectively demonstrate the need for change? You want a salesperson personality for this job.
    3. Foster an evangelist committee. In the trenches, you also need a team of people who understand how to use the new solution and are eager to share their tips. I’ve seen leaders take on a full internal campaign prior to a new system being launched, from rebranding and relabeling an enterprise software package, developing an internal PR strategy around it and creating really effective tools such as short videos to help users get up to speed.

    Balancing Continual Innovation and Lower Complexity

    A legacy system overhaul is not for the faint of heart. You must carefully consider whether your organization is ready to take on transformational change or if an incremental change, such as customization or a new integration to your existing platform, is enough for now. But if incremental change is just going to help you survive another day, it can make the necessary change you really need much more painful later on.

  • Why Boring Tech is Best to Avoid a Microservices Mess

    Why Boring Tech is Best to Avoid a Microservices Mess

    Microservices are all the rage. A Forrester study found that 76% of enterprises were rearchitecting applications for microservices.

    At the same time, microservices are definitely not a silver bullet. Among those already using microservices in production, one study found that 59% said each microservice added increased operational challenges such as data management. Even more concerning, the same report revealed that when asked to compare the difficulty of troubleshooting different environments, 73% said microservices were more difficult, and only 21% said they were easier than monoliths.

    But how did we end up like this? According to Matthew Sinclair, VP of engineering at BCG Digital Ventures, it is because that, as technologists, “we want to use the most modern tech, and sometimes we even want to do it with tech we’ve got no experience in. We see this with engineers, not because it’s the right thing to do but because they want to learn something new.”

    Sinclair, who was speaking as a part of an Indorse Engineering Leaders Online Panel, went on to point out how the problem becomes self-perpetuating: Customers hear about new tech—in this instance microservices—that will fix the problems they have with monoliths. They rave about microservices to software engineers, who also get excited about it and then go off and start adding microservices in software engineering projects.

    The problem is, there is just as much chance of getting into a tangle with microservices as there is with monoliths. That’s because people often start breaking things up into microservices too early—they’ve heard it will be a good idea and they’ve seen some of the benefits, so they try to apply it to everything. In doing so, they create what some might call a “micromess.”

    Avoiding the Micromess

    So, how do you avoid ending up in a micromess?

    You need to go back to the fundamentals. One way to look at it is understanding that microservices are distributed systems, something many people will have experience with. They should also be familiar with what another panelist, Oswaldo Hernandez Perez, engineering manager at Improbable, called the first law of distribution: “If you don’t need to do it, don’t do it.”

    So that means focusing on why you are building what you are building. What are you trying to achieve? This is a fundamental question that’s applicable to businesses of all sizes. What problem are you trying to solve, and how will your solution remove friction from its users’ lives?

    That’s what people care about. Even if you’re developing a niche app for a highly technical audience, they are unlikely to care too much about how it got to them, only that it did and it is fixing a problem for them.

    If the only way to achieve that is with microservices, then yes, you should definitely use them. If there is an alternative, then consider using that. Do not simply start breaking everything up into microservices just because that is what everyone is currently talking about.

    Don’t Stick Your Head in the Micromess

    Ultimately, microservices are an architectural pattern to reduce complexity. It does this, but it also adds complexity elsewhere. If used in isolation, then you’ll fix your complexity in one dimension and have it proliferate elsewhere. So, it is vital that you use the many other tools out there to minimize complexity across the entire organization.

    It goes back to the question: What problem are you trying to fix? The majority of businesses can do that with boring tech. Granted, the likes of Amazon and Google, which are cloud-native to their core, can use microservices, but even large enterprises should be asking themselves if it’s really the right choice for them.

    Established businesses are likely to have a significant legacy codebase, which needs integrating with. Again, microservices might not be appropriate in this situation—it might be worth saving them for cloud-native projects, and use boring tech to resolve the challenges within legacy IT.

    Do You Really Need Microservices?

    Time and time again, we come back to the fundamental question: What are you trying to fix with this product and/or technology? Not how, but what? What friction will you banish from the user’s life if they engage with your product? Only once you answer that question should you consider whether microservices is for you. Chances are, you may find you don’t need them by that point.