devopsy.kinsta.cloud

Tag: gitops

  • Why You Should Use GitOps to Experiment With AI

    Why You Should Use GitOps to Experiment With AI

    As the pace at which artificial intelligence (AI) models are being constructed and inevitably updated starts to increase, it’s becoming more apparent that the pace at which data science teams are currently experimenting also needs to dramatically quicken.

    Many data science teams today are fortunate if they manage to successfully deploy two AI models in a production environment over the course of 12 months. It’s already apparent, however, thanks in part to the rise of digital business transformation, that there are potentially hundreds of use cases for AI models within each enterprise IT environment. Truth be told, it’s hard to envision an application being developed today that would not incorporate one or more AI models. The challenge is, the demand for AI capabilities is now far outpacing the ability of data science teams to deliver.

    The best way to solve this issue is to recognize that AI models and the components employed to construct them are, in reality, just another kind of software artifact. In fact, the need to accelerate the development of applications using artifacts is largely a problem already solved. Rather than reinventing the wheel, the time has come to apply what we already know to the development of AI models.

    The Rise of GitOps-Based Processes

    Modern software development teams today routinely make use of Git-based repositories to share code and collaborate. These repositories are jam-packed with vetted software artifacts that development teams continuously reuse via an opinionated instance of a DevOps workflow based on a set of GitOps best practices.

    As a subset of DevOps, a GitOps workflow enables data science teams to employ a Git repository that already exists within their software development environment to manage artifacts. Git enables versions of files to be created using what is known as a Git branch. They also employ concepts known as tags (Git Tag) and remotes (where the server is located). Those three different concepts, or references, exist under the hood in Git. In fact, the entire flow is based on those concepts. When a new feature is implemented, a new branch is created and then merged into the production branch with a tag that says something like “This is a new version that needs to be deployed.”

    An AI model, like any other software artifact, needs to be updated. As new data sources become available, an AI model is likely to drift beyond the parameters originally intended. A rapid change in business conditions may also require changes to an AI model. The same processes that DevOps teams apply to update any software artifact can also be applied to rapidly update an AI model. Employing a Git repository to also manage AI artifacts extends the benefits that software development teams enjoy to data science teams.

    Often, organizations will have an MLOps solution to streamline AI model development that is completely separate from their DevOps tools. These solutions will have dedicated features like version tracking for data and experiments specifically for AI models. But many MLOps solutions exist in a closed ecosystem, meaning they don’t work off of existing tools like GitHub used on the DevOps side. 

    Also, data science teams have to separately buy, deploy, and manage these closed MLOps solutions. Going with a GitOps-oriented solution streamlines management of added MLOps tools and makes it easier for both application developers and data science teams to gain visibility into each other’s respective efforts. The end result is not just faster AI experimentation at lower cost; the likelihood that more AI models will successfully be deployed in a production environment increases dramatically. 

    Software and AI Teams Working Together

    The convergence of DevOps and MLOps is already inevitable. The longer organizations continue to employ isolated MLOps platforms to build and deploy AI models, the more expensive it will become to meld MLOps and DevOps processes. 

    Data science teams are conservative by nature. The difference between success and failure often comes down to how thoroughly an AI model is validated. However, time to market doesn’t have to be sacrificed to ensure an AI model works as promised. There is no such thing as a great AI model that arrives late when rivals are racing to develop similar AI capabilities—there are only missed opportunities. Speed enabled by the reusability of artifacts that drives faster experimentation using GitOps workflows is of the essence now more than ever.

  • Shhhh! It’s a Secret

    Shhhh! It’s a Secret

    In one of our regular back-and-forth breaks, Lori and I were talking shop a week or two ago, and she shared an excellent report by GitGuardian with me. Frankly, none of the information in the report was a huge surprise for me–Git is filled with information that companies probably don’t want out there, but that is nearly impossible to track proactively. That’s what GitGuardian tries to do for you, but honestly, I’ve not yet dug into them, so caveat emptor. Or, read someone else’s opinion to help you decide if they’re a good idea – I’m just going to talk about the report.

    The key takeaway is that you are asking your DevOps teams to move faster and faster. Putting a section of code they are working on into a public repository is an easy way to “take work home” or share it, especially during lockdowns. It is also a good way to hand over critical information to anyone searching for it—about your infrastructure, or even critical bits of data like SSL keys or credentials. And with the current level of cyberattacks against nearly every industry, I’d say there are people looking for that information, no matter what industry you are in.

    This is not really a DevOps-specific issue–credentials and such in code has been a problem for IT for as long as I’ve been working in the field. At some point, any automated system needs to perform AAA, and that means the data required is stored somewhere. A multitude of approaches have been attempted to resolve the problem technologically, but none have been a silver bullet. Yet. Eventually, someone will figure out a way to make it work; then a whole new attack vector will arrive, probably aimed at the tool that is settled on to protect this very data.

    This is a personnel/policy issue that is exacerbated by the speed of Agile and DevOps. Your best solution, at this time, is to write a clear policy that contains consequences for putting secret data into a public repository. We’re short on “consequences” right now, preferring to say, “Right, let’s make sure that can’t happen again,” but in the case of putting secrets out in public, it is a different level of ‘bad’ than a coding error getting checked in. Those secrets can actively be used against the organization, and if the not-to-be-shared data is clearly delineated, any violation can be viewed as intentional, be it because of convenience or malfeasance.

    According to the report, there are millions of bits of privileged information out there. Not all of them are from enterprise IT shops—but some are. And some might be from your organization. Set expectations, get feedback. Make it clear you do not want to stop progress on delivery of systems, but you also don’t want to implement a system with a built-in back door for attackers to use. It is easy enough to use alternative delivery methods for those few bits of data, and put the bulk of the source out where it is needed.

    And keep rocking it. Toward Data Science says there are 128 million public repositories out there (Q1, 2021). Think how many lines of code are in those repositories, and how many secrets potentially are, too. Just make certain yours are not included in the list. No sense keeping your next great solution from running because your critical data leaked.

  • EP 9: The Road to GitOps

    EP 9: The Road to GitOps

    GitOps takes DevOps best practices for development and applies them to infrastructure automation. In short, GitOps delivers a hermetic, immutable deployment. So, is GitOps replacing DevOps?

    Managing infrastructure as code through GitOps can be a game changer for organizations. But, when is a GitOps approach best used or not a fit? In this DevOps Unbound episode, I’m joined by Tracy Ragan of DeployHub, Parag Doshi of Tricentis, Brendan O’Leary of GitLab and Mitch Ashley of Accelerated Strategies Group to discuss these questions, the impact of GitOps on the CI/CD market and where the future of GitOps is heading in 2021.

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Alan Shimel: Hello, everyone. This is Alan Shimel, CEO at MediaOps. And you’re watching another episode of DevOps Unbound. Let me introduce you to our panel for this week’s episode, and we’ll jump right into our subject matter. First of all, we have the newcomer to DevOps Unbound, but he’s not a newcomer to our audience, my good friend Brendan O’Leary, senior DevOps evangelist at GitLab, a governing board member for CNCF. Hey, Brendan. Welcome.

    Brendan O’Leary: Alan, nice to see you, as always, and happy to be here.

    Shimel: Thank you for being here, Brendan. Next up we have our good friend and a semi regular here on many of our video segments at MediaOps, Tracy Ragan. Tracy is CEO of DeployHub, as well as the community-elected board member of the CD Foundation. Hey, Tracy. Welcome.

    Tracy Ragan: Thank you. Thank you so much, Alan, for having me here today. I always love doing these.

    Shimel: Absolutely. We love having you. Our third panel member today comes to us from Tricentis, who’s also the sponsor for DevOps Unbound, and it’s Dr. Parag Doshi. Parag runs engineering, VP engineering at Tricentis. Parag, welcome.

    Parag Doshi: Hello, Alan, everyone. Nice to be here. Thanks for having me.

    Shimel: Thank you. Thanks for being on. And then last but not least, my cohost for DevOps Unbound, Mitch Ashley. Mitchell is the CEO/founder of Accelerated Strategies Group as well as CTO at MediaOps. Hey, Mitchell, thanks for coming on today.

    Mitch Ashley: Absolutely, Alan. And thank you for all of our panelists for joining us. You are all fantastic. We enjoy having you.

    Shimel: You know, just before we jump into it, the continuing evolution of COVID times, we’ve done so much with our backgrounds, [Laughter] on our videos, right? Looking here at Tracy’s snazzy DeployHub background and Mitchell’s background and Parag’s – and Brendan’s old-school, man. Brendan’s giving us – if I’m not mistaken, he used to be down in the basement and now he moved up. I think he displaced one of the kids or something.

    O’Leary: Yeah, I moved upstairs.

    Shimel: Yeah, yeah. And that’s a real background, so good for Brendan.

    Ashley: And he’s got an MV7 microphone there. Nice job there, Brendan, on the mic.

    O’Leary: Oh, thanks. Yeah.

    Shimel: Yeah.

    O’Leary: All, you know, post COVID things to make it work, Alan. I wanted to have the background, the real background to make it work. But that was also a COVID change.

    Shimel: It’s authentic. Yeah. [Laughter] Crazy stuff. Anyway, guys, today’s show is around one of the hottest topics in DevOps – GitOps. Right? And I’ve actually heard from people, you know, that GitOps is the new DevOps or GitOps is somehow replacing DevOps, and I think DevOps has outgrown the – it’s going to be replaced by something.

    It’s transcended to a different level of existence, if you will. But certainly GitOps is on a lot of people’s lips. It’s not only on their lips, it’s in their development plans and how they are delivering software to the cloud, to cloud native, and all over. But is the right move for every situation? Is it a panacea?

    When is GitOps a good thing to use? When is it maybe not a good thing to use? Right? How much does it change sort of the classic CI/CD pipeline that we’ve started to finally wrap our heads around in DevOps? Stuff like that. So I am going to – Brendan, of course, GitOps, GitLab, they both have Git in it.

    So we’re going to make you – I’d like you to start off. Let’s start with some definitions. What’s GitOps mean to you?

    O’Leary: That’s a great question, and I think one that’s still probably [Laughter] unsettled, right? Much like defining DevOps can sometimes be seen as unsettled. GitOps came out of the cloud native community, and so a lot of folks maybe restrict the concepts of GitOps to say it only exists in a Kubernetes world. Because that’s kind of where this approach came from. But we’ve seen folks employ GitOps principles to all kinds of different infrastructure, automation, from old-school VMs to containers to Kubernetes clusters as well.

    So when I think of GitOps, I think it kind of requires three things. One, infrastructure as code. Right? You said that in the beginning. That’s kind of like table stakes to getting in. It’s something we’ve been working on for a long time. But really defining as code in Git – the Git part of GitOps – what our infrastructure’s going to look like, and then keeping that configuration stored in that code.

    And maybe the desired state is stored as code or not, but at least the way we get to that desired state is. Second, I think it’s merge request. So we call it merge request. You’ll hear other folks call them pull requests. But the concept of using the point where someone is proposing a change as the central place where teams collaborate on.

    Is this the change we want to do? Is this going to help solve the problem we’re trying to solve for? Right? Bringing that concept to the world of infrastructure is hugely beneficial, because it’s how our developers have already been working for years and years. And making that merge request, kind of that central source of truth for approvals and security checks and all those other things is really critical.

    And then last, I think it’s – the third thing is CI/CD, right? Of course. So the idea of GitOps is automating that infrastructure, update with a very simple Git workflow. And so you really do have to have embraced and adopted continuous integration and continuous delivery really in order for the whole package of GitOps to make sense for you.

    Shimel: Pretty darn good. Comments, add-ons, thoughts? Parag, Tracy?

    Ragan: I would only add one thing here. If you say what is GitOps, I like to shorten it down. I think what Brendan just defined is all accurate, but it’s operations by pull request. The goal of GitOps, what GitOps delivers is a hermetic, airtight, immutable deployment. And that is in essence what we’re trying to accomplish by using Git as the place where we declare and define what we’re going to push out to or have pulled out to the cluster.

    So it’s the concept of a hermetic deployment, something airtight that no human touched. Don’t you dare touch that and then deploy it out, because of we don’t know what you did. [Laughter] So the whole idea is to create a hermetic process.

    Shimel: I love that. That was concise and right on.

    Doshi: I have a similar viewpoint. I think of GitOps as a combination of infrastructure as code plus event-driven architecture. Right? And in this case we’re talking about merge requests or pull requests or whatever terminology you want to use, depending on the flavor of Git you love. But the way I think of this as a former physicist is entropy is there. It’s too hard to keep track of everything. Humans can’t remember everything.

    We need to be able to describe, as application developers have been doing for the past couple decades, in code the versions of things, the documentation associated with everything. And then the automation that we’ve had in CI/CD, which people think of, oh okay, that’s developers pushing their code out, well there’s no reason why we have to stop at application level. It needs to be the whole thing, because application and infrastructure must stay in sync at all times. So by putting a description of both the application, the infrastructure, and within the surrounding environment, all three of those into Git, you’re bringing the best practices of AppDev into the whole world together so they’re all synchronized.

    Ashley: Yeah. I like that description too, Parag, because in a way that’s breaking down silos that we still have around the code build and deployment process. And it’s almost like if you think back when we started using code repositories and, okay, everybody put their source code in one place, okay, let’s expand that to artifacts and other things. Well, now we’re doing the same thing of let’s abstract at one more level and make the whole thing as cradle to grave, if you will, for getting code out into product. Whether it’s any part of that stack, it’s a GitOps process.

    So to your point, Tracy, about it’s an immutable object, it’s something that we know from beginning to end, it’s all been in one environment and we happen to have manual processes that have just been automated that potentially are weak or could be broken.

    Doshi: I’ll add one more comment on top of Branden’s about – I think it was around – it came out of cloud native computing foundation, but really the way I look at containers is it doesn’t matter that’s a container or VM. We’ve just moved a control plane up to a higher level, but the same principle’s apply. So there’s no reason why you cannot use these same principles in a container type of environment or a non-container environment. The only difference I would say with containers is we can be very surgical, fine grain, down to each individual pot. So there’s a little advantage with cloud native architectures to be very fine grained.

    O’Leary: Yeah, I agree. And I think that’s important for folks to separate the concept of GitOps, the principles that we’ve all talked about. And I agree with the other panelists and how they’ve talked about it. But separate those principles from the technology, right? Because it’s not always true that Kubernetes is going to be the right technology for you to deploy every workload you have in, but still this concept of GitOps.

    And I really liked Tracy’s sealed container – well, overloading the word container there. [Laughter] But that sealed deployment that doesn’t have a lot of human – or doesn’t have any human interaction. That is what gets us to what Mitch was talking about, which is we want to take this isolated, people working in isolation, the kind of cave-like nature that operations typically used to have where you had to have this specialized knowledge that one person knew. Oh, wait. How do we deploy that service?

    Oh, we’ve got to call up Alan. He’s the one that knows how to do that. [Laughter] Right? You want to get rid of that and make it all, what Tracy said, which is like a hermetically-sealed deployment that we know we can reproduce every time.

    Ragan: Yeah. So GitOps was a brilliant way to leverage existing tools and habits to create Chef and Puppet. That is basically what was done. I mean, it has a Puppet kind of master architecture. You have a GitOps operator that’s running on your cluster. It does a pull based on what’s in Git. It very much took the basic concept of Chef and Puppet and leveraged existing tools that people already had, some open source, GitOps operators out there, and they were off and running. And that is the brilliance and beauty of GitOps. I mean, I have a lot of issues with GitOps, and I’m sure we’ll share them, but its brilliance is in that simplicity.

    Shimel: The irony though, Tracy, is that many people will tell you that Kubernetes killed Chef and Puppet, right? And we’ve seen Chef get sold maybe for less than money raises in valuation previously.

    Ragan: No, I’m just talking about the basic concepts.

    Shimel: No, no, I get it. But if we look at the concepts behind what Chef and Puppet did, which was automate that deployment, deployment automation like that, and then with Kubernetes it sort of negated the need for that, but yet we kind of snuck that in the back door. Except we put it in Git to use in Kubernetes. So the irony of that is pretty profound, right?

    Ragan: Well, it’s all about the container.

    Ashley: Right, right.

    Shimel: Yeah. You’re right. Maybe it’s because Chef and Puppet – [Crosstalk]

    Ragan: The containers simplified, yes.

    Shimel: – didn’t figure out how to work in the container. Look, for that matter Jenkins too. Right? We went to Jenkins X because of all of that and everything else.

    Ashley: That’s a really great example, yeah.

    Shimel: But what I’m reminded is of the old Who song, right? “Meet the new boss, same as the old boss.”

    Ragan: Exactly.

    Shimel: And you still have that old boss. With that being said, here’s another thing, and Parag, I’m going to ask you to kick off as I take us a direction here, and that is GitOps, yes, it’s for deployment. What about testing? I said this is all about testing. We’re doing our testing in Git. Is that the – that stops, right? Is that better? Worse? Is that the way it should be, always, sometimes, never?

    Doshi: So I don’t know if I’d call it testing in Git, but I would say now Git becomes a source of truth. So I think what Tracy said is right, that it’s just that same sort of automation of infrastructure but now moved into making Git the centerpiece of where you define this. So the Git part is unchanged, but the testing is now event driven. Right? So because you’ve merged in code, you’ve committed code or gone to do a pull request, that can trigger testing.

    And we call this continuous testing. And I think the evolution of this can even go as far as our test definitions themselves should also be put into Git. That’s the next evolution that we’ll have. I don’t believe it’s really commonplace just yet, but when that happens then, we are trying to describe as much as possible into Git as a repository and use the principles of version control added with the event-driven concepts.

    So when a developer does X or when an operator is in environment Y and something needs to change, that whole Git workflow can invite continuous testing to say do I allow that to merge back in? If you want to have an approval, you could do those kinds of things. So to me, incorporating continuous testing, and the day that we put test definitions into Git as well, we’ve gone quite a bit further of completing this vision and really making DevOps real. Right? Because DevOps isn’t just releasing really fast, deploying really fast. It’s deploying fast with confidence and ease. That’s what we need.

    Shimel: Thoughts on that, panel?

    Ashley: It sure seems like a natural evolution to include the test environment. And as you say, Parag, it’s probably not commonplace yet today. A question for you about that; do you think that there’s incentive with the folks that are taking care of the testing, whether they be the QA team or developers writing tests, things like that? Do you think they have the right incentives to move it into Git or will it be something else that causes it to happen?

    Doshi: I think as long as people have the incentive of delivering value as quickly as possible with the best meantime to recover, the lowest defect leakage rate, as long as we do break down those silos, as you said. If the testing team is, okay, well, I don’t understand what the developers do, just throw it over the wall and then I do my test, that’s not going to work. This is talking about dev testers, the full stack developers. And as long as everybody’s motivation is right, that we’re trying to deliver value faster and faster, that we will not have confidence – that a VP will not say let my applications go out – you know, they’re only going to look at portfolios of apps and say what amount of testing has been done? What amount of security checks have been done?

    That’s another thing. The environment. Okay? That’s the biggest entropy of all. We have the least control over the environment. The same code, the same infrastructure in different environments behaves differently. Until we get that environment description also into Git, coupled with the testing model themselves that says here’s how we test it in that environment or that application or this infrastructure pattern, then DevOps becomes real. Until then, this is all talk.

    O’Leary: Yeah, I agree. I think it’s critical, and it’s something that GitOps, again, this concept of GitOps, I think is exciting to a lot of people that have been trying to make DevOps real for so long, because it enables that in a way. Right? We see kind of the light at the end of the tunnel, I would say. And if that is all brought into Git and all made into something that can deploy independent of any human interaction, well now you have the ability to do continuous testing. You have the ability to stand up an identical environment, the golden goose or [Laughter] golden egg of a, hey, we want an environment that’s exactly like production to really test it and really hit with security tests and know that when we do deploy to production we’re going to have that confidence.

    So if you’re able to continue to iterate on the environment, the infrastructure, just like you’re iterating on the code, it enables you to then say, okay, now we can not only have the code to review and maybe some static code analysis that happens and some testing that happens, and then maybe some human testing that happens, now we’ve also got an entire environment spun up automatically that mirrors production, where we can have a high level of confidence that it’s going to work when it really ships.

    Shimel: So guys, you’re telling me DevOps hasn’t been real quite yet. I feel like I’m living in the Matrix, kind of living in a simulation. Tracy, I sense you’re Agent Smith here, though. You have a different view. Maybe GitOps isn’t the panacea it is or it’s been projected here. What’s your view?

    Ragan: So no hate mail, okay? No hate mail. [Laughter]

    Because I know there’s a lot of enthusiasts out there, and rightfully so. But let me just level set the way I see where we’re headed and where GitOps can’t take us. GitOps is imperative. It’s scripted. And it works really well – so think about how most people use GitOps.

    You have a code repository that you keep all your source code in. You make a change to your source code. You check that in just the same as you always did. You also have an environment repository, and what lives in your environment repository is your Kubernetes deployment YAML file. And that is where you define all of the stuff that we’ve been talking about and that’s what the GitOps operator looks at and pulls across.

    Now if you’re in a Kubernetes environment and you have containerized your application – this works great, it really does – and you don’t have a whole lot of environments – you have three environments dev, test, and prod – because remember, in a Kubernetes YAML file there’s a couple of things that are important to the GitOps operator. One is the tag of the container. So once you register your container image you get a SHA, and that makes it a unique identifier. And you have to update your YAML script with that SHA.

    Now dev, test and prod may have different values, configuration values. So you have to branch it. So you branch dev, test, and prod. So now you have three versions of that YAML file to manage your container deployment. And that’s pretty doable for most organizations today, but it is still imperative. Everything’s being defined by the script for each of those environments.

    Now let’s go and expand it and scale it. Let’s scale it to a microservice environment. Every microservice has its own YAML file because it should be independently deployed. And they should be shared, because that’s the beauty of a service-oriented architecture. They should be shared across teams. So what you do is as an application team, you’re going to make your coding changes in your coding repository and then you’re going to have different YAML files for the different services.

    Now let’s say you’re a store. I use the candy store and hipster store and the clothing store. They all use the card service. Well, they’re all going to put their card service YAML file into their environment repository, and that’s going to get deployed to their own clusters and name spaces. So you have a scaling issue, and it’s because of the imperative nature of GitOps.

    We are starting to move and have more conversations about declarative and events and catalogs, where this stuff is going to be stored and generated at runtime. So for example, instead of having all of these scripts, you’re going to have central places to go to say what’s the key value pair for this environment? I’m going to pull this registry information that has the SHA, I’m going to go get my key value pairs, and that way you don’t have to have so many scripts and you’re not managing – it’s onerous.

    It really can become onerous. We have customers who have 200 microservice and 20 clusters. That’s a lot of scripts. And if you’re an SRE and you want to update your microservice and you want to provide it to everybody, you’re going to have to figure out every single application environment repository that has it in there for every single environment and make those corrections. So it doesn’t scale as well as it should. Now I know that their CNCF has a group, a working group, around GitOps, and they’re starting to think about GitOps version two, and they’re going to have to address these kinds of core problems around an imperative process.

    Shimel: Love it. Brendan?

    O’Leary: Yeah. I mean, I like that concept and I think you’re right, Tracy, that the ability to manage the scale is definitely something you have to be prepared for if you’re thinking about GitOps, because if you just take it at its face value you end up with all those scripts and it could make the problems you have worse rather than fixing some of the problems you hope it would come into and solve. And so I think in a cloud native environment, again, it maybe pairs a little bit more simply, but I think that’s why again it’s great to think about the concept of focusing on that developer-centric approach and the experience that they’re going to have when it comes to shipping something out, to your point of if I want to ship an update to my microservice that impacts everyone, I don’t want to have to go through 78 repositories and update that everywhere.

    And so having GitOps do the infrastructure management in the same version control system as application development is great, but you also have to have a plan in place for how does that scale to the scale that you’re really going to build to? If that the scale of developers, if that’s the scale of services, if that’s the scale of environments. And so I think it’s important to have tooling and an approach in place that accounts for that as well.

    Shimel: Parag, you were going to say something I think before we –

    Doshi: Yeah. Sure. So I love what Tracy described there, and getting into the specifics of the proliferation of YAML files, and especially as we break down microservices. So this is the whole concern with microservices. What you took as monolith and you described as one big black box, now you’ve broken it down. And so now we have the proliferation of all these things. Right?

    So I like things like Helm to help to reduce that and parameterize all these things. But fundamentally with GitOps, the concepts are there. Like I said, infrastructure as code with an event-driven type of approach version control. What I like about CNCF and containerization is the purpose of containers inherently – and Branden can keep me honest on this – is really to make the environment completely separated from your solution, your application, your infrastructure. Right? So today’s industry, the best approach we have is containers or serverless, which is also part of CNCF, so that we have things like Docker files and YAMLs that allow us to reduce the entropy of the environment as we can as much as possible and describe them declaratively in things like GAML files.

    And I do agree with Tracy 100%, that now it becomes, okay, well now I’ve got all these different configs, pulled that out of my code. Even with service mesh technology and CNCF we’re pulling out things like security and rate limiters between every single microservice. All that’s taken out of the logic, the business logic, on everybody doing security different ways. Now it becomes a configuration issue. How do I – if I want to change a configuration, do I need to change 100 different YAML files or config maps? And I think the technology’s evolving.

    Help is just one example. And then I’ll associate that with all the continuous delivery. You all know Spinnaker and a bunch of different technologies in this area, Octopus and so forth, that can help.

    Shimel: Yep. Mitch, did you want to say something? Otherwise, I want to – I’ve got another –

    Ashley: All I can add to the conversation is version control across all of this also gets very complex. [Laughter] I think you all –

    Shimel: But isn’t that what Git and Repositories are supposed to help us with anyway?

    Ashley: It is, but now you’re versioning all of your scripts along with all the environments and all of the variables of what you’re trying to script.

    Ragan: And it’s not relational.

    Ashley: Yeah.

    Ragan: You know, when you have – what’s the blast radius of my microservice? I’m going to push it out because my hipster store needs the new card service. If you’re using an SOA architecture that means that you have one card service and everybody’s using that version and that’s the best way to move forward because you eliminate the sprawl. If you put it all in its own namespace, you have multiple versions of the card service. So now you have a problem with the card service and you have to go update all the namespaces that’s using that card service.

    So you have to have a relational model to be able to track what applications or what namespaces are using what microservices. And unfortunately, Git is not relational. We create relations using tags, using different repositories, but that becomes redundant.

    Ashley: Great example.

    Shimel: Fair enough. So let me –

    Doshi: Tracy, you talked about the structure.

    Shimel: Oh, go ahead, Parag.

    Doshi: I was just going to follow up and ask a question to Tracy. I think what you’re saying there is Git is just a big version control repository. If we can provide some extra structure and relationships around all these artifacts that we’re storing there we might have better control.

    Ragan: And that’s what you call catalog.

    Doshi: Yeah.

    Ragan: Right? So I think we’re starting – when we really look at a service-oriented architecture and we look at – let’s say, for example – let’s push ourselves farther into the future. And Brendan brought this up around service mesh. And I believe this can happen and will happen. I believe service mesh will be enhanced on.

    Right now you can say do a deployment and deploy out to 10% of my users. I think service mesh will start looking at active directory and say deploy out to my developers. So now we have one cluster and we have immutable services out there. We put a new service out there, it gets requests routed to the developers, they do their testing on it, once it’s good it’s rerouted to their testing community.

    Then it’s routed to a certain group of their production community. Which means that we don’t have deployments across multiple clusters. We’re going to start pushing into a one big cluster, which sounds a lot like the mainframe. [Laughter] But we’ve come full circle. And as Alan said, the song, right? Meet the new boss, same as the old boss. [Laughter]

    Shimel: Right. We won’t get fooled again. That is true. There is certainly some truth to that. I wanted to spend a little time talking about the impact of GitOps on the CI/CD market. We’ve written in devopsy.kinsta.cloud several articles, the gist of which was DevOps is the symptom of disease. CI/CD is the cure, if you will. Right? And we’ve built this whole DevOps industry around CI/CD solutions.

    And I mean look, GitLab’s a great example. They had a great Git product to begin with. But it was their CI/CD solution that kind of moved them into that end-to-end DevOps, if you will. Right? GitOps I think has profoundly changed that market because it’s profoundly changed the way we do CI/CD. Well, that and Kubernetes, right?

    Kubernetes I think was the initiator of it. What does it mean for the market? What does it mean for our audience out here? And we have a technical audience, 60% of which are practitioners, 30% are managers or higher, and 10% are just, I don’t know, vendors. But what does it mean for them out there?

    What does it mean? I mean, these CI/CD tools that they’ve gotten used to, that they’ve grown up with a bit over these last years. What does it mean for them?

    O’Leary: Well, Yeah. I think that we love to say CI/CD together. I mean, I love to say CI/CD together, right? That’s now we name our product. But those concerns have always really been separate and never really been completed merged and married. Like it’s great to try and think about them as a single unit – CI slash CD, right?

    But I think the reality is that CI is something that has been around for a really long time, probably 20-plus years if not more, but CD is something that a lot of organizations still struggle with for any number of reasons, whether that’s a technical reason, whether that’s an organizational structure reason, whether that’s a risk aversion reason. The idea of doing continuous deployment is something that really is very dependent on your organizational structure, who your end users are, and the, yes, to some degree like you said, Alan, the tooling that you choose. And what we’ve seen in traditional CI/CD tooling is this concept of applying what we do in CI, which is scripting based on an event, running through a number of scripts.

    That is what then becomes CD. Well, now we’re going to push with scripts out to the environment. And so GitOps, you noticed in my definition I didn’t say that a pull was necessary. I think you can do push-based GitOps if you have infrastructure as code, you have MRs at the point of change control, and you have the ability to do CI/CD. But I think most people when they hear GitOps think about this another new concept, which is this concept of pulling. Right? We’ve heard Tracy talk about that.

    And so having an agent, an active agent inside of your infrastructure that is constantly ensuring that your desired state is the actual state in the world – that’s what Kubernetes brought to us, that’s what – and then that’s what pull-based GitOps brings to us. And so that’s why for us we’ve been doing CD for years, like you said, Alan, and we’re doing mostly push-based. But as our customers’ needs expanded in GitOps we’ve adopted a Kubernetes operator and now we have our own Kubernetes operator, and you’re seeing this more and more in all the tools, that runs inside of the cluster and pulls changes in.

    And that methodology, while – again, credit Tracy – sounds a lot like Puppet, [Laughter] it still is something that was the alure of Kubernetes for a lot of people. Hey, I want to describe my desired state and have something else worry about that desired state always being true regardless of if I push something wrong once. I want this desired state to always be correct. And so I think that’s how we’re going to see a huge change in the tool space. Right?

    To Parag’s point, we’re talking about Spinnaker. That’s something that Netflix has been using for a long time, but no one ever thought that they would be deploying like Netflix. Well, now it’s a huge deal and lots of people are talking about it.

    Shimel: We are.

    O’Leary: And they are, right? And so it’s very interesting.

    Shimel: Mitchell, it sounds like an analysis question.

    Ashley: Well, I was thinking about that very much, and I appreciate Brendan’s perspective on it. Because any time you have a major paradigm change, you have early adopters and then you have a long tail of how that adopts out. And you’re talking about the Netflix. People are now starting – how long have we been talking about Netflix as the example? As long as we’ve had DevOps and continuous deployment.

    And I think we have a bit of the same challenge here of instead of still having layers of our stack and different responsibilities or processes or things that go into creating environments, whether it’s from code or not, when we say it’s all going to be based on a pull and it’s all going to happen in its own environment, if you will, that’s a bit of a big pill to swallow for a lot of organizations, especially in complex applications. I’m not saying it’s not going to happen, but I think it’s going to be slow to adopt. And I’m sorry if I’m going to get a bunch of hate mail.

    It’s a great concept, but I do think it’s going to be more difficult to adopt than even containers were for breaking down applications. Not that it’s not a good idea or the right idea, it’s just instead of software architectures, modern software architectures being something the developers now get their mind around with containers and microservices and fabrics, meshes, now you’re talking about different parts of the organizations going into infrastructure and ops. And that’s all getting compacted together and you’ve got multiple parts of the organization getting behind that paradigm shift. So that’s my view of it.

    Will we see things in the market around GitOps? Of course we will. But I don’t think we’re going to see this tipping point that will happen rapidly. That’s my assessment.

    Shimel: Okay. So it seems like you and Tracy have formed some sort of coalition here, and –

    Ragan: [Laughter]

    Ashley: I always go to Tracy when I need something _____.

    Shimel: Right. Send the hate mail that way.

    Ashley: _____ Brendan and Parag, too. [Laughter]

    Shimel: Right.

    Ragan: I want to say that the CI/CD pipeline can adopt GitOps quite efficiently. Think what Brendan pointed out. When you think about CI/CD you think everything’s a push. Right? And you add a script that you’ve executed that your CI/CD server, your orchestration server, your job scheduler said, “Hey, there’s a step here.” Then that step goes out and pushes an update to production environment.

    How that’s going to – because GitOps is a pull methodology it doesn’t have a promotion methodology. And what really we’re going to see is for folks using GitOps is we’re going to see the CI/CD process do the – create the pull request. And there will be an approval. If you want a pull request to go into production, there’s going to have to be probably a manual step to approve that pull request, and then once it gets there the CI/CD pipeline creates it. Now the challenge will be having the CI/CD pipeline automatically update all of those declarative scripts. [Laughter]

    Because there will have to be a different one based on the environment. But that is totally doable and it doesn’t change that. But what it does do, it says, hey, CI/CD. You need to have a hermetic process. It needs to be airtight. You need to have a single source of truth.

    What that is ultimately will still be called GitOps, even if it’s not Git that is that single source of truth. It’ll be like we reach for a Kleenex, not a tissue. [Laughter]

    Doshi: Let me back up what Tracy just said, and maybe take the view of – I think you said 30% of the listeners are looking at this from a business angle. And your question also sounded like what does this mean as our industry, to our industry? So today we’ve got, I don’t know, 170 different products, if you will, and tools around – and Brendan can probably say, no, it’s now 250 or something [Laughter] – around DevOps and all the new different concepts.

    Shimel: Closer to 700.

    Doshi: Oh, okay. My bad. [Laughter]

    I was a little behind. So we have so many things here, and there’s a reason for that too, right? So we can’t just say, hey, let me take a cloud provider’s full CI/CD DevOps vision and hope that we’re going to impose one way of doing things, because this is not all worked out. The moment you bring the developer mindset to other layers of the computer stack – infrastructure, environment, operations – it becomes so complex. And I think at an industry what it’s going to do is it’s going to actually spur a lot of opportunity, a lot more innovation.

    I don’t know if that means that 750 will go to 1,050, but there’s so much room for different companies that will do things differently – push, pull, I need an approval before production due to a segregation of duties. No, no, no. I want to approve before I even commit the code or at least run the security static analysis, the dynamic analysis, the container image scanning. Everybody will have different ways of doing things, and I think it’s going to spur on lots of opportunities for a lot of companies.

    Shimel: Fair enough. A giant petri dish. Guys, I’m going to need to pull the plug on us here because we’re up against our time limit. But what a great conversation. For our audience, in a couple of weeks – and I don’t have the date in front of me, I apologize – we’re actually going to do an interact roundtable on this subject. Some of our panel members will be back for that.

    And we’re going to allow you to come online and ask questions and comment and join the conversation on this. I think it’s a great conversation. So be on the lookout for that and come check us out. But in the meantime, Tracy, as always, you know, you bring so much to this. Thank you so much. Thank you so, so, much.

    Ragan: Oh, it’s my pleasure. Absolutely my pleasure.

    Shimel: Yes. Brendan, always a pleasure to have you on, my friend.

    O’Leary: Yeah.

    Shimel: Look forward – also, Brendan does a show with us here on TechStrong TV every month. It’s called The Lab. And he discusses – you know, the 22nd of every month is a new GitLab release and Brendan gives us the lowdown on each month’s release. It comes out once a month. It’s called The Lab, and it’s available here on TechStrong TV. So Brendan, thanks so much for that.

    O’Leary: Yeah. Thanks, Alan, for having me. I always learn so much from these panels. So hopefully we all learned something today, but I really enjoy doing them.

    Shimel: Always a pleasure. And you add as much as you learn, Brendan. Thank you. Parag, one question for you. You mentioned earlier that you were a former physicist. Is there such a thing as a former physicist? Isn’t it more like I think, therefore I am?

    Doshi: Yes.

    Shimel: Once a physicist, always a physicist?

    Doshi: It certainly is Hotel California. You’re absolutely right. [Laughter]

    Shimel: Yep. But thank you so much for joining us. Always great to have you here and bringing some great perspective to everything. I appreciate it.

    Doshi: Thank you.

    Shimel: Mitchell, what can I say? We wrapped another one, Mitch. Good stuff.

    Ashley: Yeah, we did. And I’m just so glad that Parag didn’t bring quantum physics into this, because I’d be in trouble.

    Shimel: That could be another show we’ll do.

    Doshi: [Inaudible comment]

    Shimel: Quantum computing. But, no, anyway, that’s going to wrap up this episode of DevOps Unbound. We hope you’ve enjoyed it. This is every other week we do a fresh DevOps Unbound, so be on the lookout. Every month we do the roundtable with the audience, so be on the lookout for that as well. This is Alan Shimel for MediaOps. You’ve – oh, before we go, many thanks to Tricentis for sponsoring DevOps Unbound. We couldn’t do it without them. So thank you all very much. We’ll see you on another episode.

  • Languages and DevOps: Network

    Languages and DevOps: Network

    It is fun to discuss the changes in network operations over the last couple decades. We’ve come full circle, and, honestly, ended up in a better place. Network specialists were super command-line wizards who kept us going, with few GUIs in sight. Everyone thought the next iteration would be GUIs, so network operations could be accessible to more than a few command-line specialists. “Everyone” was mostly right; we did head in that direction. It was optional, and many network operations friends pointed out that you could still do more, faster with the command line. Before GUIs had the opportunity to catch up with command lines, though, automation came along. And then DevOps. And finally,* configuration-as-code. Pretty quickly, the ability to do everything through an individual product’s GUI was determined to be a weakness. Network operations personnel were dealing with more vendors to handle the networkification (it is too a word! I just made it up) of common development activities, like some aspects of security. And now, APIs and command lines rule the juncture of DevOps and networking.

    *Finally – so far, at least

    And that shows in languages. Shell scripting and Python run this show, with little competition from any other language. We could get into a massive discussion about whether shell scripting is programming, but we won’t, because at least in networking, it most definitely is considered to be programming.

    One time, when we were tracking down a problem that involved all of IT, I was sitting with the serious network geeks and marveling at their ability to make everything they wanted happen from the command line in an instant, and their ability to parse command output. I’m no slouch when it comes to networking, but they easily put me to shame. I got to chatting with a few of them, and the consensus was that they loved computing but hated programming. Networking made sense to them. This explains why networking DevOps people are continuing to use Shell scripting and adopting Python at the same time.

    Python and Shell

    Shell is a given in networking. There are a lot of extant Shell scripts for doing all sorts of things, and the need to automate to keep up with Agile makes shell scripting a logical choice.

    Python enters the picture because it is “common ground.” It is intuitive enough for most networking people to do what they need to get done, and known or knowable to most developers. When it comes to usage, Python has overtaken Shell; (don’t take my word for it, check out the excellent work of the NetDevOps survey on GitHub, or just glance at the graphic below, created by them) in fact, Shell and Python command so much of the network development space that you are probably better off just making sure you’ve got those down, and focusing your free time on other issues like completely different networking commands, or even skills for data center versus cloud. The ROI would be better.

    But Wait, There’s More

    The key operational changes recently have been around GitOps and reporting back into the DevOps toolchain. People no longer want your appliance or application to have a pretty GUI, they want it to report in to the tools that provide automation and a larger-picture GUI. Many vendors are there already, but many are not yet. That creates a need for tools – products, open source projects, or internally developed – to do that integration while waiting for the vendors to catch up. These, again, are well-served by Python, though other languages that can handle APIs are viable, too.

    We’re headed for a world where systems detect anomalies and react to rectify them, but we’re not there yet – not on a strategic scale, at least. And we probably won’t be for some time, as automating massive network changes is a scary proposition and will take time to use these systems for detection, monitoring what they could have done, before organizations are willing to let them “just do it.”

    Carry On

    Meanwhile, NetDevOps is getting folded into DevOps proper, and in many organizations, using infrastructure-as-code or GitOps, is being deployed and managed like any other part of the life cycle. So we are moving along. For my network peeps, your job will continue to change, but it certainly won’t go away. You’re essential to smooth-running systems, and while the network may become even more virtual, it will still be networking. And under that virtual network will be … a physical network. So keep kicking it, and carving a path from users to apps, while protecting those apps and ensuring they have adequate resources. Oh, and if I spin up another instance at midnight, you’ll hop on and make sure it’s got traffic directed to it, right? RIGHT?!

  • Languages and DevOps: To The Cloud!

    Languages and DevOps: To The Cloud!

    Cloud and DevOps are almost a match made in heaven. Cloud started truly gaining steam in the age of DevOps and Agile, and has adapted – or grown up in it – well, depending on your viewpoint. If not for the combined issues of data gravity and data (im)mobility, we might have gone even farther than we did into the cloudy world.

    While the breadth of possible cloud apps means a wide variety of languages is in use, Node rules this roost. Every conceivable language is used in cloud development – because, in the end, they’re just servers – but Node is the go-to language of today. The broad use and appeal of JavaScript make it an obvious choice for enabling full-stack development, and the pool of developers is larger than any other language today. Other languages are used, but not in the numbers that Node is. Python is the challenger, and a surprising number of PHP sites still exist, but forward-looking development aims at Node or Python. Considering the distance from the hardware, there is less call for C in cloud than there is in other areas of development, but it isn’t gone, since interfacing to the OS is also often done in C.

    Targeting cloud is not so much about language as it is DevOps tooling, though. All of the major Git companies have cloud-centric extensions (this is the core of GitOps) that raise the level of “fully automated.” Cloud environments, by their nature, provide APIs for infrastructure, and the Git vendors (and many, many others) have hooked into those APIs to make deployment an integrated part of DevOps at a level of detail that other environments (with the exception of containers) require a lot of work to match.

    The negative to cloud-centric development is lock-in, of course. Creating an entire DevOps toolchain and process that supports vendor X will not work directly with vendor Y. Tools can help with this issue, but the real solution is in our next installment – container development.

    The part of development that arguably benefits the most from cloud-centric development is testing. It is easier to spin up a test environment as-needed and spin it back down when testing is done – no need for a spare set of hardware or a stand-alone VM server; just redirect the DevOps ecosystem and spin it up, then account for data (scrubbing, copying, etc), and you’re off. Codify the steps for data, and you have a re-spinnable environment for test-on-demand.

    The part of cloud that is arguably the weakest is security. Let’s face it, far more of your infrastructure is publicly (or semi-publicly, if running in a VPC) exposed than in a data center. There are far more places to lock down, and a sometimes-bewildering number of options to secure. But the flexibility of pay-as-you-grow makes it an appealing option, and there are tools to help manage all those lockdown points. Just make sure that you’re not the weak point in the shared security model.

    You. Rock.

    Cloud makes a lot of things easier, and Node makes development of any step in the stack easier by using the same underlying language on the server that is used in browsers and in many mobile development platforms. Node “feels” more right for web development, at least because one connection, one call … even if it makes long-running processes require a bit more work.

    Whatever you choose, pay attention to how deeply invested in the cloud vendor’s proprietary interfaces you are, because, like all technical decisions, one day you or your replacement will have to get around those interfaces. And keep making the software the org needs.

  • Operationalizing Cloud Native with Kubernetes Virtual Summit Speaker Spotlight: Tracy Ragan, Cornelia Davis, Ashley Ward and Rob Hirschfeld

    Operationalizing Cloud Native with Kubernetes Virtual Summit Speaker Spotlight: Tracy Ragan, Cornelia Davis, Ashley Ward and Rob Hirschfeld

    The Operationalizing Cloud Native with Kubernetes Virtual Summit will bring together Kubernetes and cloud-native community members for a unique learning experience focused on the tools and best practices you need in order to successfully secure and manage Kubernetes. Presented in a virtual reality environment, the event will take place on Oct. 1, at 10 a.m. EDT.

    Operationalizing Cloud Native with Kubernetes - speakers - Tracy Ragan - Cornelia Davis - Ashley Ward - Rob Hirschfeld

    At the The Operationalizing Cloud Native with Kubernetes Virtual Summit, Kubernetes technologists, DevOps and cloud-native experts will be educating the audience on cloud-native technologies and digital transformation with Kubernetes. Tracy Ragan of DeployHub, Cornelia Davis of Weaveworks, Ashley Ward of Palo Alto Networks and Rob Hirschfeld of RackN are a few of the outstanding speakers that will be sharing their knowledge and insights with the community at the Operationalizing Cloud Native with Kubernetes Virtual Summit.

    Recognized as a Women In Technology (WIT) Honoree for her work in DevOps, DeployHub Co-Founder and CEO Tracy Ragan is an expert in configuration management and pipeline life cycle practices with a hyper focus on microservices and cloud-native architectures. Tracy is a well-known DevOps evangelist and DevOps Institute Ambassador. She serves as board member of the CDF, where she is the elected General Member Representative, and she is also the creator of the Continuous Delivery Foundation Interactive Landscape. Prior to DeployHub, Tracy was co-founder and COO at OpenMake Software.

    During her talk, “Achieving Business Agility with Microservices,” Tracy will explain what application teams must do in order to achieve business agility. She will explore how microservices are truly the last mile of Agile practices and an essential component for the creation of a modern software architecture.

    One of our most distinguished speakers is Cornelia Davis, CTO at Weaveworks and author of “Cloud Native Patterns: Designing Change-Tolerant Software.” Cornelia is an industry veteran with almost 30 years of experience in image processing, scientific visualization, distributed systems, web app architectures and cloud-native platforms. She is responsible for Weaveworks’s technology strategy, inclusive of open source projects, commercial products and services offerings. Cornelia is driven by the desire to help enterprises transform their business through the leverage of cloud computing platforms.

    In her session, “The Business Benefits of GitOps,” Cornelia will cover key GitOps principles and practices and tie them to measurable IT outcomes, such as change failure rates and mean time to recovery. These IT metrics are intrinsically tied to digital transformation initiatives. She will also review research that has shown a correlation between these measures and where companies fall on the spectrum from high to low performers in terms of business outcomes.

    Ashley Ward, CTO for Prisma Cloud at Palo Alto Networks, is another of our brilliant speakers presenting at the virtual event. Ashley aids customers in the secure deployment and scaling of cloud native environments. He has over 15 years of experience in the operations and infrastructure architect space.

    During the session “Kubernetes Security – Defense in Depth,” Ashley covers the areas that need attention when securing the Kubernetes stack.

    “Kubernetes can seem complicated and even frightening when trying to secure it,” said Ashley. “I hope that people takeaway an understanding of why and how they can secure their Kubernetes clusters and resources.”

    He will also introduce the audience to the concept of a cloud-native application protection platform.

    Another DevOps and cloud-native expert joining as a speaker at this year’s virtual event is Rob Hirschfeld, founder and CEO at RackN. Rob has been creating software to automate infrastructure for over 20 years. He has been involved in numerous open source efforts including LF Edge (Board), CNCF (SIG Chair) and OpenStack (Board).

    Rob will show how EdgeLab.digital uses Digital Rebar to provide PXE boot, immutable O/S and API-driven automation control of RPis to build Kubernetes (k3s) and OpenFaaS.

    “This session is a ground-up demo of the entire process and background about why it’s important to have a proven reference architecture,” said Rob. “This is designed as a learning environment that can also scale to real edge and enterprise use.”

    Attendees will learn how this easy-to-duplicate-at-home demo creates a true desktop data center using the community edition of Digital Rebar to enable netboot for RPis and install CNCF sandbox project K3s. The same code learned in this demo session can run any COTS infrastructure and is currently running multinational banks in the EdgeLab.

    You can hear from these speakers and more than 30 other industry leaders at Operationalizing Cloud Native with Kubernetes Virtual Summit. To check out the agenda, the full lineup of speakers and to register (for free), visit the Operationalizing Cloud Native with Kubernetes Virtual Summit website.

  • The Pros and Cons of Configuration-as-Code

    The Pros and Cons of Configuration-as-Code

    Configuration-as-code, or “GitOps” as many are now calling it, is a simple but astounding idea: Standardize configuration, check it into version control; no need for each Operations/DevOps person to develop their own set of scripts to get the job done. Configuring or reconfiguring an environment is as simple as checkout and run, assuming everything in the configuration is tested. For a project or a team, this is a serious time-saver and limits the amount of human error introduced into our systems. For projects rolling out code frequently, it is pretty much required.

    But like everything, it has its limits. I worked with a product management engineer once who had concocted a script that had tens of thousands of lines of code and was every bit as complex as the product he used it to manage. His scenario was the typical precursor to configuration-as-code: He would go show off the product and whatever functionality he needed could be managed through his script. The problem was maintenance; as the script was central to how he did his job, changes to it were a big deal. And even though he was the only one maintaining it, he did get burned a couple of times by changes he’d made, because it was that complex.

    When our configuration-as-code starts to cross teams, or we take it to the enterprise level, we risk finding its limitations. Our environments are complex, very complex. And we can do a lot with configuration-as-code, but throwing everything plus the kitchen sink into that system will create issues we need to overcome.

    A single project is constrained. We know what we need to do to spin up the environment. We can identify weak points and build checks and balances around them.

    The enterprise environment is not a project. Agile projects and DevOps thrive because they have a stable working environment to run on. The volume of change in things such as storage allocation, routing and VLANs is not the volume of change that an agile project sees. Good documentation is as solid a solution as configuration as code for something that changes rarely and the size/number of scripts and configurations that need to be maintained is smaller.

    There is a good use case for corporatewide configuration-as-code for recovery purposes. It is far easier to restore the environment in a disaster if you have all configuration and the scripts to install that configuration available. Unfortunately, that use case is minimized by the march of technological change. During the one total destruction restoration I’ve been involved in (was in charge of), many of our configurations were no longer usable because the products we were running in production at the time of the disaster were not the products we could purchase for the rebuild. We ended up with a shiny new data center from building to servers, but it still involved a lot of manual configuration. So while the use case is good, it is not a panacea unless you are constantly replacing your infrastructure with the newest products, limiting the usefulness of this use case also.

    Some organizations have already moved to configuration-as-code via methodologies such as chaos engineering, but most of us are just starting to consider moving configuration-as-code beyond the project level. When the (admittedly large) step of truly moving infrastructure management toward configuration-as-code, start with inventorying what your operations team does on a daily basis. There are a ton of things—the famous low-hanging fruit—that can be boiled down to scripts or standardized configurations and checked into version control. This gives a central point of truth for how things are configured, tracks changes for when things go wrong, and shortens startup time for new DevOps/Operations staff by giving them a repository of tools to review and work with. When working in the cloud, everything is scriptable, so make that side of operations is completely configuration-as-code. In the data center, you end up with diminishing returns after daily tasks are automated, but assigning someone to codify all possible changes is a worthy goal, as long as the work is prioritized to make the most frequent or error-prone configuration issues a priority. Once the focus is down to hardware, the rate of return for those invested hours is going to go down a bit, since specialized hardware is full of proprietary or market-focused config info. To draw on a comparison I’m familiar with: An F5 ADC has very specific use cases that must be reconfigured centrally, whereas restoring your Kubernetes cluster will generally require less focused configuration and teams spinning up containers will take it from there. Not that Kubernetes configuration is simple; but each project handles the details they require while corporate-shared resources tend to need all of the details handled by the infrastructure team.

    We can discuss this at length, and I have with others who think differently, but small tools are better. Do one thing, and do it well—the original Linux mantra—then make scripts that call the small ones. This contains changes over time and doesn’t create dependencies or touches on things that might not need it in a given situation.

    And keep rocking it. All configuration-as-code does is reduce manhours and opportunity for human error, so embrace it, but be controlled about it. Don’t throw everything in there and then forget about it. Make a plan, make a maintenance plan. Iterate. Improve. And use that extra time to solve user problems, which is why we’re here in the first place.

  • Instana Becomes First APM Solution with GitOps Enabled Agent Management

    Instana Agent Automatically Supports Git-Based Configuration Management to Streamline Configuration Rollouts Across Multiple Hosts

    Chicago, Il– July 22, 2020 – Instana, the leading provider of real-time application performance management (APM) solutions for cloud-native microservice applications, today announced the availability of the first APM solution with a GitOps enabled agent. With the release, Instana’s agent now automatically supports Git-based configuration management across any number of hosts.

    “Updating and configuring agents in production environments is a tedious and time consuming effort for organizations today. Of course, the more agents that are running, the more tedious the task,” said Chris Farrell, Instana Technical Director and APM Strategist. “With this latest unique capability, Instana is giving more control to users while reducing manual effort, enabling them to manage agent configuration automatically during agent startup and reboot, and version pin the Instana agent to via their Git repository of preference.”

    Traditional monitoring solutions give organizations a choice of using static agents or giving up version control. Instana has allowed both options, but now makes it easier for clients using Git repositories. The Instana agent supports Git-based configuration management, delivering version pinning control for specific hosts, automatic configuration updates fetched on agent startup and reboot, the flexibility to turn this feature on and off at will, and the ability to map different groups of host agents to Git branches based on the environment that is being monitored.

    In addition to manual agent configuration being a complex and time-consuming process, it also introduces a high risk of error that could break the monitoring of critical systems. Instana’s GitOps enabled host agent reduces that risk, leading to fewer configuration errors while providing the option for easy rollback if an update causes a problem. With this novel capability, Instana’s agent delivers improved customer experience and more control for users, allowing them to keep the latest version of the agent when it is most convenient.

    Unlike traditional APM tools, Instana’s automated Application Performance Monitoring (APM) solution discovers all application service components and application infrastructure, including Cloud infrastructure such as AWS and Lambda, orchestration infrastructure like Kubernetes and Docker, application services and DevOps processes. Instana automatically deploys monitoring sensors for each part of the application technology stack, traces all application requests and profiles every process – without requiring any human configuration or even application restarts. The solution detects application and infrastructure changes in real-time, adjusting its own models and visualizing the changes and any performance impact in seconds.

    Learn more about Instana’s GitOps enabled agent and Git-based configuration management by visiting https://instana.com.

  • Trends and Benefits of Serverless Computing

    Trends and Benefits of Serverless Computing

    Gartner estimated that by 2020, 20% of the world’s organizations will have gone serverless. Serverless computing initiates a mind shift in the way businesses are run and improves the accuracy and impact of technology services. Leading to higher functionality, reduced AWS bills, increased delivery speed and multi-cloud support, serverless computing paves the way into the future. 

    The scope of serverless is yet unexplored. With new and high-potential developments (such as FaaS and GitOps) still in their nascent stages, organizations can expect massive benefits leveraging the power of serverless computing going forward. 

    Servers have been a very important part of the computing architecture for decades. But, with the introduction of cloud, we’ve moved to predominantly web-based server infrastructure, and the migration to serverless architecture has begun.

    Serverless computing–abstract and virtual server support–has made the computing landscape more dynamic, robust, intuitive and powerful. But advancement in serverless technology is still in its nascent stage. The next few years are expected to bring in numerous developments and transformations.

    First, let’s take a look at some serverless trends you should keep an eye out for.

    Kubernetes Will Become the Go-to Serverless System

    Kubernetes has become synonymous with open-source containerization. But in the next few years, we will see Kubernetes become the base for serverless infrastructure. Features such as networking, service discovery, agile scaling and multi-cloud environment support make Kubernetes the perfect choice for serverless. 

    Additionally, with Knative already released, the process of standardization and interoperability of serverless platforms may soon become a reality. 

    Stateful Applications Will Also See the Usage of Serverless

    Restriction of serverless for stateless apps is a thing of the past. The next few years will see stateful apps benefiting from the immense power of serverless. Developers will be able to use database connections, BaaS and event payloads to run their applications.

    FaaS Industry Size Will Grow 

    The FaaS market is expected to grow to $7.72 billion by 2021. Large-scale adoption of serverless will be seen across the world, and there will be a 32.7% CGAR. 

    GitOps Will Be the Next Big Thing 

    The version-controlled system of DevOps, GitOps, will see more takers in the future, especially in FaaS. GitOps is expected to improve operational workflows, especially in fast-moving projects, and enable implementation of technologies such as Kubernetes and other cloud solutions. 

    Additionally, this will enable cloud vendors to provide greater ecosystem support and tooling capabilities to users.

    Containers Versus Serverless

    Containers by nature are segmented, isolated and contained spaces of application-specific servers. While they offer high degrees of specialized (and often customized) tools, resources and support, they suffer from numerous challenges:

    • They are very resource-inefficient compared to serverless. 
    • Their ecosystem is fractured as opposed to the wholesome environment offered by serverless. 
    • Data storage and retention are complicated, necessitating frequent data migration across containers. 
    • Graphics support is limited, resulting in poor-quality deployments. 

    The challenges that come with containers prompt organizations to choose serverless. Already, many companies have implemented serverless.

    Here is how the move to serverless will benefit you, too.

    Serverless Computing Eliminates the Need for the Set-Up of Complex Application Stacks

    Developing even the most basic server-bound application is a challenging task. It requires the set-up of high-powered and complex application stacks that can be both difficult to build and difficult to manage. By going serverless, organizations can eliminate all these heartaches. 

    Serverless stacks, such as Lambda and Azure, create a robust and intuitive environment for the development of agile applications, by providing organizations with a large set of capabilities and allowing them to run at scale, at the lowest costs.

    It Reduces the Organization’s Capital Expenditure

    Unfortunately, IT infrastructure is considered a capital expenditure by many organizations. So, building your own servers can lead to a high CapEx.

    Consider this. Physical servers are vulnerable to breakdown and depreciation, which add to CapEx. If you report this value as expenses in your income statement, it may ultimately negatively affect your P&L and your company valuation. Going serverless allows organizations to avoid such a challenge and maintain their profitability and high valuation.

    Plus, serverless infrastructure reduces the need for human resource deployment as focus on batch processing decreases. Additionally, serverless allows organizations to convert workflows into step functions, which reduces the number of EC2 and SWF issues. Together, these reduce company expenses tremendously. 

    It Shortens the Standard Release Cycle Significantly Due to Microservices and Nano-Services

    Serverless computing allows organizations to benefit from the magnificent power and scope of microservices. From making minute application-specific developments to isolating and rectifying issues, to managing and deploying individual applications, organizations can do a lot of things using serverless computing. 

    In fact, a survey shows that utilizing serverless microservices can bring down the standard release cycle from 65 days to a mere 16 days.

    It Allows Easy Completion of ELT Tasks and Cron-Type Jobs 

    Again, serverless computing microservices gives organizations the flexibility, technical support and clarity needed to process data. ELT tasks become more consistent and allow the creation of structured data warehouses.

    Similarly, since isolated components and applications can be created, deployed and fixed in serverless, it’s easier to implement cron-type jobs as well. 

    It Allows Organizations to Escape Traditional Workflows Through Event-Based Computing 

    Adoption of serverless computing allows organizations to shift from the restrictive, time-consuming and draining traditional workflows and engage in event-based computing. Now, coders can just write the algorithms and allow the serverless provider to take care of the data storage and computing needs. Not only will this reduce the time of transaction processing and the cost of DevOps, but it will also free up the time and space for coders to take on other business logic tasks. 

    It Enables Green Computing 

    With climate change becoming such an imminent threat to the world, it’s important for organizations to become mindful about the way they operate. One way to become sustainable is to go serverless.

    For one, serverless computing ensures organizations don’t over-utilize electricity. Additionally, with no physical servers and data centers that emit toxic radiation as they cool down, there will be lesser concerns about health and climate change. 

    Another aspect is scalability. Serverless is highly-scalable, meaning you don’t need to set-up more servers at your office. Given that 30% of the world’s servers are unused at any point in time and most servers only utilize 5%-15% of their total capacity, serverless may be the best way to go green.

    It Promotes Deployer Autonomy and Encourages the Adoption of New Mindsets

    Serverless computing gives organizations the freedom to focus on their core business offerings and deal with only low-level OS issues. This time, money and opportunity can be used by deployers, coders and top management to develop competitive offerings, which can help the organization overtake competitors. 

    With physical servers, it’s more likely than not for organizations to focus on technical server issues than the business process.

    Conclusion

    Serverless computing could be one of the most exciting developments of the 21st century. It is breaking down old barriers and allowing organizations to expand and explore new frontiers. Its high deliverability, coupled with the immense opportunities it promises, makes it a must-adopt in any organization.

    — Subramonian

  • An Inside Look at GitOps

    An Inside Look at GitOps

    At a time when variations on DevOps abound, get ready for the latest: GitOps. This new term promises to bring some order to what can be operational chaos and empower operational teams to leverage best practices learned from their developer counterparts. It’s early days yet for GitOps, but here’s what we think you need to understand about this new facet of DevOps.

    What is GitOps?

    Everyone we spoke with has a slightly different perspective on GitOps, but this is our take: In its most basic form, GitOps is infrastructure and operational procedures defined as code, and living in a source control system, such as Git. Git becomes the single source of truth for what is running in production (or staging, or development).

    Declarative systems, such as Terraform, AWS CloudFormation and Kubernetes help enable this practice; but operations is much more than Infrastructure as Code. True operations requires a way to plan, review and approve moves, adds and changes to the infrastructure. It also requires well documented procedures, or runbooks, to be successful. This is where GitOps differentiates itself. We believe GitOps is the full operations lifecycle of infrastructure management, living as code in a git repository.

    GitOps may be used for managing both application deployments and base infrastructure made up of systems needed regardless of the applications deployed (i.e. networking, security, etc.). Both will follow the same procedures to plan, review, approve and release code into production. Done right, GitOps will let engineers constantly apply updates to infrastructure meet all the needs of the applications in a continuous delivery format. The version control system ensures everything is recorded and visible and an audit trail keeps teams compliant. GitOps will make it easy to revert problematic changes, becoming a single source of truth about what is happening in the system from both the software development and infrastructure perspective.

    What Technology is Required?

    In order to kick off GitOps, you’ll use a Git-based version control system. Then you’ll need the ability to create a declarative cloud infrastructure, using something such as Terraform and Kubernetes, with the help of Helm or Flux. Because both Terraform and Kubernetes are declarative systems, they make it simpler to create and deploy infrastructure as code in a manner that is both replicable and transparent. And you’ll also want to implement a solution such as GitLab that wraps operational procedures around these tools. GitLab, being an end-to-end DevOps solution can manage the backlog of requests, assign those requests to the appropriate subject matter expert to create the change in Git, facilitate the review and approval process, and kick off the execution of the changes using its flexible CI/CD system.

    While it is possible to achieve this sort of workflow by using different solutions to merge and ship code to Kubernetes, GitLab is certainly a straightforward path to GitOps. It allows developers and operations administrators to work from the same source of truth and with a common set of procedures and tools.

    What Are the Benefits?

    In the DevOps world, GitOps is perhaps unique in that it can potentially benefit both operations team members and developers. For starters, GitOps aims to address the sometimes cave-like nature of operations (a single person working largely in isolation) by taking all of the best practices from software development and letting them loose on the deployment side.

    Full oversight, architectural review, project management and all the other pieces of the process are brought together in GitOps in an effort to end ops versus everyone else and get all the stakeholders involved in the process. The basics don’t change–an ops engineer will still create an infrastructure–but that will just be one part of a much larger process that everyone can contribute to and collaborate on. No more worrying about an engineer creating a cloud infrastructure that can’t be replicated; with GitOps everything is tracked making repeatability easy. Suddenly it will also be simple to answer the “what changed?” question because no one will have to hunt for an answer. The bottom line is a successful GitOps program will benefit the actual bottom line because operational processes will be dramatically streamlined.

    Possible Challenges

    With any collaborative effort, change can be tricky and GitOps is no exception to that. It is usually easier to create one-offs and not follow a process, so the first thing to consider about GitOps is it will require discipline from all participants. It will be vital for teams to write everything down.

    At GitLab we live and die by this thanks to our culture of collaboration, but we know it’s not necessarily something that comes naturally to any individual or organization. It will be vital for everyone on the team to record what’s going on in merge requests and issues. The temptation to edit something directly in production or change something manually is going to be difficult to suppress, but the less “cowboy engineering” the better GitOps will work.

    If GitOps sounds like something your organization might benefit from, our best advice is to start small. Pick a tiny area to begin with, get a win and then build on that. At GitLab we call that iteration and it’s worked well for us.

     

    This article was co-authored by Priyanka Sharma, director of technical evangelism at GitLab.

    — Brad Downey