Author: Don Macvittie

  • Our Infrastructure is Still Expanding

    Our Infrastructure is Still Expanding

    You would think that sooner or later, marketers and early adopters would take a rest, but it seems that if one group slows down for a bit, another picks up. Our infrastructure is expanding in almost every possible way, and this creates more of a burden on every aspect of DevOps, specifically in IT.

    We have more security needs because we have a larger footprint, for example. We have long had APIs, but now we have massive API groups that are used and re-used, and while standard code scanning works with the source, the companies doing “API security” start with security of the interface, not the code. That is slowly changing, but API management is also impacted, and organizations that determined they didn’t need API management because adjacent markets were serving enough of their needs are slowly discovering that with proliferation comes the need to look at API management again.

    I stay up to date on most new technology because, as an industry analyst, what is coming can inform my discussions of what a given technology space looks like today. But even for me—someone who is not spending a huge chunk of my day coding or working on the toolchain or perfecting the new deployment platform—it can be hard to stay abreast of everything. While one of the foundational tenets of DevOps was no specialization and a shared workload, I think everyone now agrees with what a few of us were saying from day one: Some specialized skills are still needed, except in corner cases. But we certainly are sharing workloads more broadly and far more responsive, so DevOps is absolutely a massive success. Site reliability engineering (SRE) extends that success but focuses more on Ops. I’m really not a huge fan. I like that DevOps wanted to merge the processes and make the management of applications more holistic; I don’t think re-dividing it is the best idea out there. Like every title, “SRE” is interpreted pretty broadly. Some roles really involve DevOps (but SRE sounds cooler), and other organizations are just replacing ‘developer’ with DevOps and ‘operations’ with SRE. However, most are using SRE to manage the entire environment in which the app runs.

    Where are we going? Constant change. That’s the answer. I’ve waited for the breather for a couple of decades, but since VMs were popularized, it has been non-stop and will continue to be.

    This means a lot to you, to your viability as an employee, and to hiring. With the glut of people on the market, we’re seeing a resurgence of “must know every tool in our toolchain” type position postings. Of course, you must not, in a world where the combination of languages, frameworks, CI/CD tools, alerting, and monitoring tools would make a great college-level combinatorics equation. You add value by being adaptable. Both to your current employer and to future ones. The key thing you should seek to demonstrate is your ability to pick up and master new technologies when required. Do it on your current job to show worth, and keep a record of it for your next job. I’ve been a hiring manager off and on forever, and when the market is flooded with good candidates, that is the type of differentiator that gets the team’s attention. Do it; make it something worth doing and document that you did it. You won’t regret it.

    A group of us were helping a recent grad figure out what to do with their career a couple weeks ago, and the consensus was “avoid AI until it shakes out.” This is not the first time I’ve heard that advice—it’s not even the first group I was a part of giving that advice. And I agree in the general sense, but certainly in places where it is being implemented by all vendors in the space and it adds value, you do need to be aware of how it works and what you can do with it, and a project is one way to get that knowledge.

    It is still a rolling opportunity, and you are still keeping the lights on. I’ve seen people who worry they’re not doing enough. Any employer worth working for wants you to keep the systems running first, and everything else after—so if you’re doing that, you’re doing enough. Now go beyond. And keep rocking it. This new technology hasn’t implemented itself, and it isn’t self-maintaining (yet) … That’s all you.

  • The IT-DevOps Life Cycle is Like a Pyramid That Keeps Growing

    The IT-DevOps Life Cycle is Like a Pyramid That Keeps Growing

    Don’t let anyone tell you how much easier IT life is these days. It is, in many ways, more fun and productive … but it is not easier. In fact, we’re doing more with less, cubed, at this point. My point is—think of the life cycle as a pyramid. The base is the build++ chain, from VCS to public-facing active protection; the height is the new technologies and systems being bolted on daily—like SaaS tools someone in accounting just signed up for or the new architectures a dev team in a subsidiary just decided to adopt (but which will eventually have to be maintained by IT). And the depth is the completely new capabilities that are required by market forces or regulation – SBOM is the poster child for this group; suddenly, just about every vendor is giving it to you as part of their product.

    No matter which of these axes is growing, the workload is generally growing commensurately. Think about it: If a new product is implemented, it impacts IT’s workload. If a new technology that solves a problem is implemented, it, too, impacts workload. And if a new capability in an existing product is dictated by the compliance team for any reason, this, too, increases workload. This is just the business side, and doesn’t include the fact that most orgs are dropping newly developed applications on a pretty regular basis these days.

    Automation has helped, but it hasn’t solved the problem yet. I believe it can, given time and focus, but for the most part, the demand for new technology to solve age-old problems (“We can’t effectively test” being met with “Here is automated test generation, execution and results filtering”) has been a net increase in workload. Though far smaller than it might have been, it is still increasing. This is true across IT. The test example above applies to app dev and security, with result sets that did not exist six years ago because the process was too expensive in staff hours. It also applies to ops because the processing power to get those tests done – and it is a lot with large test environments – is ops’ responsibility.

    Painting a bleak picture? Nope. Making certain we’re realistic. Trying to ensure that we consider how much we are already doing, and take a chance to implement actual time savers. For example, applying that same test automation to testing (for most orgs, I’m thinking security testing) that was actually being done already will reduce the cost of getting it done. Using SBoM for SCA to contribute to existing compliance saves a lot of time generating that information. Using new coding assistants shortens coding time and (arguably at this point, but inevitably, given the fullness of time) improves code quality, reducing rework, which is costly, even in an agile environment.

    We are growing our capabilities almost daily. I’m just suggesting that, in the process, we backward-apply the good stuff to reduce existing overhead. And as always, this big, complex beast exists and keeps clipping along because you’re there, keeping it fed and stable. Keep rocking it, and save yourself some time by using new technology smartly—there is plenty more to do than whatever you streamline.

    Image source: Photo by Michael Dziedzic on Unsplash

  • Is it Time to Reconsider CI?

    Is it Time to Reconsider CI?

    DevOps is old, in technology terms, at this point. Many of you have never developed under a waterfall and have no idea what tools like Visual Test were, and that’s actually very cool. While knowing other processes and tools does help make informed decisions, being immersed in the tools that matter today is more important. We have entire DevOps stacks built up that allow for the development, building, testing and, increasingly, deployment of apps in an agile environment.

    As always happens, market leaders and those who compete with them, complement them or serve a subset of the market have grown up and become standards. Indeed, application release orchestration (ARO) was a phrase coined in the early days of DevOps that fell away completely; I heard it recently and thought, “Oh, are we bringing that name back?” Vendors have come and gone; some merged into other products, and some are just plain gone. Again, that is a good thing as the market, its needs and the customer base are all mature. While it is terribly painful to be the user of a product that just goes away one day, the prevalence of open source in the CI/CD landscape does buffer that pain a bit. And the market gets better as tools fall away and new ones take their place.

    But sometimes, the movement is slow, just simply due to inertia. Most of us – and by us, I mean IT shops, and by most I mean a lot – use Jenkins. The number seems to be on a steady downward trend, and the tool’s maintainers have definitely taken steps to improve the parts that make organizations consider other solutions, but the trend is there.

    And I, personally, think that is a good thing. Stagnation is what made waterfall such a weakness in enterprise IT. Everyone did it, so that makes it good. Same is true for the old phrases we’ve all heard at least one variant of – “No one ever got fired for buying IBM”, or “No one ever got fired for buying Oracle”, “…Cisco”, etc. The obvious answer to those statements then, as now, was, “Well, maybe they should have.”

    The same is true for Jenkins. It is a great system that can do everything. And that’s part of the problem. If you have small projects or are a team using it standalone with light requirements that all revolve around getting a quality product out the door quickly, Jenkins just might be too much for you. In the early days, Jenkins wasn’t, but the plug-in architecture and years of competition and changing market needs have made it huge. And while there is arguably better documentation than any other CI, and inarguably the largest user base to ask questions of, it’s still weighty to use in place of a SaaS that comes pre-configured and you can just hand it a repository with build files in it.

    There are a lot of enterprises that need what Jenkins has grown into–it did not expand in a vacuum. But there are many, particularly new, small, agile orgs or standalone teams that just don’t.

    There are a lot of options out there, and it is worth taking stock of what you are trying to do with your build/test process, and build a list of things required, then look seriously at what’s available. For some, lightweight is good.

    Of course, as much as one can be in our technological age, I am a minimalist. I like my D&D to be a rules-light variant, my dev tools to have enough functionality but not bury me in extraneous stuff, laugh at Excel for including abstract functions and then selling to consumers, etc.

    At my company that does development, we do indeed use Jenkins. But our projects range from simple knock-it-out one-offs to complex systems with multiple servers and clients, so we feel it is best for that range of variation. And like many others, the investment to move old projects would be questionable without a driving need. Not that I get to do much development these days myself, but I do stay in it just enough to be dangerous. But we have asked the question and determined the answer. For now.

    If you have issues with Jenkins, look around. There are options. They are not for everyone, but your org isn’t everyone. Find the best tool, not the most convenient, and keep kicking rear.

  • Data Gravity Will Remain an Issue

    Data Gravity Will Remain an Issue

    Things are improving for IT at a rate we haven’t seen since the first wave of container adoption. We have massive automation in the entire SDLC—with AI-based plug-ins for IDEs to help developers write code that is both higher quality and more secure, security deeply integrated into the DevOps toolchain and the unification of testing/security tools across the SDLC. We are once again turning out more code of higher quality faster than we were just a year ago.

    And what is finally starting to catch up is data. I love the current iteration of data virtualization technology, where it can be set to move from region to region or cloud to cloud at leisure in the background. At the same time, data that is accessed in the interim can be instantly transferred. I used to cover storage and data, and frankly, this is the dream world.

    But it doesn’t change the fact that data location matters. While bulk moving data to its final destination will be useful, separating data from applications that access it is simply adding another weak link in the application infrastructure chain.

    This is where the true strength of containers and Kubernetes come in. With Kubernetes, we can put containers with the entirety of our apps into the environment where our data resides. And that means that our application infrastructure is consolidated and that, in a pinch, both application and data can be moved anywhere–because Kubernetes is truly portable (as long as you don’t use customization features of some vendors/platforms anyway).

    The environments we currently choose to use for enterprise applications–from mainframe to public cloud and everything in between–all are capable of running containers, and while there are still deployment differences, customizing a bit to move an entire application is far better than rewriting entire apps to target a new host environment. And the amount of retargeting required is constantly going down.

    So, decide where you would like to host your solution, use data virtualization to move the datasets there, spin up your apps in Kubernetes at that location and enjoy the fruits of your labors. Mergers, acquisitions, moves from one architecture to another, moving from public cloud vendor A to public cloud vendor B … All made far easier by data and Kubernetes portability.

    While people like to call it “cloud,” what we really should be calling it is “portable.” Whichever marketing person decided that cloud included Kubernetes is my sworn enemy, btw. Because the cloud is used without including Kubernetes, so of course, that just confuses the issue. Portable programming is a far better description of data mobility and Kubernetes architecture used in tandem. Doesn’t need a cloud at all.

    And we’re headed there for even more productivity. Just pick where to move it and carry on. We do indeed live in pretty cool times, and you all are the ones making it real. Keep up the good work.

    Image source: https://vecteezy_businessman-holding-social-planet-showing-networking-skills_1268342.jpg

     

     

  • We Will Control the World!

    We Will Control the World!

    One thing about centralizing control is that there always has to be someone in charge. That can be a very good thing or a very bad thing. In IT, it doesn’t even get far enough to be a terrible thing.

    Generative AI and increasing integrations have renewed the insistence from the crowd that it is certain they should control every little thing in your (virtual) data center. The idea that a given vendor should be the configuration and management point for a wide variety of tools – their own, competitors and nearby market tools – has been around forever. Vendors love the idea that they have that level of input into your infrastructure and, let’s be honest, many of us love the idea of having it taken care of for us.

    But there are several questions to ask here. First, what is the likelihood that vendor X will have a great level of support for the tools from other vendors in your infrastructure? I mean, before we even get into the idea that this desire for control is not at all altruistic (we’re not talking Stalin here, but we certainly aren’t talking Washington, either), there is the question of how AI will be able to learn about and configure/manage a competitor’s products. Watching wire traffic? Well, if the competitor’s systems are super-well-known and the format of configuration APIs/files are well known—maybe? But this leads us to the real problem.

    My advice to vendors would be to get their own products under control first. We work in a high-complexity field where the tools require a lot of specific domain knowledge to manage, and even more to manage well. Vendors have generally taken for granted that this level of knowledge would be available. Before making that level of knowledge unnecessary for competitors or spiderwebbing into competitor installs, make your tool(s) damn simple to use. That doesn’t mean you have to simplify it—you have the power of AI and are going to focus on configuration and management. So nearly eliminate configuration issues with your own products.

    Example: “Great Security Tool, I just added an API at https://devopsy.kinsta.cloud/rocks. Please lock it down so it is accessible only to users in the WeRule group, then generate a WAF rule to keep it from being used to access more than three records a day, and only allow GET and POST calls.” Most vendors across the industry aren’t even close to this level of simplicity, so why in the world would you trust them to configure/control other vendor’s products?

    There is a trend to make products cover more of the space they are in. This is very big in security right now, but you also see it in DevOps. CI/CD/CDD/VC/kitchen sink (KS)—I’m okay with that, as it does offer us simplified configuration and better cross-app communication. I think this is a better route than trying to serve as command-and-control for an entire tech stack that the vendor does not own. I still want point products and best-of-breed available, but a lot of us just don’t have the time and resources to implement five different products to get the job done—even if they’re “free,” as we all know, they are not free—so one vendor doing an entire chain is great for that use case, and often the shared information about the app, security, the network, etc. can work together better than the sum of its parts. So let’s stay in the “umbrella app” lane and not get carried away with trying to drive in the “manager of the entire world” lane.

    We’re busy, we’re tired and many of us are jaded. So don’t tell us how you can manage other vendors’ tools, tell us how you’re making it easier to manage yours. Thanks, from all of us in IT.