devopsy.kinsta.cloud

Tag: ci

  • Is it Time to Reconsider CI?

    Is it Time to Reconsider CI?

    DevOps is old, in technology terms, at this point. Many of you have never developed under a waterfall and have no idea what tools like Visual Test were, and that’s actually very cool. While knowing other processes and tools does help make informed decisions, being immersed in the tools that matter today is more important. We have entire DevOps stacks built up that allow for the development, building, testing and, increasingly, deployment of apps in an agile environment.

    As always happens, market leaders and those who compete with them, complement them or serve a subset of the market have grown up and become standards. Indeed, application release orchestration (ARO) was a phrase coined in the early days of DevOps that fell away completely; I heard it recently and thought, “Oh, are we bringing that name back?” Vendors have come and gone; some merged into other products, and some are just plain gone. Again, that is a good thing as the market, its needs and the customer base are all mature. While it is terribly painful to be the user of a product that just goes away one day, the prevalence of open source in the CI/CD landscape does buffer that pain a bit. And the market gets better as tools fall away and new ones take their place.

    But sometimes, the movement is slow, just simply due to inertia. Most of us – and by us, I mean IT shops, and by most I mean a lot – use Jenkins. The number seems to be on a steady downward trend, and the tool’s maintainers have definitely taken steps to improve the parts that make organizations consider other solutions, but the trend is there.

    And I, personally, think that is a good thing. Stagnation is what made waterfall such a weakness in enterprise IT. Everyone did it, so that makes it good. Same is true for the old phrases we’ve all heard at least one variant of – “No one ever got fired for buying IBM”, or “No one ever got fired for buying Oracle”, “…Cisco”, etc. The obvious answer to those statements then, as now, was, “Well, maybe they should have.”

    The same is true for Jenkins. It is a great system that can do everything. And that’s part of the problem. If you have small projects or are a team using it standalone with light requirements that all revolve around getting a quality product out the door quickly, Jenkins just might be too much for you. In the early days, Jenkins wasn’t, but the plug-in architecture and years of competition and changing market needs have made it huge. And while there is arguably better documentation than any other CI, and inarguably the largest user base to ask questions of, it’s still weighty to use in place of a SaaS that comes pre-configured and you can just hand it a repository with build files in it.

    There are a lot of enterprises that need what Jenkins has grown into–it did not expand in a vacuum. But there are many, particularly new, small, agile orgs or standalone teams that just don’t.

    There are a lot of options out there, and it is worth taking stock of what you are trying to do with your build/test process, and build a list of things required, then look seriously at what’s available. For some, lightweight is good.

    Of course, as much as one can be in our technological age, I am a minimalist. I like my D&D to be a rules-light variant, my dev tools to have enough functionality but not bury me in extraneous stuff, laugh at Excel for including abstract functions and then selling to consumers, etc.

    At my company that does development, we do indeed use Jenkins. But our projects range from simple knock-it-out one-offs to complex systems with multiple servers and clients, so we feel it is best for that range of variation. And like many others, the investment to move old projects would be questionable without a driving need. Not that I get to do much development these days myself, but I do stay in it just enough to be dangerous. But we have asked the question and determined the answer. For now.

    If you have issues with Jenkins, look around. There are options. They are not for everyone, but your org isn’t everyone. Find the best tool, not the most convenient, and keep kicking rear.

  • We Will Control the World!

    We Will Control the World!

    One thing about centralizing control is that there always has to be someone in charge. That can be a very good thing or a very bad thing. In IT, it doesn’t even get far enough to be a terrible thing.

    Generative AI and increasing integrations have renewed the insistence from the crowd that it is certain they should control every little thing in your (virtual) data center. The idea that a given vendor should be the configuration and management point for a wide variety of tools – their own, competitors and nearby market tools – has been around forever. Vendors love the idea that they have that level of input into your infrastructure and, let’s be honest, many of us love the idea of having it taken care of for us.

    But there are several questions to ask here. First, what is the likelihood that vendor X will have a great level of support for the tools from other vendors in your infrastructure? I mean, before we even get into the idea that this desire for control is not at all altruistic (we’re not talking Stalin here, but we certainly aren’t talking Washington, either), there is the question of how AI will be able to learn about and configure/manage a competitor’s products. Watching wire traffic? Well, if the competitor’s systems are super-well-known and the format of configuration APIs/files are well known—maybe? But this leads us to the real problem.

    My advice to vendors would be to get their own products under control first. We work in a high-complexity field where the tools require a lot of specific domain knowledge to manage, and even more to manage well. Vendors have generally taken for granted that this level of knowledge would be available. Before making that level of knowledge unnecessary for competitors or spiderwebbing into competitor installs, make your tool(s) damn simple to use. That doesn’t mean you have to simplify it—you have the power of AI and are going to focus on configuration and management. So nearly eliminate configuration issues with your own products.

    Example: “Great Security Tool, I just added an API at https://devopsy.kinsta.cloud/rocks. Please lock it down so it is accessible only to users in the WeRule group, then generate a WAF rule to keep it from being used to access more than three records a day, and only allow GET and POST calls.” Most vendors across the industry aren’t even close to this level of simplicity, so why in the world would you trust them to configure/control other vendor’s products?

    There is a trend to make products cover more of the space they are in. This is very big in security right now, but you also see it in DevOps. CI/CD/CDD/VC/kitchen sink (KS)—I’m okay with that, as it does offer us simplified configuration and better cross-app communication. I think this is a better route than trying to serve as command-and-control for an entire tech stack that the vendor does not own. I still want point products and best-of-breed available, but a lot of us just don’t have the time and resources to implement five different products to get the job done—even if they’re “free,” as we all know, they are not free—so one vendor doing an entire chain is great for that use case, and often the shared information about the app, security, the network, etc. can work together better than the sum of its parts. So let’s stay in the “umbrella app” lane and not get carried away with trying to drive in the “manager of the entire world” lane.

    We’re busy, we’re tired and many of us are jaded. So don’t tell us how you can manage other vendors’ tools, tell us how you’re making it easier to manage yours. Thanks, from all of us in IT.

  • Monitor Toolchain Health, Too

    Monitor Toolchain Health, Too

    A funny thing happened on the way to being agile. We adopted a ton of tools to make us more responsive. They really do help us deliver software quickly and efficiently, and they have improved IT processes along with our specific process improvements.

    They also created a toolchain that resembles areas of responsibility of old. That makes perfect sense; we still need to store source code, build applications, deploy somewhere, worry about security, etc. And we need to do all of that faster, which is where automation and the toolchain really shine.

    But you need to keep in mind the overall impact of the toolchain upon the development process. Now the organization is dependent upon those tools. We went from slow and steady to fast and with a large series of steps along the toolchain. Things are better, we’re delivering quality software at a faster pace, but in some ways, the process risks becoming more fragile. If a tool in the chain shows signs of trouble, you need to be aware of it and on top of the issue almost immediately. The price of ignoring issues in the process can be catastrophic disruptions in software delivery. Losing a CI tool is akin to shutting down customs at an international airport: The source of the problem is big, and the downstream problems are show-stopping.

    So while you are tooling your applications, make certain you are tooling the build chain, also. Know when a build that always takes ten minutes suddenly spikes to triple that. Know when a test suite delivers non-breaking errors at an increased rate. Know when space is low on a shared drive that the build tools use. Tool everything, because the software delivery toolchain is now the core of application development and deployment.

    Tooling these systems can also offer a warning of problems in a given application. It is possible that the increase in build time is because there are a lot of previously unused libraries being included … and that those libraries have not been properly vetted by security. It is possible that the disk space on that shared drive has PII test data that needs to be cleansed. There is a lot of good that can come from over-tooling the build chain, and unless you have a bit of tooling that slows the process, there is no downside. It won’t take too long to build a list of what to watch, and modern DevOps tools have enough reporting and APIs that it won’t take long to begin watching those items, either.

    That is where the people part comes in. Don’t ignore the tooling once it is in place. Build a process to regularly review results and take action when necessary. No tooling is useful if the results are ignored; sadly, we have a long history of spectacular failures that started with, “We saw indications, but didn’t think it was important at the time …” So act on results, even if that just means flagging them for review in the next reporting cycle—just don’t ignore them.

    And keep rocking it. The organization thrives on the software you build, deploy and support. It is a massively complex system that has IT staff at its heart, so keep it thriving and watch for indicators of illness in the core toolchain.