devopsy.kinsta.cloud

Author: Don Macvittie

  • Functional Programming: Island of Misfit Functional Programming

    Functional Programming: Island of Misfit Functional Programming

    The age of functional programming is dawning. The hype cycle has heated up, and IT departments are getting both real and perceived benefits out of the idea that a function runs on demand only.

    For some, the benefits are very real. Bursty processing, or things that are frequent and quickly resolved, see a benefit from being implemented on functional architectures, and the cost of functional architectures is arguably far better than the cost of cloud instances. I say “arguably” because you can always cram more onto an underused cloud instance, but you pay as you go—even more literally for functional programming, so additions each have an incremental cost. But it is incremental, and you have the power to choose whether functional is right for any given bit of the architecture.

    So why the title? Remember The Island of Misfit Toys from the much-beloved Christmas story, “Rudolph the Red Nosed Reindeer”? They were all just a little off. In a world flooded with perfectly good toys, they were left unattended and unwanted.

    That is a risk functional runs in the long run, also. The idea that you can pop a function out there, configure the access point (almost ubiquitously a web application firewall entry point, but things change so we’ll stick with the generic “entry point”), and start sending requests is great. The fact that you only pay for that bit of time when your request is actually being serviced is great, too.

    But this ideas assumes security is baked in at the functional level. If you are deploying this function anywhere with a public IP, it assumes hardcore security is baked in.

    And failure to have that security baked in is the first reason you might have a misfit toy.

    By definition, functional programming-based systems are complex systems. They are the ultimate evolution of microservices. They will suffer all of the headaches and heartaches associated with SOA and add on the issues with REST versioning, etc. There will be 50 or 500 functions, each clipping along and doing its thing, composed to make a single application (in a purist world, composed in a functional programming app).

    Some of these functions will become outdated and unused. You can’t leave them laying around, waiting to be abused. Indeed, a new form of financial attack might be to look for just such abandoned toys and start calling them repeatedly and rapidly.

    Imagine coming in one day and having Finance ask you why your bill at functional provider X tripled in the last month, only to find a function you left hanging around was being called for no good reason?

    But the complexity inherent in lots of small bits of code, each doing a specific task, pretty much guarantees there will be cruft left lying around unless you are very careful.

    And that, my friends, is (finally, I know) the point of this blog. Our monitoring tools today lean toward watching the performance of the application, but you need more than that before you go all-in on functional programming. You need build tools that can tell you when a given function is no longer called—from anywhere. That is a tall order which none of the previous distributed methodologies has managed to master, but it is more important in a world where you have functions sitting in various public clouds with public-facing IP addresses.

    There also needs to be a definitive source code scanner. Those are slowly getting better, but imagine if a library the build system pulls in and uses for 20 or 30 different functional programming microbits becomes compromised? We’ve seen enough instances of ignoring the problem is begging to be a victim, so scanning on build is more important with more systems using external libraries. Or more accurately, with external libraries being pulled into more microservices. There are companies/service providers that will validate your external dependencies and only let you load open source libraries they have certified as safe. I’ve never used one, but they’re worth looking into.

    A list of entry points and places they are accessed from is useful for internal-only applications. You can’t do that with a publicly exposed REST API that anyone can call, of course. I’m a developer and a D&D player, so I’ll use that example: Wizards/Hasbro cannot control where this API is being called from. The same is true for any web services APIs that exist to give others access. But internal APIs that are an integral part of internal systems and not called externally? Those you need an automated way to find references to them in internal source and keep a count. Much like garbage collection, you’ll want to take down ones whose count reaches zero.

    The thing is that in many ways functional programming reduces operational complexity. It certainly reduces operational overhead. But that complexity is replaced with a different kind of operational complexity; maintenance becomes a bigger issue, and the already occurring breakup of code into smaller chunks increases, bringing cross-reference complexity along with it.

    Keep that in mind. You will have orphaned functions, you will have security concerns, you will have changed operational complexity. Plan for these things from day one. They are not at all deal breakers, and the benefits of functional are already clear in certain verticals with more general benefits being discovered as I type. But they’re the new problems that will require answers in this new architecture.

    So, build in oversight and tools from the beginning and go see how functional programming could benefit your project. At this time, it isn’t the answer to every problem, but—like most cool new IT things—it is another tool to have at hand when solving problems for the business.

    And don’t let your functional programming microservices end up on the Island of Misfit Functional Programming.

    — Don Macvittie

  • Repeat Warning: There is No Silver Bullet in DevOps

    Repeat Warning: There is No Silver Bullet in DevOps

    As the cycle of DevOps is repeated in a growing list of organizations, I feel compelled to reiterate the warning that this is not the pinnacle. Anyone who has been around for a while knows that the current fad is not the last fad.

    [[Cue DevOps people screaming at my choice of the word “fad”]]

    Of course it’s a fad. DevOps (and to a much lesser extent, agile) is the cool toy in the midst of Gartners’ (in)famous hype cycle. That doesn’t mean nothing will come of it; it means that people freak out when you tell them it doesn’t solve every problem.

    All of the process/solution fads that have come along have offered some benefit to IT. DevOps promises to offer more, and in many environments it does. But grabbing your broadest brush and screaming, “DEEEEVVVVVOOOOOPPPPPSSSS!” while painting everything in sight is not problem-solving; it is thinking there is only one path to quality and it flows through the Chosen Process.

    The reason DevOps is different from previous hype cycles is obvious: It has the built-in assumption that this is not the end state. It assumes you will keep moving forward and things will change. Previous “new way of doing things” ideas have assumed that there was some nirvana you would get to and everyone would relax there, sipping ambrosia and enjoying immortality. DevOps makes no such claim. About as close to a final state as it gets is increased communications between the various parts of the DevOps life cycle. And that is because there is only so far you can go with increased communications before you’re flooded. And even that leaves room for new tools such as Slack to augment communications.

    But don’t look at every problem with DevOps blinders. This is the real world, and while some will (possibly correctly) claim that every software dev/deploy cycle can benefit from DevOps, the real question is not “Can it benefit?” but rather, “Is the cost/investment/risk less than those benefits?” And to the chagrin of those who live and breathe DevOps/agile, I’m afraid the answer to that question isn’t always a resounding “Yes.”

    I’m undeniably a fan of both agile and DevOps. What I’m not a fan of is finding the nearest round hole and cramming everything—including square pegs—into it. Look at each project/process objectively. As a simple and blog-length example, let’s examine standalone app that does some important back-end process and is only updated once or twice a year: Why ever would you rip the process for that apart, invest man-hours to bring it into the DevOps and agile folds, then … only update it once or twice a year?

    If you are building a new project, and you have DevOps processes in the building, it would take a convincing argument to not use them. The same should be true the opposite direction. If there is an existing project that doesn’t have delivery delays, it should take a convincing argument to move it to a whole new development/deployment methodology. Choose wisely. Invest resources where they’re required, not where ever the broad brush falls. IT has enough work to do, don’t add to it, just so you can say “We’re a DevOps shop.” Better to say “We’re a right tool for the job shop.”

    And keep kicking it. Business doesn’t run itself, IT runs business. Keep making it work.

    — Don Macvittie

  • Watch Your Language: Behavior Driven Development

    Watch Your Language: Behavior Driven Development

    There are two basic design methodologies in the software development world: top down and bottom up. In bottom up, the goal is to piece functionality together to “build up” to the desired result. In top down, the goal is to break the problem into smaller and smaller parts until you have implementable pieces.

    Both have strengths, and by correlation, both have weaknesses. Analysis and design phases in pre-agile days were almost ubiquitously top down. They would take business requirements, translate them into functional requirements, then break those down and assign the constituent pieces to developers. As the code was implemented, weaknesses in the design would be exposed, and feedback would go back up to the level of functional requirements—or, in really bad cases, to business requirements—and a mini-iteration would take place to resolve the problems.

    When agile came along, this was all too slow a process. To be sure, it was (and is) still used and has produced some of the world’s best software, but an agile team can’t sit idle waiting for a long process of analysis. Kan-Ban and similar tools/processes were introduced to address this issue, making the pieces being designed smaller and thus speeding the process somewhat. But smaller pieces implies bottom up to some extent, and the waters got a bit muddied.

    Test driven development (TDD) was designed to make sure the code written met the technical requirement by writing a test for the expected result before the actual code. This was one of the most fruitful things to go into/come out of agile development. While it has flaws—the most significant being the human factor—it does make more thought go into code being written and more rigor come out of code produced. The act of writing the test makes the developer think of ways to solve the problem that satisfy the test. But the source of those requirements still needed to come from some form of process.

    Enter behavior driven development (BDD). It’s not really development, though the name is catchy. It would be better named behavioral design for TDD, but that’s a mouthful.

    In short, the idea is that business owners and analysts speak in terms of the needs of a given role, while developers speak in terms of functionality or the results of a given test set. This creates a gap that causes misunderstandings. The goal of BDD is to get everyone speaking the same language. It tries to extend the spirit of TDD up the design/analysis stack—”shift it left,” in today’s parlance—so that the tests being developed are a hierarchy stretching from acceptance testing down to unit testing, and everyone is speaking the same language.

    En vogue today for the language is Gherkin, a simple requirement specification language that writes requirements and test criteria with (relatively) strictly defined sentences. I say “relatively” because, the language is a bit loose from a unit test perspective. But from role-based specification, it is locking down options to make the environment and test cases clear.

    I actually got to thinking about all of this while considering SmartBear’s recent acquisition of HipTest. SmartBear has made an investment in making its tools BDD-compatible, and HipTest is largely a BDD platform. The combination of the two should see some movement in the BDD space that will be interesting. I spoke briefly with SmartBear about the acquisition, and it seems the company really is merging the strengths of the two organizations, so it is worth watching over the next few months.

    Thus far, the benefits of BDD that jump out are the ability for everyone to talk about an element of the product in common terminology, while the developer can think “down” to the code required to meet the requirement and the business analyst (or owner) can think “up” to determine how to market this cool new feature.

    Tools are out there to generate TDD test classes/cases for a variety of languages—notably from my stable, Python and Java. This might be a huge step in the right direction also. Just the other day I was having a bit of a lighthearted conversation with a peer and said, “Get to the point where you can auto-generate tests at build time for us, so IT managers aren’t stuck deciding which lines of code are/aren’t worth maintaining test code for, and then we’ll talk.” Tools and frameworks such as Cucumber and JBehave actually take steps in that direction, though their approaches are different.

    We are not currently using BDD on our in-lab test application (Android/web clients, Spring API server) at work, but will have to consider it moving forward as another bit of DevOps tooling and practice we can experiment with. The tools are there, and the process bridges that sticky part of analysis and design where top down meets functional.

    After all, the end goal is giving users what they need. If BDD helps make that process smooth and faster, it is a fit for any DevOps environment.

    — Don Macvittie

  • Catan vs. DevOps Implementation

    Catan vs. DevOps Implementation

    My family has been playing a fair amount of the board game Settlers of Catan lately and, as happens with me, I got to considering the balancing act of the game versus the balancing act of moving to full-on DevOps. Most orgs have some amount of DevOps running at this point, but most have not made the transition, and I think the similarities have something to do with why. So, let’s consider …

    In Catan, you put a settlement at the intersection of three hexagonal tiles. Each tile has a number on it, and those numbers run the quasi-bell curve created by rolling 2d6. There never seems to be a great set of numbers, so you go for the best you can arrange (there are more considerations, such as the type of resources you get for a given hex, but we’ll get to that later). Then you hope that your numbers come up enough to keep you in the game. The game is pretty hot/cold, with the dice seeming to pick a number (regardless of bell curve) and roll it more often in a given game.

    The resources part is important. In Catan, each hex represents one of five resources, each of which you need to win the game. But you can trade resources (at a loss) for other resources, so you don’t have to sit on a hex for each type. This leads to some interesting attempts to get around the fact that the dice rolled against you or there were no good resource/number combinations when it was your turn to set up.

    The parallels are intriguing, though not perfect. You have resources (people/systems/software) and you have goals (project/product/portfolio) that you are trying to reach. Random things (system failures/hackers/budgeting changes) interrupt you more than is convenient (this last one is equivalent to die rolls in the game, but also another game concept called the robber that I only mention for those who play, since explaining the game would make for a long blog). And you have to be a success.

    Choosing to move people to DevOps creates a short-term problem with an eye to long-term gains, since those people moving are both out of their comfort zone and using tools that are new to them. Choosing to dedicate systems to DevOps processes/deployments removes them from the pool of resources available to longstanding systems/processes. It creates a tension of temporarily stretching things so that in the long run you will be better off (much like saving to build a city in Catan when the dice really want you to build roads).

    The difference is, of course, obvious. Catan is a game, and if you lose horribly you just reset the board and demand a rematch. You can laugh at the pernicious nature of the dice when they turn on you just as you were about to do something cool. But in DevOps transformation, you just can’t do that. People’s jobs are on the line, and even if you are at a place where job security isn’t an issue, people’s stress levels still will change based on such wild events. If you haven’t experienced this, have your company announce budget cuts that keep you from implementing part of a transformation stage or have your systems compromised that require the attention of staff that were in charge of moving an application portfolio. It isn’t pretty.

    The key is to keep the goal in everyone’s mind. The end result of a DevOps transformation should be the ability to get changes done faster with better focus on business needs, more complete testing and better communications. Keep talking this up, as there are always roadblocks to large change in any organization. Those improved communications should help people work together to overcome issues and move on. Even if something such as a data breach brings the effort to a halt for a while, when it resumes, an increased focus on the security of data will only make the transformation stronger.

    So figure out how to deploy those resources to the best advantage, keep rolling the dice (so to speak) and keep trying to improve on what the last iteration did. The odyssey will continue unless the org gives up on it; your task is to make the most of each step while you continue to keep those core business systems running.

    — Don Macvittie

  • DevOps: Role Clarity is Important

    DevOps: Role Clarity is Important

    Those who talk a lot about DevOps, particularly those who talk a lot about culture change, sometimes gloss over or “think it is obvious” that some roles change more than others when moving from Dev and Ops to DevOps.

    I was having a lighthearted chat with a friend who works for Compuware and realized that, even though several of us have mentioned it before, it bears repeating for those who have not yet dipped their toe into the DevOps pool too deeply.

    The driver for DevOps is increased agility, and the belief is that Dev and Ops will merge. What is often left unclear is that they may well merge into one team, but every team member will not become an expert at every role. Pundits seem to think this self-evident. I’m not so certain it is, if you read a lot about DevOps.

    By way of example, your Cisco ACI specialist (or ios specialist) will not suddenly become an expert in the intricacies of your development toolset and environment so they can contribute code. Granted, that is an extreme example, but there are two reasons why someone is a specialist in what they do: 1. They know/prefer it, and 2. The company had a need for their specialty. Don’t lose sight of those facts while moving to DevOps.

    A better example is DBAs. They’re not going to become network engineers just because they were put on a DevOps team. They’re still going to be DBAs … because you need DBAs. You need network engineers also, but if you need them, you already have them.

    Remember that increased communications and increased latitude are the social aspects of DevOps. The DBA is still a DBA, but faster access to queries and those people writing the queries means more optimization of some of the busiest sections of code. If your EMC specialist has access to the entire team, conversations along the lines of, “We’re running low on disk on LUN X, and it will cost us a billion dollars to upgrade, what are our options?” can happen quickly and include views from across the DevOps team (normally across the application or application portfolio).

    One of the most thankless jobs in IT is being the single/small shop. Anyone who has been IT for an entire org—particularly one with in-house apps—knows the struggle. My time as a developer hired to work on list serv software but who also inherited the entirety of systems administration (the sys admin quit and changed all the root passwords the day before I started) was not optimal. I learned a ton, which was cool, and I never mastered most of it, which was not.

    The goal of DevOps is not to re-create that environment in a team. It is to optimize team interaction, and make things go more smoothly. Make sure you are not on an implementation plan that leads to burnout or lack of accountability. Keep your eye on the real prize: increased responsiveness to business needs.

    And as always, keep kicking it. I have yet to walk into an environment that wasn’t unique, or one that didn’t have cool stuff to learn about. And you keep that running.

    — Don Macvittie

  • DevOps and Low-Code/No-Code Tools

    DevOps and Low-Code/No-Code Tools

    Helping out a friend, I had reason to revisit low-code/no-code solutions and poke around a bit. The industry has been relatively quiet on the topic, but changes in systems and marketing made me feel the need to offer an update on the relation of current solutions to DevOps.

    Low-code/no-code solutions have been a panacea in the eyes of some both in and outside the industry for decades. 4GLs were the earliest widespread attempt to make an environment that eliminated the need for developers, and while some years are quieter than others, the movement has not slackened.

    There are even some things we should consider adjuncts to low-code/no-code environments that normally are not considered. Traditionally the drive for low-code/no-code solutions is to move faster than IT normally turns out solutions. This would force us to consider the spreadsheet in accounting with 50,000 lines of special functions as a companion solution—faster than waiting for IT and more agile.

    The thing is that low-code/no-code environments have increasingly realized that IT is going to be their best customer, adjusting their offerings to focus more in that direction. At the same time, agile development and DevOps have made low-code/no-code solutions more palatable to an IT shop that is looking to continue the gains brought about by these technologies.

    The future appears to trend toward low-code/no-code solutions being an adjunct of agile and DevOps. I don’t say this lightly, but the signs do seem to point that way. Agile shops, having become more responsive to business needs, will hit a limit on what they can deliver, while the appetite for faster and more responsive systems delivery will not abate. Low-code environments that offer the following will become a very real option to extending the gains of agile and DevOps:

    • A clear and understandable way to code when needed in the language of choice for the IT environment.
    • A high level of feedback on what is wrong, possibly including suggestions on how to fix it.
    • Transparency in what is actually happening behind the scenes with the “no-code” part of the system.

    The idea is simple. By handling tasks that we, as developers, do over and over and over, we are freed to spend more time doing the truly innovative and environment specific work. This would seem to offer the next step in the growth of agile.

    Those tools that offer options such as automated deployment will step into ARA land also, making the Ops side of DevOps happy—in shops that haven’t invested a fortune in setting up ARA already, anyway. If low-code can offer the ability to say, “Here is a problem,” then the developer can submit a fix, and it is submitted to test, and if tests pass is sent to production (and some of the tools do), then the only questions are, “Does the code generated by the low-code part perform? and, Is it easy to integrate existing libraries, etc.?” If the answers are yes, then we also have an add-on for DevOps that delivers value.

    We live in interesting times. It will be very cool to see this all play out, and interesting to see where it heads after this. At least one vendor is betting on artificial intelligence (AI) and machine learning (ML) to offer increased productivity.

    Of course, it isn’t a panacea. I read where a vendor said, “We automatically update your queries when a table is modified,” which spawns another dozen questions for me, starting with “Really? What do you do about code that was counting on that field much later in processing? Do you check that screens are displaying defaults, or that variables used to make decisions aren’t always the default value? Do you flag this stuff so devs can ‘fix’ what automated query changes ‘broke’?” I’ll be digging in some more, because now they’ve got my attention. So you’ll probably be hearing more from me about this topic.

    Meanwhile, let the future worry about the future, and keep kicking it today. Just stay aware of what else is going on out there that might be handy in your toolbox.

    — Don Macvittie

  • DevOps: It’s an Odyssey, not a Journey

    DevOps: It’s an Odyssey, not a Journey

    It has become somewhat of a standard practice to tell people, “DevOps is a journey, not a destination,” which attempts to convey the fact that there is no end point that is set in stone. But a journey is a well-planned, point-to-point trip, while DevOps transformation is a messy, figure-out-the-next-steps-as-you-go trip.

    So I propose we quit calling it a journey and start calling it an odyssey. This summer, when my mother takes her 10,000th trip back to Germany, it will be a journey. She’ll start in Cincinnati, Ohio, and travel through a couple of states before finally making her way to Hamburg, Germany. That’s a journey. She knows where she is going, and the discrete steps along the way.

    Her original flight from Germany? That was an odyssey. The trip went Berlin->Naples->Shanghai->Hawaii->California->New Jersey. Of all of that trip, only Berlin->Naples was a choice. Or, rather, the decision to go to Shanghai was a choice that required Berlin to Naples—very much in line with your DevOps choices making decisions for you along the way. And just like her family, you have no idea what your choices will be in a few iterations. They knew they had to go to Shanghai because that was the only port that would allow them in without visas, and they couldn’t get visas. After that? They knew they wanted to get to the United States or Australia (one assumes anywhere far from the insanity that was engulfing Europe and Asia during World War II), but how to get there was a total unknown. “First we start, later we finish,” was her father’s description of the goal.

    And that’s pretty much what DevOps is: First you start, and in the fullness of time you will discover things you hadn’t even considered when starting that just make sense and improve your overall position. There will be fits and starts, wrong turns and massive improvements. It is an odyssey, and one that is designed—just like my mothers’ family’s flight—to make things better in the long run.

    For example, the choice of Ansible or Puppet will make other choices easier. Giving teams more autonomy and increasing communications will make other choices more obvious. It will evolve into more than the first iteration’s goals would suggest, and that’s a good thing if increased agility and reduced manual intervention are the results.

    So keep on that odyssey, but don’t expect a simple end. More likely, the identification and acquisition cost of future improvements will cause more than the benefit they introduce, and you’ll find yourself in a better place, ready to breathe a bit and let the market find new ways to improve all things DevOps.

    Which is cool, because that should be your final goal: “We have improved so much that any improvements we could make right now would offer minimal incremental improvement for a significant cost.” At that point, your journey isn’t done, but you’re able to sit and take a rest, consolidate gains and assess future possibilities.

    And until you hit that point, be excited that every iteration of DevOps is improving responsiveness and taking you closer to your goal.

    Shameless plug: Read about my mothers’ odyssey here.

    — Don Macvittie

  • DevOps: What a Wonderful World it Would Be

    DevOps: What a Wonderful World it Would Be

    The promises of DevOps are clear. They offer faster iterations, allowing ops to keep up with agile development and make an IT organization more responsive to business needs.

    DevOps doesn’t make less code. It doesn’t reduce configuration files. It doesn’t eliminate deployment issues (though it might reduce them). It doesn’t eliminate testing; indeed, it creates more if Agile and DevOps are taken to their logical conclusion:  More testing, earlier. DevOps doesn’t reduce the need for security.

    In short, it doesn’t reduce workloads. It might automate some of them, freeing up man-hours, which is absolutely a positive result—one of the more positive results, in my opinion. But it also introduces the overhead of the DevOps toolchain, so some of that automated freed-up time is already spoken for.

    And automation of testing (for example) is something that’s been around for decades. The first UI-based automated testing system I recall was Microsoft Test. I managed a team that used it extensively to automate test runs of the software we delivered to financial services. It worked well, and there was much gnashing of teeth when Microsoft discontinued the product.

    The same is true of most of the time-saving benefits of DevOps: We’ve tried them before, but DevOps approaches them from a different perspective—and, in many cases, a more sustainable perspective.

    I’m a DevOps aficionado, make no mistake. The reason I’m stressing what it won’t do here is because misperceptions drag it down. Don’t go into it looking to reduce costs, because at least in the short run you won’t. Don’t go into it looking to save work, because at least in the short term those savings will be minimal. Going into it for the wrong reasons means the things you are measuring will ultimately look like failures, even if all the good DevOps offers is reaped.

    Instead, go into it looking to be more responsive. Go into it looking to beat your competition to market. Go into it looking to increase communications across the team. Delve into DevOps for the right reasons, and you’ll be measuring the right outputs.

    And remember, it is an iterative process. Don’t overstate expected benefits on the first pass. Low-hanging fruit makes it look as if DevOps will do a lot of things, but low-hanging fruit often has tight ties into firmly rooted trees, making it harder to grab than originally thought. Use a measured approach and look for consistent improvements. In some organizations those improvements might be small at first. But use your knowledge of the organization and application portfolio to create a measured success plan.

    Reaping the benefits is worth the change DevOps brings. The ability to tell a line-of-business leader that the new functionality they are considering could be mocked up and ready for review in a week or two is a huge benefit to the organization. And it doesn’t hurt at all that such responsiveness will raise the rest of the organization’s opinion of IT.

    So it is a wonderful world, if your goals are the right ones.

    — Don Macvittie

  • Defects: Pay More Attention to What, Less to Who

    Defects: Pay More Attention to What, Less to Who

    Software defects have been with us forever, and while DevOps and agile can reduce the number of defects generated, it is unlikely this number will ever get to zero for a given organization—and even if it does, it won’t likely stay there. Simply put, we’re developing increasingly complex systems on increasingly short timelines. The reduction possibilities that DevOps offers cannot completely combat this combination.

    But the quest to keep defects down is necessary. The goal should always be “Less, not more.”

    One of the things we have historically done is associated defects with developers. It is natural to want to know “who broke the entire payroll system,” but quite often “who” is the wrong question.

    A huge benefit of both agile and DevOps is that it allows us to look closely at where the process fails and determine why. While every developer in a shop knows “you should avoid modifying that module,”—and those are the easy ones to target for early bug reduction—there are other modules and source files that might be a hidden source of a certain percentage of defects. Using the defect tracking toolset to keep tabs on where bugs are occurring might show that the problem is a given developer, but more likely it will show that the problem is a given set of source.

    So if you are not already, it is time to start tracking defects closely and looking for patterns. There are some great documents out there about relatively common break points that might be useful. However, reviewing your own data is more useful, as it isn’t about common—it’s about actual problems in your source pool.

    This is all part of the process for agile and DevOps, I’m just saying it a different way than the usual “blameless” discussions because I think they fail to connect for some shops. Of course blameless is great, the point here is focus on problems—whether that is blameless or not depends upon which problem you find you have.

    Meanwhile, keep cranking it out. If you’re working on a viable product, then defects are an annoyance, but not something to gnash your teeth over. You are developing a viable product, after all.

    — Don Macvittie

  • Standardization, De Facto Style

    Standardization, De Facto Style

    The first decade or so of DevOps had one truth to it: It was about change. Not just change in the way that IT worked, but change in the way DevOps worked—particularly in the tools space, where products and product preferences both are in flux. We’re still seeing a massive change, but increasingly the effect is more like the ripples created when a rock is thrown into a pond than a tsunami, dumping change upon change.

    You could look at this one of two ways (and no doubt those who have more reason to dig into details than I have already done so): Either the slow but steady standardization typified by tools such as Jenkins and Puppet have enabled participation of enterprises in the DevOps life cycle, or the demands of enterprises for some amount of stability in their massively complex systems have driven standardization.

    Either way, the result is the same. We are seeing a lot of change further out, but the core technologies for build/test/deploy are settling into a few products by a few vendors. That is good for adoption, even if you would argue that it is bad for innovation. It means that skill sets are, at least to a certain extent, portable across DevOps teams and organizations, which for a long while was not true. When I was knee-deep in deployment technology, knowing how to use product A did not mean you knew how to use product B. The more people using one or the other, the better for employee mobility and the pool of available resources, as both people and support tools like standardized scripts.

    I recently was going through the offerings of a service provider in a space that doesn’t traditionally have DevOps enablement. The company’s offering included DevOps, so I drilled down. Seeing the company supported Jenkins, Jira and Xebia Labs made me feel better about the program it was putting together. It meant the company was adapting mainstream tools to a specialty market.

    That’s what spurred this blog. It’s been a long time that these things were seen as “new and shiny”; now, for a service provider to include them as proof it is serious about customers’ problems shows just how far we have come.

    There’s still a long way to go. The DevSecOps community is currently in the throes of growth, and lots of outer-edge DevOps tools are incompatible with each other and chosen per shop without a clear market leader. But we’re getting there. And at this point, a person going into DevOps who asks me what they should know will actually get an answer of “Learn X, Y, Z” rather than “You should learn X, but pick up some of X.1, X.2, and A too, then there’s Y …”

    The thing is, though de facto standardization is happening, to many (particularly enterprise) shops, this is all still new. So if you don’t know the recognizable tools yet, don’t worry too much; there will be opportunity to learn as they are rolled out across a given org.

    — Don Macvittie