devopsy.kinsta.cloud

Author: Greg Pollock

  • Time and complexity; or, Why are some things hard to do?

    Time and complexity; or, Why are some things hard to do?

    Why are some things hard to do? If we want to make complicated tasks easier, which is one of the goals of DevOps, we need to have some theory of why they are difficult in the first place.

    Richard Garfield, a mathematician and game designer, presents two factors that make tasks more difficult: time constraints and complexity. Our ability to solve a problem depends on the amount of time we can spend working on it and the complexity of the problem.

    Time and Difficulty

    DevOps discussions generally focus on time as the relevant dimension. Business value is measured in time: time to market, time to recover, and lead time for change are all important metrics for DevOps. Doing the same amount of work in a smaller amount of time is better. But an emphasis on time can also be misleading. Allotting less time for a task isn’t the same as truly being faster.

    So what makes something take more or less time? Part of the difficulty of a task is relative to the person doing it. Provisioning a new server might be straightforward for an administrator, challenging for an application developer, and impossible for a graphic designer. Hiring good people and matching them with the right tasks reduces the difficulty of the task and the time required to complete it.

    Complexity and Difficulty

    But regardless of who’s doing it, the task to be done is the same. The specifications have an objective existence outside of whatever person is doing the work. The objective part of a task’s difficulty is its complexity.

    The parallel between time and complexity is evident in the thought experiment Richard Garfield proposes. Imagine you are standing in front of two doors. One has a dragon behind it, the other has treasure. Each door is printed with a math problem that reveals whether the door contains a dragon or treasure. If you have a very short amount of time to decide, you will only be able to solve a fairly simple problem before being forced to guess which door is safe. Similarly, if you have a lot of time but must solve a very complex problem, you will again be reduced to guessing. Either insufficient time or excessive complexity can force a person to guess in a situation where a knowable answer exists.

    Automation and Complexity

    DevOps typically aims to reduce complexity through the “automation” leg of the CAMS definition. Now that we have examined the problem more broadly it’s clear that automation is a subset of the class of “things that reduce complexity.” Before automating, or if automation efforts at your organization have yielded disappointing results, a good check would be to ask “will this reduce the complexity of deploying code?”

    Automating without visibility into what/how you are automating will move complexity around but it won’t reduce it. You’ll be able to spin up new builds really quickly—once you’ve tracked down the person who pushed the latest version and forgot to mention the three little configuration settings you need to change for it to work. Configuration drift or pockets of automation reintroduce complexity. Without visibility and monitoring, these problems grow on automation systems like mold on untended fruit.

    This is not to say that automation is bad. When done in a mature environment with testing and visibility, automation is the greatest tool for reducing complexity. But without independent validation and monitoring, you’ll find that automation can also be one of the biggest sources of complexity.

  • Config Management & F***ing Shell Scripts

    Config Management & F***ing Shell Scripts

    Joke tech projects pop up all the time. Some, like the lamentable titstare, should never have seen the light of day. Others, like Mark Zuckerberg’s Facemash start out as a joke and grow into something that is anything but. Most come and go without anyone really noticing.

    Whilst their lives are typically short we often learn more from people’s reactions to these projects than we do from their use of them. An excellent example of this popped up recently when a couple of frustrated Chef and Ansible users came up with a simple configuration management alternative, the regrettably named F***ing Shell Scripts (FSS). As they explain it, as in “Why can’t we just use f***ing shell scripts?”.

    Whilst usability issues with configuration management tools – particularly in the Enterprise – is a topic close to my heart, I would not normally have had much more than a passing interest in FSS. What grabbed my attention was the reaction it elicited when posted to Hacker News. With over 150 comments FSS clearly touched a nerve.

    Now to clarify, the authors of the project were quite definitive in assuring the mob that they had created FSS as a joke:

    We wrote it literally in an afternoon, and were giggling like schoolgirls the whole time.

    People either missed this, or chose to ignore it:

    “This is simply brilliant! I’ve dreamt about such tools.”
    – artellectual

    “I’d love to see a tool like yours be brought to full maturity, then I think I’d love to use it.”
    – sdegutis

    It provided a perfect platform for many to voice their frustrations with current tools:

    “Even for an experienced sysadmin, using Puppet and Chef to do even a trivial project (replacing a ~10 line shell script) took a painful couple of days.”
    – mattjaynes

    “In my experience, there’s a huge amount of work involved in just getting Ansible (and other CM systems) to work right in the first place”
    – s_kilk

    “We’re using Chef right now, and evaluating other options. They seem to either be between way too over-engineered, or not flexible enough, but nowhere in between.”
    – sdegutis

    “Keep it simple. puppet/chef have horrendous DSL’s that make it really complicated to reason about.”
    – sp1982

    In the interest of balance I should point out that there were some voices of support amongst the crowd. Ansible in particular won praise for the effort its team has put into simplifying things.

    Overall though I was still blown away with the level of frustration being expressed. Automation tools have been around for years now; CFEngine, the grandfather of the modern tools was created in 1993. Despite this, a tongue in cheek project suggesting ditching them all in favor of the technology they set out to replace (or at least abstract away), shell scripts, is considered credible. For me this is akin to people ditching virtualization for physical boxes. It just doesn’t make sense.

    Unless of course something is still fundamentally broken in the approach the current set of tools is taking; that they are still too hard; that they still fail to address key prerequisites like visibility and accountability. I believe that this is the case because I’ve experienced it first hand; I’ve seen it a hundred times in businesses I’ve talked to, and threads like the Hacker News one mentioned above leave little room for alternative interpretations. Heck, we’ve based our product GuardRail on this entire premise; these are not empty words on my part.

    Incremental improvements such as those offered by Ansible are a good thing, but what people seem to be searching for is a step change. As things stand this is less likely to come from the current batch of tools, and far more likely to come in the form of a new approach. Docker anyone?