devopsy.kinsta.cloud

Tag: application configuration

You Have a Box: The Impact of Containers on DevOps

To continue the discussion from my last blog post, you have a box. Increasingly, you only have to ask, “What do we want to put in it?”

It is interesting to see the growth and change in IT over time. The advent of containers is the focus of this particular blog, but there are many other topics that would be fun to cover in the same manner.

The fact is that containers have become our blank puzzle pieces. We take it, make it what we need and drop it in. They relieve us (for the most part) of the need to worry about deployment issues and allow us to focus on getting the part of the app they represent right.

But those deployment issues didn’t go away, they’re simply obfuscated. Networking, security, storage and even capacity still have to be concerns of IT, just not concerns of a given app or its developers. That creates a separated world of issues, sometimes a split world of issues—such as networking inside/outside the container environment—but these are largely things we’ve had to deal with since the advent of virtualization, just in a larger scope.

The true promise of multi-cloud comes with containers, also. Each cloud vendor wants to lock users into their container architecture, of course, but nothing stops a shop from running containers and container management on cloud instances of their own, maintaining the highest level of portability for the cost of some extra management headaches. In the end, where it is running is even less important in the container world, and that allows what it does to gain more importance.

Management of containers has a bit more overhead; after all, hardware or cloud management needs are the same, and the configuration/management of containers is layered over those needs as an additive. But that’s a worthwhile cost to bear if the result is that tomorrow the container could be thrown to a cloud instance, and the next day for whatever reason—capex versus opex, data accessibility, whatever—could be brought to a local server. Maximum flexibility makes the cost of maintaining another layer of management.

And much of that layer of management can (and usually is, in my experience) be folded into DevOps practices, precisely because a container is 100 percent software. That means the overhead is not nearly as large as it seems. And what isn’t well-covered by DevOps is simply an extension of virtualization.

So with containers, you have a box. The management of the box is handled by DevOps. The only question that really matters is, “What are you going to put in the box?” And that’s pretty cool: Less to worry about during development means more time to make whatever goes in the box rock.

And the innovation around those boxes is just getting geared up, I think. We already see things such as shops/projects with a container life expectancy of less than a minute that are inconceivable in other environments. Not that they’re all good ideas, but the more people doing things in a different manner, the more opportunities for really great discoveries to come along.

And you will be there. Filling boxes with cool new apps. Pretty sweet.

— Don Macvittie

November 1, 2018
Choose Your Toolstack — and Manage It — Wisely

Every DevOps toolstack has a control point, be it a bus, an app, a management app or whatever. Something always is in control and driving the other pieces of the architecture to do their part. It is a basic tenet of automation that jobs are mostly kicked off by something in the software hierarchy.

In DevOps, most of us have defaulted to Jenkins for a large chunk of that automation. It was the center of the continuous integration/continuous delivery (CI/CD) side, and increasingly it is used to run both sides of DevOps (as I have called it previously, both DEVops and devOPS). CloudBees and recognized this trend, and an increasing amount of control and process functionality has been turning up in Jenkins.

On the other side, there are organizations running all of devOPS from deployment tools such as Ansible or Puppet. Again, that’s not a terrible solution, since they can configure the software on the server and make sure all is ready to run.

But neither of these toolsets was originally envisioned with the idea it would control as much of the DevOps stack as they currently do.

If you read me regularly, you know I’m not a huge fan of this type of default control structure. There is a lot of functionality that can be laid over the CI/CD and environment automation tool. But to do some of it, the tool needs to be above the day-to-day functionality these tools specialize in.

ARA can do it—and honestly, for a first brush at connecting the worlds represented by CI/CD and configuration management tools, is pretty impressive. Other vendors have come at it from a service bus perspective, allowing everyone to communicate through their tools and doing roll-up reports based upon what has been integrated. That’s not a terrible solution either, if it works with or can be made to work with your tools.

ARA as a market is short on the monitoring and management end of the process when the app is deployed and clipping along. It’s getting better, and hopefully ARA that is fully integrated with monitoring tools will be the next step in this evolution, but for now most companies are handling tooling, monitoring and response to that monitoring outside of the ARA system.

The queue systems tend to lag behind ARA in their ability to correlate and coordinate, but again this is getting better. They know only what is reported through them, so as they gain add-ons, expect they’ll handle the overall stack better.

The long-term goal is, after all, to run from commit to release with only designed “go/no-go” checkpoints involving humans, and then to monitor, with humans getting notifications only as needed. How we get there may not be so important—or it might.

In the market, the sand is still shifting too fast for vendors to keep up. In any given implementation, how fast you adopt all the new products is a function of choice. Use controlled chaos, changing things out for the better only when you have a quantifiable benefit and they fit into your overall architecture. Most importantly, no matter which toolset you choose to use to control the check-in-to-release cycle, new tools you introduce should integrate with it.

So what am I recommending? That you choose a control tool that fits your specific environment and needs, and that you make certain when choosing new tools you have a path to get them integrated with that control tool. Nothing more. Don’t put a tool in charge of half or all of your DevOps toolstack just because it was there. Have a real reason, then use that reason as a gauge when adopting new tools.

— Don Macvittie

August 6, 2018
Don’t Underestimate the Work

It is relatively common in human endeavor to attempt to figure out how much work will be involved in a project, and be wrong. The more complex the work being done and the organizational system doing it, the further off estimates can be. This was one of the drivers for both Agile and DevOps: to take smaller steps with less dependencies and complexity.

The complexity problem is true in the path to application release automation (ARA) as well. ARA merges continuous test/continuous delivery with deployment automation, but both of these toolsets are complex and continuing to evolve. When you think about it, it is somewhat amazing that ARA can handle the breadth of functionality it is covering, while both sides are shifting and growing as well as it does.

Construction Crane, compliments of Pixabay

When you approach ARA, you should make certain you have a handle on the amount of work that it will take to get the system up and running. It’s not simple, even if you’re already using a full CI/CD environment. But it’s worth the work, so here are some of the things to watch out for.

Multiple Targets

First on the hit list is on the deployment side. One of the strengths of ARA is the ability to deploy to any environment. The ARA system has to know about each environment in enough detail to enable deployment on it. It must know how to deploy servers, where to get OS, what needs to be added on to the OS, and how to get the application onto these systems. Increasingly, programmable network elements and security configuration fall into this category also. While tools such as Chef and Terraform can help in this regard, if you’re not already using them, they’ll have to be configured if not already, and then linked into your ARA tool. Understanding the path from here to there is critical to early success with ARA.

Databases

This is one of the more vexing problems. While most ARA tools offer the ability to configure databases and database connections, your datasets generally don’t hop around so easily. So if you’re testing and deploying on separate networks, you’ll need to make sure that the relevant tables are on those networks. This struggle impacts moves to the cloud, changes from a fully test network to production or any other change that requires the database server be accessible when it hasn’t been in the past. The work to solve these problems is pretty wide. In some cases, such as purely internal or purely cloud, merely routing from the relevant app instances to the database server will do it; but, in the case of across-the-internet changes, it will take a lot more work. Make certain you understand what challenges you will face before moving.

Tooling Sets

Much of the external needs of an application are baked into the app build process through tools such as NPM or Git. But those tools don’t cover everything necessary for the entire app ecosystem. Agents, per-tool databases, small tools used occasionally—make certain you have a list of what makes the app work, and make certain they’re configured to be built in by the ARA tool. This sounds obvious, but watching a team learn by trying to deploy and fix broken parts only to do it again is not only painful, it also leaves a “Did we get everything?” impression. I saw one app break on deploy because the team didn’t include curl. Yes, curl in the deployment. Test had it in the environment, production did not.

Get an inventory of both application development dependencies (libraries, etc.) and deployment dependencies (databases, reporting tools, agents for monitoring, etc.), and save yourself some pain.

Decrease Your Work by Asking Questions

ARA gives you a repeatable build/deploy process that can be used to target the environments you need while reducing manual scripting and steps. But before you throw in, sit with your vendor’s tech people and ask them to walk you through stumbling points, they likely know far more gotchas than I do, because it’s their job, and unless you got the new guy for an SE, have probably seen more. And getting ARA up and fully functional quickly allows you to focus on other DevOps needs, such as improving automated testing.

— Don Macvittie

August 11, 2017
FLAP, Part 2: Application Provisioning

In part 1 of this series we talked about server provisioning tools. In that post, we mentioned some of the more prominent application provisioning tools on the market with regard to integrations. This post will focus on the application layer of full layer automated provisioning (FLAP).

Application Provisioning Who’s Who

Application provisioning technologies most likely are familiar: Puppet, Chef, Salt, etc. We’ll take a look at each, how they compare in the areas of control mechanisms, operating systems support and ease-of-use features (integration with server provisioning tools is mostly covered from a server provisioning tool perspective in part 1 and part 3 of this series, because integration must be managed in the server provisioning tool). A blog is too short to go much deeper than that, but this will help you determine which ones should be considered in your environment. Information here is valid as of the day this blog is written, but the market is moving fast, so don’t hesitate to check at the provided links for improvements. Because this topic is a popular one, don’t hesitate to look for up-to-date information provided by other sources, either.

It is worth mentioning that all of these tools are moving to support configuration of network equipment. For a guy who spent years worrying about programmability and interoperability at a networking gear vendor, that is very cool news. I haven’t had time to actually try any of that support yet, so I’ll leave evaluations to you.

A word of advice: This entire marketplace has cutesy-naming disease when it comes to really important concepts. While you are grimacing at “recipes” or “grains” or whatever, learn that terminology. It may be cutesy, but the heart and soul of these products hides behind those names and the details of what concepts they encapsulate. During your evaluation period preferably—but certainly once one is chosen—you should get those terms and their meanings down pat. It will make your life much easier. The same is true with the variety of scripting formats. They’re all communicating basically the same information, but so do English and Japanese—knowing one doesn’t automatically makes you an expert in the other. Take the time to truly understand the scripting language for your tool of choice.

Ansible

Ansible is a full-app provisioning tool that allows users to install and customize applications through a combination of a command line and scripts. The tool is well-documented, easy to install and works very well at its designed task. Ansible offers support for Red Hat and Windows running on Docker, OpenStack and AWS. Notably missing is VMware support, but since the machines are available via IP and the OSes are standardized, VMware virtuals can be managed/orchestrated with Ansible.

The open source version of the tool comes with command line interfaces and management is handled via command line and editing scripts. The premium add-ons such as Tower include UI, analytics and programmable APIs.

Since the purchase by Red Hat, it is reasonable to expect an increased focus on RHEL for this tool and a commensurate reduction in non-Red Hat OSes. Not that this is assured, but history says it is a likely long-term result. If you’re a Red Hat shop, Ansible is a good place to start—though looking at Satellite—and trying to predict how Red Hat’s various open and add-on provisioning tools will evolve and emerge—is a good idea for any evaluation at this point in time.

Chef

Chef is one of the original application provisioning tools out there. It is managed through command line and scripts in the open source version, and through these plus a GUI in premium. Unlike Ansible, many Chef APIs are bundled with the open source product, only APIs for premium services like analytics and reporting are premium.

Chef supports most variants of Linux and Windows, with most cloud and virtualization vendors available as platforms. In terms of completeness of support, it really is up toward the top of the list.

If you have a largely heterogeneous environment, Chef is a good tool to look at for automating application provisioning—be it for deployment, refresh or continuous delivery.

Juju

Juju advertises itself as an orchestration layer above the other products in this space. While there is a small amount of truth to that claim, most of what you can do in terms of orchestration with Juju can be done with the others. In fact, Puppet has kind of found a home in entire complex multi-server systems. Juju comes with GUI in the open source version, but the product does not have a traditional API.

Juju supports Ubuntu and Windows, and is interoperable with most of the major cloud providers. Deep server provisioning integration is really only provided for MaaS (another Canonical project), but users have made it work with other server provisioning tools. Juju scripts (charms) are primarily written in Ruby, though other languages are definitely possible. For Ubuntu shops, Juju would be the item I’d place on the top of the application provisioning list. For Windows shops, it would definitely be worth considering. For Red Hat shops, other solutions are probably more appealing.

Puppet

Puppet should need no introduction to readers of devopsy.kinsta.cloud. The standard in application provisioning, it uses scripts and command lines to deploy applications with a pretty broad base of user support that equates to lots of user shared scripts. Other vendors in this list also have vibrant communities contributing scripts, but Puppet, by sheer weight of numbers (and marketing), has a larger selection available.

Supporting all major operating systems, Puppet’s documentation is solid. The only quibble is that you have to be careful when coming from a search engine about whether you’re on the open source documentation or the Professional Edition documentation. Puppet comes with APIs, though the GUI is only available in the commercial version. Puppet is the progenitor of Razor, the server provisioning tool, and while it has been used with all of the provisioning tools in the first iteration of this article, expect it to work best and provide the most value with Razor going forward.

For highly heterogeneous shops, or shops that include multiple server provisioning tools, Puppet is a solid bet. While it will work fine in other situations, the value-add of a more focused product is worth considering if your environment maps to another tool’s strengths.

SaltStack

Salt is an interesting member of the application provisioning crowd. They don’t share name recognition with some of the other players, and yet have a respectable user base. Their product is certainly comparable to the others in this market, and the user community is pretty active. Not being deeply associated with any of the server provisioning tools, SaltStack has been integrated with most of them.

The Salt Master runs on almost all flavors of Linux, and can manage all major Linux distributions plus Windows, OS X and Solaris.

Like all of these other tools, SaltStack can manage any machine that is accessible over the network and is running a supported operating system., Additionally, the dev branch has Salt Cloud built in to the Salt Master, which spins up cloud images, automatically puts them into salt and starts managing their configuration. Expect that this type of functionality will become the norm for the market (Juju does some of this also, and others have varying levels of cloud support)—it just makes sense, and it compliments server provisioning. There is a tiny bit of overlap in the VM space, but then you can choose whether to use server provisioning or Salt Cloud to handle those instances.

For highly heterogeneous shops that wish to use best-of-breed (actually “best fit” is a better name) server provisioning along with a solid application provisioning tool, SaltStack is a heavy hitter worth considering. If you have a more specialized environment focused on a single OS, SaltStack will work, but its advantage of broad support for both OS’s and server provisioning tools will not come into play.

Satellite

Satellite is Red Hat’s provisioning solution. Unlike most of the others in this list, it is a cross between server and application provisioning. In actuality, it is that “layer above provisioning” that Juju positions itself as. Under the covers, Satellite (as of release 6.0) is using Foreman for server provisioning and Puppet for application provisioning. The reason it still makes this list then is because it does do application provisioning, and does orchestration too. You can use it to spin up MariaDB or OpenStack, and it is a marketed and supported product, so it fits the list.

As one might wholly expect, Satellite supports RHEL. It can manage instances of RHOSP and RHEV, along with VMware and Amazon, and does bare metal through Foreman also. Satellite is deeply tied to Foreman and Puppet, so if you have issues using this tool set, it’s not going to be your first choice. Because Satellite extends and improves Red Hat Network access, for Red Hat shops it should be No. 1 on the list of viable options, but if you use a lot of non-Red Hat machines, including (as of version 6) Fedora and CentOS, it is less appealing. It used to be that Spacewalk was the viable option for those CentOS and Fedora installs, but now they are separate code bases, meaning you have to maintain two different systems. At that point, it is worth looking at other options available on the market.

The newer version 6 architecture is designed to provide a platform moving forward that is more powerful and adaptable, so expect to see good things from it if you are a Red Hat customer, possibly even cancelling out the disadvantage of no support for CentOS and Fedora.

Summary

All of these products are very good in their best environment, and most of them will give you pain in their very worst environment. The question is, which environment do you have? While only employees can answer that for a given environment/tool combination, hopefully there are some pointers here—and elsewhere—that can help make the decision.

In the third and final installment, we’ll look at the combination of server and application provisioning to offer full stack automation.

April 7, 2016
FLAP, Part 1: Server Provisioning

We have reached the state where end-to-end data center provisioning is a reality. It’s early yet, so there is a certain amount of work involved, but today a DevOps team essentially can turn on the automation and move on to other DevOps priorities. It takes some effort, depending upon the solution chosen, but the end result is an infrastructure that provides for installation of operating system (OS), hardware (if needed) and apps, along with configuration of the whole, at the push of a button.

There are a number of solutions ready for your data center today, and this three-part series will explore them. In the first part, we’ll talk about server provisioning—the part that takes you from uninstalled server or virtual to a functioning OS configured for your environment. The following two articles will cover application provisioning and merging the two to achieve FLAP. (Yes, I made up FLAP. Use it or not, as you see fit.)

Disclaimer: I work for a vendor in this space. While I do not believe that impacts my objectivity, I will note where my employer is mentioned, and leave it to the reader to decide if I’m blinded to some items/issues based upon my association.

Server Provisioning: What and Who

A server provisioning system takes a physical or virtual server from nothing to fully installed OS. This layer of FLAP is fundamental to automating the entire process. If the system has to be installed manually, using tools and scripts to install applications just doesn’t offer the overall benefit as when the entire process is automated.

For this discussion, we will be sticking with the major provisioning tools out there. To be sure there are more, and I have information about them, but for this short blog post, we’ll stick with the ones that hold something special—market share, history, integrations, associations—that make them a good long-term bet. Our list is alphabetical, and does not show preference by ordering.

Cobbler

Cobbler is one of the original server automation tool sets, and definitely has the history and stability to warrant your attention. For shops that use a lot of Fedora and CentOS, Cobbler in combination with Spacewalk is particularly appealing, as we’ll talk about later. Support includes most Linux distributions. Users have created ways to include Windows in the list of installable OSes, but there is not yet an official solution for Windows.

Cobbler is integrated with Ansible, Spacewalk, Saltstack, Puppet(user created), and older versions of Satellite. There is some amount of concern about Cobbler’s future growth, simply because Red Hat moved Satellite off of Cobbler with release 6.0, though Red Hat still supports Cobbler. It is too soon after the split to predict how this will fall out, but currently it seems to have motivated the Cobbler core team to do more. Users concerned should check GitHub analytics to see what the trends are for committers and commits at the time they are evaluating (good advice for all of these products, but more pertinent for one where there is a question of its future).

Cobbler’s hardware support is better than most, and in terms of application support it is a pure-play server automation tool; there is no support for applications at all—application provisioning is expected to deal with that aspect.

Foreman

Foreman is an open source project that is currently enjoying the sponsorship of Red Hat, but support is not limited to Red Hat OSes. In fact, Foreman will install just about any flavor of Linux, and can target most physical/virtual/cloud platforms. Using WDS, Foreman can also install Windows.

One of the big strengths of Foreman is its interoperability—it doesn’t much care which application provisioning tool you are using. It is integrated into Satellite (with Puppet), and has support for all of the other major application provisioning tools. This means that no matter what you’re using to get your apps installed, Foreman is worth looking at for getting your servers installed.

Like most products out there, Foreman provides CLI, API and GUI. They’re relatively stable in relation to the market, simply because Foreman implemented them earlier than most competitors, and is constantly improving them.

In short, if you are in an environment that has a wide selection of OS, architecture and application requirements, Foreman should be at the top of your short list. For more focused solutions (in terms of OS, whether or not cloud, or application provisioning tools), it makes more sense to compare specific features/functionality with some of the more focused tools in this list.

MaaS

MaaS was the sleeping giant of the automation world, but it does indeed look as though it is awakening. MaaS is part of Canonical’s Ubuntu project, so there is plenty of backing, but historically its goals have been modest and its documentation more so. That changed some time over the summer/fall, and MaaS is showing increased support for OSes, better documentation and a more focused attempt to get the word out. If you are primarily an Ubuntu shop, it is well worth considering.

Like most of the products in this offering, MaaS uses PXE to get the machines into its system, and then uses predefined images that have been imported (Ubuntu images don’t need importing) to install a machine from bare metal/virtual to working OS. MaaS proclaims support for CentOS, RedHat, SuSE, Ubuntu and Windows. For those unfamiliar with the space, that’s an impressive list. But the last time this author used MaaS, it was still only Ubuntu, so I cannot comment on the level/quality of support for each of the OSes. Due diligence should involve testing with those OSes your organization needs.

The weakest part of some installers is non-standard hardware, such as RAID controllers, specialized network cards and the like. MaaS supports these components and includes UCS support in its packaging. Directly mentioning UCS support is only to be found with MaaS and Stacki, so if you’re a UCS shop, it is worth considering.

In addition to CLI, MaaS touts an API and a GUI. These are important tools in the DevOps sense, because the API lets you tie MaaS into your automation architecture, and the GUI helps find issues specific to MaaS and its domain. Particularly interesting is the collection of logs that show install results and operations results clearly for each server.

The one major drawback to MaaS is its limited application provisioning support—it is well-integrated with Juju, but none of the other big names in the space.

In short, for Ubuntu shops using Juju, this is the tool to put at the top of your short list. For everyone else, the tool is viable, but should be considered against the features/functionality of other tools.

Razor

Razor is Puppet’s newer bare metal installer. It is young yet, and there are still issues to be worked out, but the large cadre of developers that are dedicated to Puppet say that Razor will continue to improve as time moves on. At current it directly supports most flavors of Linux and Windows installations. The catch to Razor is on the other end: It is designed to work with Puppet, and support for other application provisioning tools is low—indeed, is limited to user-designed solutions. Needless to say, because it is sponsored by Puppet, it is expected that official work in this area will remain slow, as it is not a priority.

Razor does not have a dedicated GUI; instead, it uses Puppet’s GUI to report results. As with integration, this is okay for Puppet shops but pretty much eliminates Razor from consideration for other shops. (More on this in a future blog in this series.)

Because Razor is new, there are a couple of serious caveats with the product. First, install is one of the worst in the market, with a lot of pre-install steps, a lot of install steps, and configuration of an array of underlying systems by hand. Second, each currently installed server that you do not want Razor to re-install must be entered into Razor manually on the command line. This is so limiting in a generalized data center environment (where Puppet shines) that you can expect this limitation to be resolved quickly.

In short, for Puppet shops that do not see their application provisioning tool changing, this is the tool to put at the top of your short list. For other shops, it’s worth looking at competitors first, simply because Razor requires Puppet.

Stacki

Stacki is an open source project sponsored by StackIQ (as promised, this is my employer). Having roots that stretch back more than a decade, Stacki specializes in RedHat/CentOS installations with additional support for clustered software. As of this writing, CoreOS and Ubuntu have been added to Stacki’s list of supported OSes, and expect that list to grow over time.

The biggest appeal of Stacki is the variety of ways to get systems under management. All of these apps will auto-detect a new server on the network, grab it and install it. Stacki adds the ability to list just the servers that should be clean-installed in a spreadsheet, the next time each of those machines boots they will be installed, but no other machine will. While there are ways in all of these systems to simulate this functionality—you can add exceptions to the “install all machines that boot on this subnet” rule via the command line in Razor, for example—using a spreadsheet to affirmatively manage what machines are to be installed and how gives an offline bulk way of managing installations and re-installations.

As of this writing, Stacki does not have an API, and only with commercial add-ons can a GUI be had, but expect one or both of these things to change as the community begins guiding development efforts. Stacki is optimized to spin out hundreds of servers in minutes or hours instead of weeks or months, utilizing a peer-to-peer parallel installer to service installations faster as more servers come online. While it works as well as others in smaller environments, it really comes to its own when there are a large number of servers under management. On the initial installation front, while the other systems in this list require hand-installing and/or configuring prerequisites and sub-systems, Stacki comes as an ISO with a UI that asks a few questions and configures everything.

Stacki has been used with most (Ansible, Puppet, Saltstack) of the major application provisioning tools this series of blogs will consider, making it versatile on the application side of the equation.

Overall, if you’re a Red Hat shop that uses CentOS a lot—particularly a larger shop—Stacki should be the first stop on your short list. For everyone else, the tool is viable, but should be considered against the features/functionality of other tools.

Stay tuned for part two of this series, which will cover application provisioning.

April 5, 2016
Rugged DevOps: Good, Bad & Ugly

The Good: DevOps has been a success for Dev.

With higher expectations, better processes, more management focus and new tools like advanced PaaS and containers, developers are churning out more and more software faster than ever. Success! Or is it?

The Bad: DevOps has been a success…for Dev.

For Dev teams, this may result in pats on the back and high fives. But for Ops, this can look like one thankless tidal wave of work after another. The same software velocity that is praised in Dev is currently overwhelming their counterparts in Operations. And frankly, many DevOps tools released in recent years happen to benefit Dev much more than their counterparts in Ops. (My pet theory is that developers at IT automation firms are acting and thinking like…developers – focusing their creative energies on solving Dev problems.) A similar problem holds true for IT Security teams who want to build quality in by “shifting left” – see https://devopsy.kinsta.cloud/2015/12/10/getting-rugged-devops-right/ . From the vantage point of IT Operations and Security teams, they are doing more work than ever but without any more human or IT automation resources. To add insult to injury amid their increased workloads, many IT Ops teams have to invest countless hours (away from family and personal time) in training courses and migrations to reorient their processes to fit into shiny, new Dev-centric “DevOps” tools and paradigms.

The Ugly.

Work. Overwork. Glitch. Search. Find. Rework. Repeat.

A quick glance at IsItDownRightNow.com will show you which IT Operations teams are in a living hell at the moment. The other thing you will notice is that these IT Ops teams are from some of the most reputable companies on the planet. They are talented teams but stuck in rework cycles of web performance problems, security vulnerabilities and outages.

These problems get the attention of everyone. Operations…and Dev…and Security…and customers…and Finance…and auditors and the press.

No one likes rework. And no one likes the panicked fire drills that cause panicked rework. So what causes those rework cycles?

Drama-free IT. Visibility Cures Chaos.

Much of this chaos can be tied to miscommunication. Boring? Maybe. Simple? No. Part of miscommunication is cultural. (Hint: OrcaConfig’s “How to Screw Up DevOps” white paper covers the cultural aspects of miscommunication). But miscommunication is also cured by automation solutions that promote transparency and visibility.

If we want drama-free IT we need IT automation solutions that eliminate miscommunication and missed assumptions between Dev, Ops and Security.

What better way to do this than with full transparency and visibility?

Bird’s Eye Views:

Many IT automation tools cater to SysAdmins who are interested in managing configurations of individual servers. That’s great if your focus is on individual severs. But IT Operations, eCommerce teams, Application Owners, Compliance & Security teams and IT Managers often care more about the health and performance of their application ecosystem (including related applications, middleware, databases and configurations that tie them together). So there is often a mismatch between the tools that SysAdmins and Developers have versus what IT Ops needs.

Operating at Enterprise Scale Requires at-a-glance Visibility:

IT Ops teams need bird’s-eye views of applications, databases, middleware and ecosystems to promote ease of ongoing operation. They need to be able to see, at a glance that a configuration compliance problem exists and they need to know right away the location and nature of the error. Large scale IT Ops teams do not have the luxury of toggling back and forth between multiple reports and multiple screens searching for that non-compliant, broken needle in an ecosystem haystack. If they are faced with a glitch or an outage, they need to find and fix it right away. One size does not fit all. Dev, Ops and Security each have their own interests and their own need to view their application through a lens that is most meaningful for them. Multiple ecosystem views eliminate guesswork between Dev, Ops and Security and save precious time in the find-fix cycle.

Forrester Research spells it out well in their 7 Habits of Rugged DevOps white paper, Habit 1: Increase Trust And Transparency Between Dev, Sec, And Ops:

“I&O pros pride themselves on understanding the interconnectedness of the hardware and software environment, and they are unnerved by complexities that poorly governed delivery practices create because the firm rates I&O pros on their ability to improve the performance and stability of the production environment. To maintain the environment, they must reduce the chances of an outage or interruption of service, and they must reduce the MTTR when an issue does occur.“

Understanding that interconnectedness depends greatly on how much intuitive ecosystem visibility we provide to our colleagues in IT Operations. But it’s not just IT Ops that benefits from visibility, Dev and Security do too.

Developers are able to write more secure code when they get early, in-context feedback. As David Mortman, distinguished engineer and Dell chief security architect, said, “Security issues are product quality issues, and no one wants to write buggy code.”

Even better than efficiently finding and fixing errors is of course to prevent them in the first place and to proactively eliminate the damage they cause. Here too, the answer is better visibility. With true application ecosystem visibility, Dev and Ops teams would be able to model relationships between applications, middleware and their configurations so users will know that “changing this, affects that”.

My teenage son has better visibility via his smartphone as to his driving route and weather conditions than I have for my multibillion dollar IT operation.

Visibility into the Future:

Ecosystem visibility is a key to communicating between Dev, Ops and Security and running complex application environments in the present. But what about the future?

Release automation tools and custom scripts are excellent at deploying changes across large environments. Unfortunately they can be just as efficient at deploying typos and other mistakes across large environments, causing the chaos that we discussed earlier. Here again Operations teams would benefit from improved visibility – the ability to preview and pre-validate proposed changes across the entire ecosystem well before those changes are deployed. This is the real promise and savings of ecosystem visibility. Better visibility, with more use-case lenses cuts out the miscommunication that causes too many performance glitches, security vulnerabilities and outages. This chaos takes a toll on our IT Operations teams, our employers and our customers.

What Next?

We are soon entering a new season with new budgets. The temptation to “buy something” will be great. In the spirit of Rugged DevOps my recommendation is to take a full, bird’s eye view of your SDLC and to shift Ops and Security leftwards, building their critical considerations into your processes and into your IT automation tool selection.

December 22, 2015