Even organizations with a mature DevOps services model inevitably will hear from executive sponsors about the “need” for a dashboard for DevOps information. Organizations just starting to embrace DevOps won’t have the time to construct one yet, but over time will reach the same conclusion. The need is real, and the value is significant—it is how you get there that makes the difference. Vendors are keen to promote the idea that they already have a dashboard built into their respective products. And this is partially true.
But the problem with dashboards is not a matter of information, it is a matter of perspective and scope. Ultimately, all of the information an audience wants can be found in the various DevOps tooling used in the continuum of services your organization created. However, locating some of it takes profound knowledge of how each tool operates, adequate security profiles and an elephant’s memory as you click on multiple windows to try to correlate the data you ultimately are seeking across the various tools.
Before you know it, executives and perhaps even practitioners are calling for a consolidated dashboard that would present this information cleanly. The danger here, of course, is an attempt to consolidate every perspective from every individual audience into a single dashboard for your company. Instead, it is wiser to begin such an effort by breaking down the number of audiences into categories (by need, perhaps) and examining what is already available and what is missing to make a dashboard project effective, and not one originating from hell.
Dashboards: The Practitioner Perspective
There is a keen difference in what a typical engineer wants to see as it relates to DevOps from what the other audiences will be looking for: namely, detail. If you consider this part of a dashboard effort the foundational section (i.e. providing the right detail information to the engineers who ask), the remaining higher-level presentations can be built on top of this data. Most engineers are content to just “look into the given tool” to get what they want. But sometimes consolidating this information into a dashboard begins to make more sense.
Ideally, your company has only one build tool and, again ideally, only one deploy tool. If you have multiple types of both kinds of tooling in these disciplines, you would be better served to pick a winner there than try to use a dashboard effort to consolidate build and deployment data across multiple build and deployment tools. Allowing multiple kinds of build and deployment tools across an enterprise usually winds up in the demand for multiple dashboards from multiple audiences, and before you know it, the dashboard project from Hades once again has infested your organization. A clear winner in build, deployment and release orchestration cuts out this kind of problem.
Most of the time the need for a dashboard emerges when users are trying to see information in a way the native tooling did not plan for. For example, I want to look at a given server, or class of servers, and a list of everything that is on them (by version). Or as an example, I want to look at these technology components of this type and version and find out every server in my entire data center that they may have been deployed to.
Some vendors offer internal reporting capability intended to meet these needs. However, using those reporting features should be labeled with “caveat emptor” signage. Typically, vendors apply their reporting directly against online or production data within their respective tools. So an engineer presses the report button on his build tool, and the build reporting function begins to look at its own data from a production viewpoint, instead of from an offline reporting copy of the data. When you start running reports against “live” tools, you get slowdowns, sometimes significant slowdowns.
To avoid this issue, start with determining what data the engineers need, how often they need it and to where you can copy this information (to form the basis of your dashboard system). You also may wish to examine how the base data is inherently constructed and join some of it where it makes sense, so you are not issuing very large queries joining key data from many tiny tables together before anything sensible emerges for your new dashboard system. Speed of presentation will matter still, even though the data becomes only near-real-time instead of real-time.
Dashboards: The Business Perspective
Once you have ascertained the detailed information to satisfy your engineering or practitioner community, it usually needs to be married to your business segmentation to meet the needs of this audience perspective. For example, I may have 30 servers that I use regularly in my e-commerce products or line of business division. My engineers may care less about this designation, as they only care which servers are content servers or database servers or application servers. But my e-commerce business teams look at this information differently. They only want to see “their” servers, not every server in my company. So to accommodate the next perspective audience, I need a way to marry servers and technologies to the product lines or lines of business my company understands.
This may require an outside or additional database that tracks the raw server name and marries it with the business unit that cares about it. It may also track the application names the same ways, marrying them with the appropriate business unit. But this marriage now offers the ability to pull reports (or dashboard screens) by line of business—it is nearly certain your basic build and deploy tools weren’t designed to show information in this way. This kind of business segmentation, or filtering (in some tools), provides an entirely new way to examine the data. The need to drill down to the details may still exist, but a picture of change relative to a product line or business line becomes a high value-add for the business community who looks at DevOps data.
The business community looks at DevOps information to help it make decisions based on the state of change at any point in time. E-commerce customers, for example, might get special company notifications of upcoming changes or improvements in their capabilities, as the business community is made aware, and stays current with the planned release events and quality of testing underway. Or those same notifications might be delayed based on a quick look at the same information presented the same way.
The point is, business users are an important community or audience of DevOps information, and the lens they use to examine data is from a product or line of business construct rather than the raw data aggregated by a practitioner.
Dashboards: The Executive Perspective
Executives, no matter the level, usually are more concerned about trends in the information than point-in-time data. For example, a given executive may want to know how many deployments per month are happening out of the development class environment for a given line of business. If you have constructed your dashboard with the architecture laid out above, you should be able to provide this kind of information. The discreet data is stored offline (copied from the build and deploy and other DevOps tools themselves), then married with the business segmentation data. Now it can be presented in trend-over-time manners that executives invariably want.
It is much more likely that a business team member will use the drill-down features of your dashboard than your typical exec. But having it available up to the executive layer is a huge win when demonstrating the detail of what is going on where and potentially framing the why. The other significant win of having a single dashboard for presenting trending data is the consistency in the presentation across the enterprise. Consider for a moment how many Excel spreadsheets are sure to spring up around the company when execs task their people to produce reports that aren’t available in a dashboard at their fingertips. Each minion of the exec likely will grasp certain data in unique ways, often resulting in bar charts that never seem to line up.
Having one dashboard—or, rather, one method of presentation of the same data—eliminates having multiple unique minion versions that lack context and often misrepresent the real story. In addition, without having a central dashboard you construct and control, minions will pull data requests against the production copies of build and deploy and release tooling, perhaps slowing these tools down to a crawl while data is gathered. The act of looking at a thing will certainly change the speed of that thing, in this instance.
Impact to the Bottom Line
Meeting the 80/20 needs of your primary audiences of DevOps data can be done without tying up programming resources forever. Once you have delivered a core dashboard system with mainline capabilities, your executive or business teams can decide how much more resources they are willing to fund to continue to update and enhance the dashboard. As long as they fund it, you can dedicate the resources you need to keep making it better (see agile). But for most shops, this kind of dashboard is a three-to-six-month effort from end to end.
Once the dashboard is up and running, it becomes “the” tool in the arsenal of transparency for DevOps services. Granting access to this kind of tool (i.e., setting up user IDs) is significantly simpler than doing it in the multiple of underlying tools you have to administer without it. The cost of administration in a large firm can be substantial if a dashboard is not present. Imagine having to set up a typical user ID in your build tool, deploy tool, configuration management database, testing tool(s), release orchestration tool and monitoring tool—and that’s just to start. Within these types of tools that provide a real service or function are secondary concerns such as security or permission profiles, group memberships, etc. A dashboard only needs a valid company ID, and you are ready to “see” change in motion via DevOps.
The work effort typical executives “task” their minions to produce reports is usually no small effort, either. While this work effort is rarely captured or understood by the executives who request it, it definitively puts a drain on company resources (both man and machine). A functional dashboard can eliminate this work nearly altogether, and produce a consistent work-product output. Your DevOps strategies should be created to reduce costs and facilitate innovation at every turn. If you need help feel free to contact me, I am still looking at the moment 🙂 .
To continue the conversation, feel free to contact me.
