devopsy.kinsta.cloud

Tag: graphs

  • Making the Data Work for You: Data Visualization Techniques

    Making the Data Work for You: Data Visualization Techniques

    When Peter F. Drucker, the father of modern management, performed a two-year in-depth analysis of the General Motors corporation in the 1940s, he undertook what may have been the world’s most ambitious big data analysis ever at that time. However, when he gave the results to GM chairman Alfred Sloan, Al was not happy.

    In fact, Sloan refused to acknowledge Drucker’s work. He never mentioned it again and banished his colleagues from referencing the resultant book (Concept of the Corporation) around him.

    Alfred Sloan may not have been ready to act upon the harsh reality of his company’s data, but modern businesses thrive on this sort of analysis. Big data has consumed every industry on earth, from automobiles to cosmetics to hotels and far beyond.

    Businesses that use the power of data visualization will outperform the ones that don’t, every time. If you are responsible for the success of a B2B business, this article is going to show you why data visualization is one of the most powerful tools at your disposal.

    The Goal of Data Visualization

    There are two ways you can use big data visualization tools to make your data work for you. You’ll make it easier to parse your own data, identifying trends and finding opportunities to improve your business. You’ll also use it to convey your ideas to other people, improving your conversion rate for B2B sales that can make or break your bottom line.

    Understanding Your Data

    When it comes to better understanding your own data, visualization is everything. Even a small business has an incredible amount of data at its disposal. Social media statistics, front-end stats such as how long users spend on certain web pages, sales and refund statistics by time of day and product … the list goes on and on.

    You have to start by figuring what data is most relevant. Ed Baker, former head of growth at Uber, points out that every great business needs a “north star metric.” This is the single variable that, if it improves, the entire business does better. Once you identify your north star metric, you can see how other variables feed into that metric. Data visualization is crucial for parsing the information and making these big decisions.

    Conveying Ideas to Other Businesses

    You can use well-visualized data to convey ideas to other people that might be near-impossible to express otherwise. Have you ever sat in a meeting, desperately trying to show somebody why your idea makes sense, and yet completely failing to do so?

    I’ve had conversations where my brain is about to melt out of my ears, because I think I’m making perfect sense yet the other party simply does not grasp the idea. In a B2B context, this failure to communicate can cost hundreds of thousands of dollars. That’s a critical failure to get the job done. Data visualization is a very cost-effective way to improve your odds of making the sale.

    When you’re trying to sell a premium service or product to another business, don’t just tell them why it makes sense—show them. Use the visualization techniques in the rest of this article to remove all doubt that you’ve got the right idea.

    The 7 Basic Tools of Quality

    There are dozens of different ways to visualize data. It depends on what kind of information you want to look at. However, the best way to start is by understanding the “seven basic tools of quality.”

    Made famous in post-war japan by Kaoru Ishikawa, these tools are simple yet potent data visualization methods that most people can learn in a few hours. While not the most intricate or powerful visualizations available, they are elegant and widely understood. When you use one of these tools, you can rest assured that everybody at the table will be comfortable following along.

    You want to use these methods of visualization whenever it is possible. Only move into more esoteric or powerful tools when the situation necessitates it. By the time you are working with complex multi-variate data visualizations, you’ll probably need a dedicated data scientist to make it all work. These tools are an easy and cost-effective way to get started.

    Here are the seven basic tools, with a brief synopsis of how each one works:

    Cause-and-Effect/Fishbone Diagram

    A quick and simple method to use when you are working backwards from a known outcome, asking the question: “Why did it happen?” This tool is often too simple for detailed analysis, but can be great for mid-conversation visualizations on the back of a napkin.

    Check Sheet

    We’ve all used check sheets in our day-to-day lives. This is a great way for small groups to keep track of simple data, such as, “How often are the following conditions being met?”

    An example of a check sheet in action would be a coffee shop that wants to track common issues with customers. You could have the barista check a box whenever something goes wrong: Customer says drink tastes bad, customer leaves before drink is ready, customer gets confused while ordering and becomes agitated, etc.

    Control Chart

    This is a X & Y axis chart where the horizontal axis is the flow of time and the center of the vertical axis represents the average value of the other variable. The top and bottom of the vertical axis are the outer bounds for that variable. The control chart is a simple way to visualize data for a single variable moving through time.

    Histogram

    You’ve probably heard the word “Histogram” a lot, but may not know what it is. This is a chart where the horizontal axis consists of columns representing one variable and extending upwards to show the relationship with a second variable on the vertical axis.

    Pareto Chart

    A chart consisting of columns and a line, sort of like a blend between the Histogram and the Control Chart. However, Pareto charts do not necessarily reflect the passage of time. The columns are always ordered from largest to smallest and the line is the sum of the data represented by the columns.

    Scatter Diagram

    This is a simple diagram that plots two different variables with lines across an X and Y axis, meant to show correlations between them.

    Flow Chart

    Also known as an “if-than” chart, this shows the potential sequence of actions or thoughts within a mental or physical process. For example: If the client likes coffee, ask them how they like it. If not, offer them water. If the client says they like the coffee with cream, add cream. If not, prepare it black. This sort of data is awkward to express through pure language, but quite easy to grasp when placed in a flow chart.

    Advanced Data Visualization

    There are many, many forms of advanced data visualization. This is the stuff that Peter Drucker was trying to do back in the 1940’s—capturing huge amounts of data across an entire corporation and then synthesizing it into the most elegant possible form.

    If you want to go beyond the seven basic tools, you’ll need help. This isn’t the sort of thing that an employee or even an ambitious executive can tackle over the course of a few days. A team of data experts will need to spend a lot of time and energy to craft the perfect visualizations for you. With that said, here are a few advanced techniques to consider:

    Heat Maps

    Heat maps are used to show density of activity across a geographical space. This can be utilized to understand city traffic, or to look at where your employees spend time within an office building.

    Beyond the physical world, you can use heat maps on websites to see where users tend to move their mouse, or even where they tend to glance at. For example, video game developers and social media platforms use heat maps of user eye-gazing data to optimize their visual layouts.

    Word Clouds

    A word cloud is a visualization where many different words are scattered about in a “cloud,” with larger words being more common. You can use these to understand the most common sentiments being expressed in customer reviews, company emails or other text communication.

    While cheap word-cloud-generators exist, there are many reasons why they do not work. An effective word cloud needs to deal with factors ranging from color to glyph shape and beyond.

    Comparative Graphs

    Comparative graphs combine several copies of the basic power tool charts, by for example putting several control charts on top of each other to compare larger numbers of variables and factors. While comparative graphs are simple to understand in theory, they are difficult to put together in a way that is practical.

    Using Data Visualization for Your Business

    The best way to understand data visualization is to give it a try. You can get started with basic visualizations on your own, but you’ll want to contact an expert to really make your data work for you.

    After all, it isn’t enough to put a few variables into a chart. You need to understand how to collect the right data, compile it into useful visualizations, and then act upon that data in an effective way.

    About the Author / Alexey Semeney

    Alexey Semeney is the CEO of DevTeamSpace, where he leverages his knowledge and expertise in product design and remote team management to build a unique software development platform. Connect with him on LinkedIn.

  • An Operational Approach to Benchmarking Microservices

    An Operational Approach to Benchmarking Microservices

    Benchmarking microservices often falls behind on the engineering team’s to-do list. However, those that aren’t are missing out on a key component. The fundamental goal of benchmarking is to better understand the software, and test out the effects of various optimization techniques for microservices. In this post, I’ll describe an effective approach to benchmarking microservices.

    First, create a spreadsheet for tracking your benchmarking. One convenient way to document a series of benchmarks is in a Google Spreadsheet. It allows collaboration and provides the necessary features to analyze and sum up your results. Structure your spreadsheet as follows:

    • Title page
      • Goals
      • Methodology
      • List of planned and completed experiments (evolving as you learn more)
      • Insights
    • Additional pages
      • Detailed benchmark results for various experiments

    Then, before you engage in benchmarking, clearly state (and document) your goal. Examples of goals are:

    • “I am trying to understand how input X affects metric Y”
    • “I am running experiments A, B and C to increase/decrease metric X”

    Next, pick one key metric, stating clearly which one metric you are concerned about and how the metric affects users of the system. If you choose to capture additional metrics for your test runs, ensure that the key metrics stands out.

    Now, you’re going to have to think like a scientist and perform a series of experiments to better understand which inputs affect your key metric and how. Consider and document the variables you devise and create a standard control set to compare against. Design your series of experiments in a fashion that leads to the understanding in the least amount of time and effort. You’ll then need to define a methodology for running your benchmarks. It is critical your benchmarks be:

    • Fairly fast (several minutes, ideally)
    • Reproducible in the exact same manner, even months later
    • Documented well enough so another person can repeat them and get identical results

    Document your methodology in detail. Also document how to re-create your environment. Include all details another person needs to know:

    • Versions used
    • Feature flags and other configuration
    • Instance types and any other environmental details

    In most cases, to accomplish repeatable, rapid-fire experiments, you need a synthetic load generation tool. Find out whether one already exists. If not, you may need to write one. Understand that load generation tools are at best an approximation of what is going on in production. The better the approximation, the more relevant the results you’re going to obtain. If you find yourself drawing insights from benchmarks that do not translate into production, revisit your load generation tool.

    It is then important to validate your benchmarking methodology, and repeat a baseline benchmark at least 10 times and calculate the standard deviation over the results. You can use the following spreadsheet formula:

    =STDEV(<range>)/AVERAGE(<range>)

    Format this number as a percentage, and you’ll see how big the relative variance in your result set is. Ideally, you want this value to be less than 10 percent. If your benchmarks have larger variance, revisit your methodology. You may need to tweak factors like:

    • Increase the duration of the tests.
    • Eliminate variance from the environments.
      • Ensure all benchmarks start in the same state (i.e. cold caches, freshly launched JVMs, etc).
      • Consider the effects of Hotspot/JITs.
    • Simplify/stub components and dependencies on other microservices that add variance but aren’t key to your benchmark.
      • Don’t be shy to make hacky code changes and push binaries you’d never ship to production.

    Important: Determine the number of results you need to get the standard deviation below a good threshold. Run each of your actual benchmarks at least that many times. Otherwise, your results may be too random.

    Now that you have developed a sound methodology, it’s time to gather data. Some tips include:

    • Only vary one input/knob/configuration setting at a time.
    • For every run of the benchmark, capture start and end time. This will help you correlate it to logs and metrics later.
    • If you’re unsure whether the input will actually affect your metric, try extreme values to confirm it’s worth running a series.
    • Script the execution of the benchmarks and collection of metrics.
    • Interleave your benchmarks to make sure what you’re observing aren’t slow changes in your test environment. Instead of running AAAABBBBCCCC, run ABCABCABCABC.

    Now, you’ll need to create enough loads to be able to measure a difference. There are two different strategies for generating load:

    1. Redline it! In most cases, you want to ensure you’re creating enough load to saturate your component. If you do not manage to accomplish that, how would you see that you increased its throughput? If your component falls apart at redline (i.e. OOMs, throughput drops, or otherwise spirals out of control), understand why, and fix the problem.
    2. Measure machine resources. In cases where you cannot redline the component, or you have reason to believe it behaves substantially different in less-than-100-percent-load situations, you may need to resort to OS metrics such as CPU utilization and IOPS to determine whether you’ve made a change. Make sure your load is large enough for changes to be visible. If your load causes 3 percent CPU utilization, a 50 percent improvement in performance will be lost in the noise. Try different amounts of load and find a sweet spot, where your OS metric measurement is sensitive enough.

    As you execute your benchmarks and develop a better understanding of the system, you are likely to discover new factors that may impact your key metric. Add new experiments to your list and prioritize them over the previous ones if needed.

    In some instances, the code may not have configuration or control knobs for the inputs you want to vary. Find the fastest way to change the input, even if it means hacking the code, commenting out sections or otherwise manipulating the code in ways that wouldn’t be “kosher” for merges into master. Remember: The goal here is to get answers as quickly as possible, not to write production-quality code—that comes later, once we have our answers.

    Once you’ve completed a series of benchmarks, take a step back and think about what the data is telling you about the system you’re benchmarking. Document your insights and how the data backs them up. It may be helpful to:

    • Calculate the average for each series of benchmarks you ran and to use that to calculate the difference (in percent) between series — i.e. “when I doubled the number of threads, QPS increased by 23 percent on average.”
    • Graph your results — is the relationship between your input and the performance metric linear? Logarithmic? Bell curve?

    Finally, you’ll need to present your insights. Here are some final tips for exhibiting to management and/or other engineering teams:

    • Apply the Pyramid Principle. Engineers often make the mistake of explaining methodology, results and concluding with the insights. It is preferable to reverse the order and start with the insight. Then, if needed/requested, explain methodology and how the data supports your insight.
    • Omit nitty-gritty details of any experiments that didn’t lead to interesting insights.
    • Avoid jargon, and if you cannot, explain it. Don’t assume your audience knows the jargon.
    • Make sure your graphs have meaningful, human-readable units.
    • Make sure your graphs can be read when projected onto a screen or TV.

    — Stefan Zier