devopsy.kinsta.cloud

Tag: measuring performance

  • An Analytics Workload is a Critical Data Management Tool

    An Analytics Workload is a Critical Data Management Tool

    From an organizational perspective, an analytics workload is a way to gain a data-driven business advantage. Analytics workloads can serve a primary objective that translates to an actionable business impact (such as analyzing A/B testing results, understanding user retention or finding anomalies in a flow, for instance) or can serve in a supporting role for analysis (such as improving dataset quality, preparing a dataset for user consumption or adhering to a data-related regulation).

    Ideally, a workload should have a measurable ROI and should be monitored on key performance metrics such as business impact, time to deliver, cost of execution and total spend versus allocated budget. Managing workloads this way is extremely challenging for data teams. In reality, the impact of big data analytics projects – and the effort and resources spent on them – tends to be unpredictable.

    You can place workloads on a spectrum: on one end, you have workloads that are very close to their business consumers. On the other end, you have workloads that serve auxiliary and support purposes. The latter are the hardest to track, budget and measure their return on investment (ROI).

    Query vs. Workload

    When data platform teams are tasked with evaluating and implementing the infrastructure needed for an analytics workload, they tend to look at it from a technical requirements perspective. An analytics workload is a set of queries that read and write datasets; it consumes compute, storage and memory resources and has an expected level of availability, robustness and performance.

    One traditional approach for improving cost and user experience in a workload is to focus on specific queries – identifying the slow, heavy and expensive ones and optimizing them one by one. While many single queries can be made better (faster, cheaper) by rewriting and optimizing, this approach often misses the big picture, leaving money on the table.

    Knowing Which Queries to Focus On

    Stripped of workload context, queries can only be evaluated on cost or response time. But business-critical queries may not be the most expensive, nor the slowest ones. Having the full view of the workload allows for prioritizing efforts much more effectively.

    Behavior Fluctuations

    The most important query on Mondays is probably very different from the most important query on Fridays, so the effectiveness of optimizing one or both queries may be small. The workload perspective enables you to identify patterns and leverage insights for effective optimization.

    Cloud Elasticity

    Unlike legacy data platforms, the cloud enables a brute-force approach to solving performance problems by adding elastic resources on demand. While this approach can work to improve a specific query, cost quickly gets out of control when applied to all queries. Understanding the workload is key to distinguishing when a brute force solution is right, and when you need to spend the time to build a different data flow.

    Cloud Decoupling

    In legacy IT environments, where queries were competing for a limited pool of resources, optimizing a single query had significant impact. But in cloud environments where resources scale elastically, queries compete with each other on a much smaller scale, if at all. Therefore, optimizing single queries has very little impact on the overall performance of workloads.

    Measuring Performance by Looking at the Median

    The heaviest or slowest queries are in the top percentile of queries. When trying to improve the overall user experience, the median performance can matter more, but it is indirectly affected when handling the extremes, so it tends to be less effective.

    The Long Tail

    In many workloads, a few highly visible queries can paint a misleading picture. It is common to have less than ten queries responsible for 20% of the load, and thousands of undistinguished “simple” queries responsible for 80%. Therefore, when data teams are trying to reduce cost, focusing on the most expensive queries may draw attention away from lower-hanging and easier optimization opportunities in “simple” queries.

    “Big Picture” Optimization

    Actions that improve a query (such as changing the layout of a dataset) may result in unintended consequences to other queries. Only a workload perspective can help evaluate the wider impact across other queries.

    The Cluster Approach to Optimizing Big Data Analytics

    Another traditional approach to optimizing data platforms is analyzing behavior from the cluster perspective. Most data platforms offer decent cluster monitoring of technical metrics, such as CPU load and RAM usage.

    But, focusing on cluster-based KPIs will draw attention away from the elements that have the highest impact. Understanding the workload entails understanding the business logic of queries: repeating query patterns, hidden dependencies, failure modes.

    Looking at this from a cluster perspective limits you to technical parameters, such as CPU usage. Without understanding the workload, data teams will miss issues such as a repeating calculation (to be made into an ETL flow) or an inconsistent calculation flow (like having different ways to compute the same business KPI).

    In environments where clusters are shared between different workloads, data teams often encounter additional challenges:

    Workloads Have a Business Context, Clusters Don’t

    From a cluster perspective, there is no inherent prioritization between queries, and data teams find it difficult to focus only on business-critical workloads. The outcome is often reverting to focusing on specific queries, which doesn’t offer an effective solution.

    Workloads Have Different Needs

    Take, for example, an ETL workload which may require a large amount of RAM, run hours and which must be finished by 7 am. An interactive data product workload may be composed of thousands of queries that each must return in under 2 seconds. Analyzing the cluster as one unit can lead to suboptimal behaviour of both workloads. While this example can be considered simple to detect, many workloads that seem similar can have widely divergent needs.

    As data lakes gain momentum as production-grade, big data analytics architecture, data teams need to shift attention from optimizing queries to measuring, analyzing and optimizing workloads. In the early days of big data, data warehouses were isolated data platforms tasked to deliver insights for a very specific set of business questions. But as the cloud rapidly evolved, the mere migration of data warehouses to the cloud is not enough. The next stage is moving production workloads to the data lake.

    Storage is so much cheaper, and organizations want to run queries on any dataset, whenever they need it. It’s this heterogeneous collection of many different queries that requires workload-level visibility to ensure each workload can truly meet performance and budget requirements.

  • Four Ways to Modernize Processes and Measure Performance With APM

    Four Ways to Modernize Processes and Measure Performance With APM

    Change is everywhere and the stakes are high. For 2019, businesses are on track to spend over a trillion dollars on digital modernization, according to a recent study from IDC.

    A crucial part of modernization is the growing number of IT organizations using the latest in dynamic, cloud-scale technologies to develop applications. As a result, technology professionals in the businesses you serve must now work with a much more complex and dynamically changing tech stack.

    When building and maintaining an app, it’s becoming even more common to have hundreds (or thousands) of pieces of code in containers being continually reconfigured and brought in and out of production. And many of these operations take place in customer-facing applications.

    The stakes are high, because if something goes wrong operations could halt. It’s therefore vital to understand the underlying technologies throughout an organization’s IT environment to better equip customers with the tools needed to be successful. Application performance management (APM) tools that simplify stack management, whether it’s on-premises, hybrid or fully cloud-native, provide a level of visibility needed for modern businesses—and they work across multiple cloud environments, such as AWS or Azure.

    As a channel partner, you need to be prepared to help your businesses and their customers through this modernization process. 

    Try the following to help your customers modernize business processes.

    Help Them Realize Software Is Now Business-Critical

    As IT departments move from on-prem infrastructure and a mix of commercial off-the-shelf (COTS) and monolithic custom applications to increasingly adopted cloud-based Kubernetes microservices applications, they’re accelerating the adoption of software at the core of all their business processes. The result? Business is increasingly dynamic and responsive—new applications are built and delivered to market faster than ever. But companies are also at risk now when things don’t go well—that strategic app could degrade or fail.

    All the tech layers underpinning these applications are also becoming more sophisticated, dynamic and interwoven—and making it still harder to administer as they continuously change at an even faster rate.

    Focus Your Customers on Core Competency

    There are plenty of open-source tools out there for your customers to use to build their own monitoring. The biggest obstacle for businesses rolling out their own apps while using an open-source tool is core competency. 

    If, for example, your customer is a wholesale plumbing supply company, is their competency sourcing plumbing supplies or knowing how to write instrumentation code for all their plumbing supply chain applications? Help them focus on what they’re good at and give them tools to manage the rest.

    Perhaps you have a large financial services customer who is spending tens of millions, across multiple years, on a team of outsourced experts to write precisely the code required. This is an extremely time-consuming and costly effort, but financial institutions do this because their needs are highly specific and mission-critical, and the cost of not doing it is even higher. 

    If you look at real-time high-frequency trading environments—where billions of dollars are moved in matters of tenths of a second—you come to understand the stakes justifying these herculean efforts. All of which supports the need for APM tools to help empower the tech pros in-house who monitor and manage these high-speed trading environments.

    At the other end of the spectrum are the small tech companies with a new idea they’re out to promote. But when it comes to most companies, their core competency is often not monitoring and managing their infrastructure and applications. It’s understanding the market requirements for their custom apps and getting those apps to market quickly. At the same time, organizations of all shapes and sizes are still confronting the reality that to successfully deliver modern applications we must shift from pure infrastructure performance metrics to the KPIs that best demonstrate end-user experience.

    Ultimately, many of the organizations described above have either focused on IT as a core competency or, in the absence of dedicated tech teams (at SMBs, in particular), they have outsourced management of their complex, dynamic tech stack and custom application builds. In both instances, APM tools are a necessity and it’s prime time for the channel to deliver those services.

    Be the Most Viable Modernization Partner

    By understanding the needs of modern organizations as they grapple with their complex tech stacks, you can ensure you have the right set of tools for your customers. 

    Provide full-stack capabilities for today’s businesses to manage their applications and keep operations running smoothly. Cater to unique business problems and equip your customers with APM tools based on their needs—advanced logging, trace metrics, performance optimization and full-stack visibility across applications and the compute infrastructure on-premises, hybrid and cloud. Help them check all their functional boxes.  

    Measure Performance with Full-Stack APM

    In all cases, your customers should be measuring performance. Because every company uses a mixture of digital technologies, arming them with full-stack APM will help gain visibility into performance throughout the business. With APM, businesses can see if their system is degrading—and fix it before disaster strikes. That’s the business value for APM—it gives visibility into the complexity underpinning today’s application environments. Full-stack APM provides this visibility, whether the apps are cloud-based, on-premises or hybrid.

    But here’s the reality—not all APM tools are the same. Traditional APM solutions weren’t built for the environment IT departments now have, with the addition of containers and microservices-based applications to their existing monolithic and n-tier/SOA applications. Those solutions were made to manage hundreds or thousands of objects with slower rates of change, not the tens of thousands, or even hundreds of thousands to millions of objects IT works with today.

    Also, note the ephemeralness of container-based microservice applications able to live and die in a matter of seconds or less. There’s no time to automatically install a traditional agent, let alone do so manually. The container’s life span is shorter than the time it would take to do such instrumentation. Direct them instead toward a full-stack APM solution with complete coverage of all application architectures: monolithic, n-tier/SOA and microservices based. With anything less, they’re running at a higher risk of brand damage and revenue impact.

    By being the experts on APM, providing full-stack visibility and measuring performance, your customers will find modernization success.

    — David Wagner