devopsy.kinsta.cloud

Category: Cloud Management

  • How Observability Can Improve Gaming

    How Observability Can Improve Gaming

    Competing in the gaming industry is becoming increasingly costly. To attract new players, millions of dollars are spent in user acquisition (UA), development and cloud infrastructure. As the space becomes more expensive, revenue and cost optimizations can become a matter of survival. 

    The cost to businesses because of server outages, game response lag or generally poor performance can equate to billions of market cap of gaming companies. The value of observability in modern games is about ensuring that revenue opportunity is not missed in a costly operating landscape. Observability assures that all investment spent on user acquisition is protected and that gaming user experience (UX) is maximized. And user experience depends not only on how the game’s UI is performing but also on the entire gaming environment performance and reliability.

    A modern gaming environment is usually highly distributed—a lot of pieces have to come together. The engineering team develops code for the gaming application, which can consist of many microservices, containers, managed cloud services, third-party APIs and serverless functions. Game designers create game levels, the art team produces visual effects and animations and the audio department creates game audio. Once these moving parts are packaged together, the game service has to be deployed on cloud infrastructure and the game itself shipped as executables for consoles, mobile devices and laptops.

    gaming observability

    All these elements generate massive amounts of observability data, particularly if a game is popular and played by millions of users. Thus, making sure that all gaming elements are working as expected means that gaming companies have to implement efficient ways of collecting and analyzing the massive amounts of that data. That data comes from gaming services, consoles and other sources in real-time. For instance, it is not uncommon to deal with hundreds of terabytes of ingested log data per day. As a result, improving the gaming experience can be quite challenging for gaming companies with large-scale operations. The companies’ success depends on how efficiently they can harness all the data generated by gaming environments.

    This is where observability comes in. Observability provides critical insights into what is—or what may be—wrong with the game, all its moving parts and user experience. Powerful observability capabilities are essential, including processing vast amounts of data (logs, metrics, traces, etc.) in real-time, managing fluctuating observability data from variable user loads and running concurrent real-time queries on a massive scale.

    So, what, exactly, needs to be measured for a great gaming user experience?

    Measuring the Gaming User Experience

    Gaming companies are under continuous financial pressure to deliver new features that attract and retain new users in order to offset mounting operating expenses. Attracting new users is a balancing act from the gaming companies’ perspective. If they move too slowly, they will lose gamers. If they move too quickly, the gaming user experience can deteriorate due to newly introduced bugs. Gamers can be very vocal on social networks, especially if their intent is to express their dissatisfaction with compromised gaming quality. As a result, companies are forced to balance the rate of feature introduction with user satisfaction. That’s why they need to measure user experience continuously.

    To improve gaming user experience, gaming organizations must establish measurable key performance indicators (KPIs), which can drive improvements and overall quality. KPIs include, but are not limited to: 

    • User video quality (frames per second)
    • Memory usage
    • Crash reports count
    • Database performance measurements
    • Service latencies 
    • User request count
    • Game-specific errors and warnings count
    • Code quality measurements, such as automated test coverage success criteria (a part of the CI/CD pipeline)

    Monitoring Service Latency for a High-Quality Gaming Experience

    Gaming applications are often bursty when it comes to user load, especially during the introduction of new games or new features. One of the critical indicators to monitor is service latency to manage fluctuating user demand during those times. 

    DevOps teams and SREs need to collect logs at a massive scale to analyze the performance of services, infrastructure and individual host utilization metrics such as high CPU utilization. This information gives engineers clues about the sources of bottlenecks and potential deterioration related to service latency. And logging tools need to be able to deal with this fluctuating demand, too. This level of insight into the state of the gaming application helps engineers troubleshoot issues before gamers face increased service latency that can impact their gaming experience.

    Analyzing Game-Specific Observability Data

    Game developers can create custom logs and metrics and combine those with observability data from the gaming engines. Both must be analyzed to assure a high-quality user experience. Some of the common gaming data parameters available for analysis are:

    • Log data that indicates when a player joins and leaves the game (associated with the player’s name)
    • The location of a player reported at specified times
    • Logs related to significant gaming events associated with each player (such as goals achieved or actions performed)
    • Logs related to chat messages between players, player location, the time when the message was generated and message context

    The more data collected, the better the chances to understand any issues that can turn players away from the game. Observability data management should evolve in parallel with game development. As new features are added, the code should be instrumented with KPIs to ensure the quality of the newly introduced code in the long run.

    Database Performance Monitoring is Critical for Gaming User Experience

    To store time-sensitive gaming data, DevOps teams must use fast databases. The database performance and how fast the game’s properties are retrieved from a database directly impact user satisfaction. Slow database performance negatively affects the gaming user experience. For DevOps teams to avoid any service issues, it is a must to measure key database performance indicators such as response times metrics, out-of-memory error logs and others.

    Fast databases could be expensive in the extremely competitive market where gaming companies are always looking for cost optimization to survive in the business. It’s non-trivial for DevOps teams to monitor user experience related to database performance and, at the same time, to look for ways to optimize the cost of fast databases.

    Build a Pipeline and Delivery Monitoring

    For a large-scale video game, new code is often pushed many times per day. Builds can be significant and deployed across many gaming consoles and end devices. Continuous build pipeline and delivery monitoring are crucial for frequent and commonly large software deployments.

    With large builds deployed daily over the network to thousands of gaming platforms—where each platform receives a massive amount of data—monitoring of build deployments should be automated. Errors from log data should trigger alerts and drive redeployments. QA automation is necessary to reduce the burden of QA efforts and make build deliveries less error-prone and less tedious. 

    Better Observability Improves In-Game Commerce

    Insights into user behavior are as important as monitoring possible issues affecting user experience. Observability tools can provide key insights into the overall engagement of all game users. High engagement then drives more excitement. Higher user excitement may increase the success of ad campaigns and accelerate in-game purchases.

    Observability Tools for Managing Gaming Experience

    When thinking about observability tools that can be instrumental in driving continuous improvement of user experience in the gaming industry, the following functionalities are important:

    • Ability to ingest and analyze log data at petabyte-scale and beyond in real-time to avoid gaming experience deterioration.
    • Zero-schema and ability to accept observability data in various formats from various gaming platforms, cloud servers, gaming consoles, and mobile devices. Because gaming environments use different third-party software packages, logs may come in many formats.
    • Ability to create alerts on collected observability data. Alerting can improve user experience by monitoring properties such as the number of active users experiencing client errors. 
    • Capability to handle backpressure when critical issues occur is essential. An alert storm can clog the data pipes, and the observability tool can be in the dark for hours unless it is designed to cope with backpressure.
    • Ability to collect data from build pipelines to monitor frequent and commonly large software builds being deployed daily over the network to numerous gaming platforms.
    • Capability to collect data from cloud environments such as cloud infrastructure monitoring. 

    Observability is key to protecting gaming revenue opportunities and assuring the success of video games, as long as you know what to look for in observability tools that are used to gather data from gaming environments.

  • A Closer Look at Azure DevOps

    A Closer Look at Azure DevOps

    Microsoft’s Azure DevOps Server is emerging as one of the most popular cloud application development environments. Like most continuous integration/continuous delivery (CI/CD) platforms, it provides features such as version control, reporting, project management, automated builds, and testing. It covers all aspects of the application life cycle and enables DevOps. Here’s a closer look at this cloud computing environment.

    Azure DevOps provides a number of different developer services that allow DevOps teams to plan work, collaborate on code development and build and deploy applications. It is highly collaborative and brings together developers, project managers and individual contributors to develop software. It allows organizations to create and improve products much faster than with traditional software development approaches.

    Services or Server?

    Teams can work in the cloud using Azure DevOps Services or on-premises using Azure DevOps Server.

    The cloud offering, Azure DevOps Services, is a scalable, reliable and globally available hosted service. It’s backed by a 99.9% SLA, is monitored by Microsoft’s 24/7 operations team and is available in local data centers around the world.

    The on-premises offering, Azure DevOps Server, is built on a SQL Server back end. The on-premises version is preferable when organizations need their data to stay within their network or when they want access to SQL Server reporting services that integrate with Azure DevOps Server data and tools.

    Although both offerings provide the same essential services, Azure DevOps Services offers some added benefits:

    • Simplified server management.
    • Immediate access to the new features and updates.
    • Improved connectivity with remote sites.
    • A transition from capital expenditures (servers, etc.) to operational expenditures (subscriptions).

    There are also some other key differences depending on your organization’s or your team’s needs. These can also be helpful if you’re migrating from on-premises to the cloud:

    Differences in Specific Feature Areas

    Although Azure DevOps Services is a hosted version of Azure DevOps Server, there are some feature differences. Some Server features aren’t supported in Services; for example, Azure DevOps Services doesn’t integrate with SQL Server Analysis Services to support reporting. Process customization and reporting also differ in their support.

    Integrated Features

    Azure DevOps provides integrated features that can be accessed through a web browser or an IDE client. One or more of the following standalone services can be used based on your organization’s business needs:

    • Azure Repos: Azure repos provides Git repositories or Team Foundation Version Control (TFVC) for source control.
    • Azure Pipelines: This service provides build and release services to support continuous integration and continuous delivery of applications.
    • Azure Boards: This suite of Agile tools supports planning and tracking work, code defects and issues using Kanban and Scrum methods.
    • Azure Test Plans: Azure Test Plans provides several tools to test applications, including manual/exploratory testing and continuous testing.
    • Azure Artifacts: These allow teams to share packages such as Maven, npm, NuGet and more from both public and private sources and integrate package sharing into pipelines.

    Use of the following collaboration tools is also supported:

    • Customizable team dashboards with configurable widgets to share information, progress and trends.
    • Built-in wikis for sharing information.
    • Configurable notifications.

    Azure DevOps supports adding extensions and integrating with other popular services, such as Campfire, Slack, Trello, UserVoice and more. In addition, DevOps teams can also develop their own custom extensions.

    Azure DevOps Services supports integration with GitHub and GitHub Enterprise Server code repositories. Azure DevOps Server supports integration with GitHub Enterprise Server repositories.

    Getting Started

    When you sign up for Azure DevOps, you must first register an organization, project and version control type. DevOps teams can create multiple projects and differentiate them with separate names and access permissions. Projects can be set to public or private. Teams can also use Azure Boards for collaboration and project management, and can also configure use of Agile practices. After setting up your team, you can use the platform to begin deploying your applications to Azure.

    In addition to a collaborative environment, Azure DevOps provides access to an Azure Pipeline service that supports multiple languages and platforms. The Azure DevOps pipeline is a set of services that are grouped into individual projects. 

    Additionally, Azure Test Plans feature facilitates automated testing. It is also possible to set up deployment pipelines with different components for multiple applications. 

    Azure Pipeline can track work status, manage backlogs and create custom reports. With the help of the Azure DevOps pipeline, developers can create, manage and deploy their projects in a single environment. With this solution, DevOps teams will have a single point of contact and can communicate with each other.

  • Log Management Challenges Grow With Observability Adoption

    Log Management Challenges Grow With Observability Adoption

    It appears that, in many IT environments, observability is becoming too much of a good thing. A global survey of more than 315 IT executives, cloud application architects, DevOps professionals and site reliability engineers (SRE) conducted by Dimensional Research on behalf of log data management platform provider Era Software, found 96% are witnessing an explosion of log data.

    The survey also found 79% of respondents reported the overall cost of observability and log management will skyrocket in 2022 if current tools don’t evolve.

    More than three-quarters of respondents (78%) also noted attempts to manage volumes of log data have had mixed or unwanted results such as an inability to access data. Nearly two-thirds (65%) are also evaluating their observability options while the other 41% are considering it.

    Stela Udovicic, senior vice president of marketing for Era Software, said the survey makes it clear that as organizations rely more on logs to analyze IT events, the cost of storing all that log data is becoming a significant challenge. Overall, the survey finds usage of observability tools and platforms has jumped by 180% as IT teams struggle to manage increasingly complex IT environments.

    On the plus side, more business users are benefiting from log data. A full 83% of respondents report that business stakeholders outside of IT use insights from log data, with 96% reporting log data is being used to solve business problems.

    Observability has always been a core tenet of DevOps best practices, but achieving it has always been a challenge. Monitoring tools are designed to consume predefined metrics to identify when a specific platform or application is performing within expectations. The metrics tracked generally focus on, for example, resource utilization. However, whenever there is an issue, it can still take days, sometimes weeks, to discover the root cause of an issue through what amounts to a process of elimination.

    In contrast, observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshot issues without relying solely on a limited set of metrics that have been predefined to monitor a specific process or function. DevOps teams can employ queries to interrogate data in a way that makes it easier to discover the root cause of an issue. Observability platforms correlate events in a way that makes it easier for analytics tools to identify anomalous behavior indicative of an IT issue.

    The issue that arises is that IT teams need to find a way to store all the data generated by the logs they need to analyze. Each IT team can, of course, could reduce the amount of log data they retain. However, there is always a concern that in the event of an incident critical log data won’t be available.

    Udovicic said most IT organizations have historically viewed storing log data as a necessary evil. The primary issue most of them encounter is the simple fact that log data is not easy to work with, she noted.

    Regardless of how organizations view log data, the amount of it that needs to be managed will only increase as DevOps workflows become more mature. The issue IT teams need to come to terms with now is finding a way to minimize the cost of storing it all.

  • Managing Endpoint Compliance-as-Code

    Managing Endpoint Compliance-as-Code

    The rapid pace of modern software delivery and increasing fleet sizes have transformed how organizations see and handle security strategies. Compliance-as-code is required for today’s organizations that need security as a fundamental part of business processes. It is no longer possible to manually manage compliance with dedicated security teams.

    Balancing security with growing infrastructure needs means IT security and compliance are non-negotiable. There is no room for more uncomfortable trade-offs between risk and an organization’s ability to deliver market-ready solutions quickly and efficiently.

    Complying with growing security and compliance regulations in today’s world of rapid innovation is a constant challenge that affects all organizations, large and small. Organizations are implementing automated solutions to eliminate reliance on traditional, slow, error-prone manual processes because of tighter regulations in the industry and greater risks associated with security attacks and compliance violations.

    What is Managing Endpoint Compliance-as-Code?

    Compliance-as-code is the codification of compliance controls to automate their adherence, application and remediation. It includes the tools and practices that enable DevOps and developer teams to incorporate the three key compliance activities:

    1. Detect: Discovering non-compliance through automated estate scanning and notifying stakeholders when offending infrastructure is discovered.
    2. Remediate: Correcting non-compliance by implementing immediate infrastructure changes to ensure the highest level of compliance at scale.
    3. Automate: Avoid non-compliance by automatically verifying that planned changes comply.

    There are many important use cases for managing the compliance of endpoint state as-code including confidential data protection, detecting shadow IT resources, network security, data exposed to public access and code licensing compliance.

    The Importance of Compliance-As-Code

    Most organizations still struggle to stay secure and comply with regulatory standards. Many organizations lack visibility across heterogeneous infrastructure and applications. They also possess inconsistent language for communicating requirements between development, security and operations (dev, sec and ops) teams and are unable to remediate findings.

    Compliance-as-code enforces a comprehensive compliance strategy that involves a robust set of controls; for example, managing data storage locations and access control management.

    Ensuring that these are followed at scale is critical to the success of a digital business as more organizations shift to the cloud, there are more possibilities of non-compliance. Reports suggest that a regulatory compliance violation can cost businesses $15 million on average.

    This violation is avoidable if DevOps, development and DevSecOps teams can automate compliance by adopting a different mindset and writing understandable code. The adoption of the “as-code” strategy led to infrastructure-as-code (IaC) which is one of the success factors in DevOps teams. Audits are streamlined and allow individuals to focus on higher-value activities.

    The Challenges of Managing a Fleet Through Compliance-as-Code

    Managing a fleet through compliance-as-code is not without its challenges. First, teams must bridge the talent gap as endpoint compliance is typically managed by IT admins, who are not inherently comfortable with an as-code approach.

    Typically, compliance requirements are dense and difficult to comprehend by other stakeholders. Understanding the compliance requirements of the organization and collaborating with other stakeholders to convert them to as-code is the biggest hurdle to getting started.

    Developer teams must find the right set of tools and integrate them into DevOps and DevSecOps workflows. This is not just a process change but a culture change. A boil the ocean approach will not just be painful but success will be close to impossible. Taking small steps and continuously improving is the mindset one has to adopt.

    Visibility Throughout the Ecosystem

    Whether you are talking about end-user devices like desktops, laptops or mobile devices or non-server edge devices like point of sale (POS) or IoT devices, endpoint devices are extremely vulnerable to exploitation from bad actors. Thus, it is necessary for organizations to bring them under the organization’s security policy and governance requirements just like server-side devices.

    There are a number of advantages to adopting an endpoint compliance-as-code approach:

    • Users can easily understand what is going on across the organization in terms of compliance status at any time.
    • The organization can keep an audit log of everything that occurred throughout the development life cycle and calculate an appropriate risk acknowledgment rating.
    • Organizations can evaluate any deviations from fundamental compliance requirements.

    The Continued Prioritization of Compliance-as-Code

    Managing diverse IT infrastructure fleets through compliance-as-code ensures that endpoint devices meet a variety of security standards. The IT team is empowered to create rules that can enforce compliance and establish security baseline standards within the organization. Compliance checks are validated and more visible at each stage of the software development life cycle.

    This guarantees detailed visibility and thorough compliance standards throughout the IT ecosystem.

  • Google Focuses on Enterprise Cloud With Mandiant Acquisition

    Google Focuses on Enterprise Cloud With Mandiant Acquisition

    Thomas Kurian became the CEO of Google Cloud Platform (GCP) with a modest mandate: Turn Google Cloud into an enterprise cloud platform behemoth. Easy, right? By 2019 “Google” was already a verb because of its search engine dominance and the company’s revenue was north of $136.8 billion. So, why have I been closely tracking Google Cloud’s progress on winning enterprise cloud business? Let’s back up.

    When Kurian left Oracle and joined Google, the company was an extremely successful advertising company. At the time, over 85% of Google’s revenue came from advertising. The search engine, email service and lightweight office productivity apps all served the ultimate goal of keeping eyeballs on Google sites so that users could be served more advertisements and increasingly more personalized ads based on a user’s Google interactions.

    If You Build it, Will They Come?

    Enterprises Require More Than Just Technology From Their Cloud Providers

    Google clearly has the scale, tools and technical expertise to provide businesses with what they need from a cloud vendor. When Kurian left Oracle, he was one of the highest-paid technology executives in the world (according to Oracle, his 2017 compensation package was around $36 million). So why would Google hire Thomas Kurian? Because enterprises require their technology and services partners to have more than just great tech. When advising enterprise clients, the following are the top three topics that come up when discussing the selection of a cloud vendor:

    • Industry Expertise: For the enterprise, understanding industry nuances is critical. I recently talked to an executive at a large airline company that fired their large, publicly-traded technology services company because the partners and the services company didn’t truly understand the airline industry. They weren’t using the right terminology and didn’t understand the business pain points.
    • Predictability: Why have large enterprises been hesitant to go all-in on the cloud? They need predictable technology. That doesn’t just mean “stable;” it means that costs, security, usability and service levels must all be predictable. Why, for example, is the mainframe still critically important for many enterprises? Because it is predictable and secure.
    • Ecosystem: Clouds can’t live in isolation. Cloud vendors must have an ecosystem to support enterprise client needs. My number two cloud prediction for 2022 is that we will see an increase in industry-specific ecosystems. Amazon Web Services (AWS), Microsoft Azure and IBM Cloud all have cloud ecosystem strategies focused on financial services.

    Thomas Kurian is well aware of the need to play well with other cloud providers and enterprise cloud providers. In fact, the widely held belief is that Kurian left Oracle because he and Oracle CEO, Larry Ellison disagreed on Oracle’s multi-cloud strategy—Kurian wanted Oracle’s packaged applications to be available on other vendors’ cloud platforms.

    What Does the Mandiant Acquisition Mean for Google?

    It’s clear that Google Cloud is focused on the enterprise. Google has no shortage of DevOps, data, machine learning and artificial intelligence tools to satisfy developers and data scientists. However, what Google needs are more features that enterprises expect from a cloud partner.

    Google Cloud isn’t new to cybersecurity—the company has been securing its own cloud since its foundation. In addition, the company has been gaining enterprise traction with Google Workspace and Gmail and the security around these SaaS offerings is paramount.

    By acquiring Mandiant, Google Cloud can more closely partner with cloud customers throughout the entire life cycle: From code creation (through a combination of Google’s existing Google Chronicle tools) through testing, threat detection and advisory work around incident response.

    My Analyst View

    The debate around security and the cloud isn’t even close to being resolved. Many will tell you that cloud vendors invest millions upon millions of dollars more than any other company in the cloud. It’s imperative that cloud vendors don’t experience infrastructure or SaaS security breaches. However, security continues to be a roadblock for cloud adoption—and, most importantly, enterprise cloud adoption. While every enterprise is doing some work in the cloud, many are still reluctant to move critical workloads (especially their sensitive data) to the cloud.

    Google Cloud should use Mandiant as the bedrock on which its enterprise cloud security business is built. Rather than having piecemeal parts and partners, businesses want their cloud providers to have opinionated frameworks and best practices. Mandiant has some of the best security minds in the industry and can help Google Cloud customers create secure instances. I expect Google Cloud to provide ready-to-provision cloud environments tailored for specific regulated industries by leveraging Mandiant’s expertise. Of course, customers aren’t going to want to necessarily adopt everything that Google Cloud suggests, and customers should be able to swap out components. However, many businesses are already overwhelmed with the number of cloud partners they have to engage with to create a secure cloud environment.

    What Will Microsoft and AWS Do?

    I expect AWS and Microsoft Azure will focus their attention on building out enterprise-focused security offerings. I recently wrote about why it made sense for Microsoft to purchase Mandiant. My general feeling was that Microsoft had many security offerings—ranging from simple consumer-grade virus scanning to enterprise offerings—but that the company lacked a cohesive enterprise security strategy. I still believe Microsoft needs to clearly define its enterprise security assets (and more coherently group them together) either through a reorganization and/or an acquisition. Like Microsoft, AWS has hundreds if not thousands of security offerings and partners. Although AWS is the cloud leader, the company is well behind Microsoft and Google in defining its cloud security strategy. The company needs to outline its enterprise cloud strategy, and cybersecurity needs to be one of the linchpins of that enterprise strategy.

  • Multi-Cloud Computing is a Major IT Headache

    Multi-Cloud Computing is a Major IT Headache

    A CloudBolt Software survey of 500 senior IT, DevOps, security and finance leaders found 79% are struggling under the weight of managing multiple cloud computing environments.

    The survey from CloudBolt Software, a provider of a cloud management platform, found specifically that more than three-quarters of respondents (78%) said they have too many cloud tools throughout their enterprise, and 80% struggled to achieve comprehensive visibility into cloud usage and spending. A total of 80% also noted that their current approach to cloud computing created new security vulnerabilities.

    A full 88% said they need a “manager of managers” that would enable them to unify the various silos that have multiplied across their IT environment.

    Nevertheless, 78% of respondents said that moving to the cloud enabled them to achieve cost savings compared to running applications in an on-premises IT environment.

    CloudBolt Software CEO Jeff Kukowski said the issue is not so much that IT organizations regret shifting workloads to the cloud; instead, the issue is finding a way to manage multiple clouds, each with their own set of tools and frameworks that need to be mastered.

    It’s not clear whether organizations are deliberately employing multiple clouds or if, because of simple inertia, they have found themselves managing multiple cloud platforms because of decisions made by various development teams. Some IT organizations, for example, employ multiple clouds because they are concerned about either becoming locked into a specific cloud computing service or whether an outage might take all their applications offline at the same time. Others are simply trying to take advantage of services that are better provided by one vendor than another.

    Regardless of how IT organizations found themselves managing multiple clouds, the total cost of IT rises as each new platform is added to an IT environment. CloudBolt Software is making a case for an IT framework that provides a layer of abstraction through which the management of multiple clouds and on-premises IT environments can be unified. It is more a question of time and degree to which IT organizations will eventually consolidate the management of disparate IT environments under a single control plane that provides the foundation for true hybrid cloud computing, said Kukowski.

    There are, of course, a few organizations that have standardized on a single cloud computing platform. The degree to which those organizations can maintain that standard will naturally vary as, for example, new workloads that may be optimized for a specific cloud environment are built or as companies acquire other organizations that are running workloads on other cloud platforms. In many of those cases, organizations discover that the cloud services invoked expose a range of proprietary application programming interfaces (APIs) and disparate virtual machines that make migrating workloads a major challenge.

    Like it or not, most organizations will be managing multiple IT environments for the foreseeable future. The only thing that is uncertain is the volume of workloads that might be running on any one of those platforms at any given time.

  • How Stream Processing Goes Beyond Real-Time

    How Stream Processing Goes Beyond Real-Time

    How fast is real-time? According to Robert Miller’s 1968 seminal study on human-computer interaction, we perceive a response time of 100 milliseconds or less as instantaneous. This is much quicker than the average human reaction time, which one online test clocks at 273 milliseconds.

    Nowadays, customer-facing applications need to respond in as close to real-time as possible to win our attention and meet standard requirements. For example, 100 milliseconds is a golden rule for credit card processors to meet SLAs. Especially in our new remote-first economy, other areas like media, health care, retail and manufacturing need to take full advantage of real-time data processing to enhance digital business and stay relevant. Yet, leveling up with real-time requires some technical nuances that might be challenging to implement with the current developer skills gap.

    According to Manish Devgan, chief product officer at Hazelcast, the problem with most data management techniques is that data isn’t being processed in-stream at the edge. Instead, data gets stored and thrown into a data lake where it accumulates, unfulfilled. To make matters more complicated, decentralized data management means organizations may be overseeing many different data lakes, adding to the potential for unused data and increased latency.

    I recently met with Devgan to learn how in-memory stream processing goes beyond real-time data. According to Devgan, the secret sauce is processing incoming data in-stream while tying it to persistent storage to create context. By doing so, companies could unlock more innovative real-time experiences, he said.

    Use Cases For Real-Time

    Real-time presents many opportunities for business operations and end-user customer engagements. This is especially true in the financial sector. For example, Devgan explains how one bank was able to use real-time data at the moment of customer interaction at their ATMs. The system was able to fetch a history of the user account and extend an offer for a low line of credit, increasing their total loans by 400%. Capturing value at the moment of user intent is a golden opportunity, said Devgan.

    Capital One is another example of a financial institution leveraging advanced technology to improve the product experience. “They’re one of the leading banks in terms of technology,” said Devgan. Capital One’s threat detection system calls a machine learning model in real-time to detect and prevent fraud. Only by acting on incoming data in the moment of interaction can preventative responses like these be signaled.

    Other scenarios are also beginning to incorporate increased real-time analysis. “This split-second preemptive decision-making is important for many other use cases,” said Devgan. Physical retail might loop into customer loyalty programs at point-of-sale systems to increase in-store spending. Or, automobile manufacturers may want to use real-time processing to detect a problem with their robotic arms sooner rather than later.

    Real-time vs. Stream Processing

    When an event comes in, it usually has very little information. But when you look up a customer profile and connect it to a greater context, you can direct further actions. “Correlating data in motion with data at rest is where the magic happens,” Devgan. This distinction uncovers the difference between real-time data and stream processing.

    The real-time label tends to get thrown around a lot, and it’s quantified differently for different groups. Real-time might be five minutes for some applications but 10 milliseconds for others. Real-time could simply mean that a system accesses data at rest very quickly, delivering perceivably “real-time” results.

    On the other hand, stream processing incorporates data at rest and data in motion. It computes incoming, in-stream data and correlates it with data at rest. “You don’t let the data land — you process it as it comes in,” Devgan said.

    For years, enterprises have been collecting and storing exorbitant amounts of data. A survey by Seagate and IDC reveals that 43% of data collected is largely unleveraged and other studies estimate the amount of this “dark data” is over 50%. And, not acting upon data effectively makes it just another pile on top of the growing mountain of technical debt.

    There is a significant storage expense for holding on to too much data, whether on-premises or in the cloud. But, the paradigm is shifting a bit, said Devgan. Organizations don’t want to hold onto everything. The attitude is beginning to shift toward “garbage in, garbage out.”

    Technology Behind Real-Time Processing

    So, what sort of technology does stream processing require? Processing data at the edge requires some fundamental changes to design thinking. For one, since data must be computed in-stream, there’s more emphasis on in-memory processing. Devgan explained this as “pushing compute around instead of pushing data around.” To enable this, Devgan pointed to open-source projects like Jet that provide powerful distributed batch and stream processing.

    SQL has a storied history of acting upon data at rest. Now, Devgan said, he noticed the reemergence of SQL in a stream processing context. Though the industry is still defining how the SQL dialect will conform in a streaming world, he’s bullish on the prospect of SQL guiding this area. “Streaming SQL will be the lingua franca for streaming,” he insisted. We’re also in an early era of determining how to bridge the stateful and serverless worlds, Devgan added.

    But regardless of computing location, it will come down to the net customer experience outcomes. “People want to build their business applications and are less worried about the operational management of the system,” said Devgan. Whether it’s building real-time offers for e-commerce or implementing fraud detection, it will come down to the numbers. There is also a role for stream processing in health care to optimize physical resources and enhance patient experiences with real-time data. In all these scenarios, rapid access to data and processing at the edge will determine much of their success.

  • How Does IT Management Govern EverythingOps?

    How Does IT Management Govern EverythingOps?

    The number of tools engineers use to provision software architecture in the cloud is starting to get a little out of hand. There are many competing ways to do one thing. Most often, people simply adopt a new tool to solve a DevOps problem in the moment, not always considering the longevity of the solution or how other departments are handling it. This could result in fragmentation and technical debt, making it difficult to find consistency and increase speed simultaneously across all company divisions.

    I recently interviewed Jeff Kukowski, CEO, CloudBolt Software, to explore why these issues persist and how executives should approach cloud management within their organizations. According to Kukowski, many flavors of Ops now permeate large companies. Different approaches often exist per department or interdepartmental, with teams using various software stacks.

    According to Kukowski, a new frame of mind will be necessary to remain agile in the new cloud order and to address this state. Doing so will require organizations to erode false dichotomies around ownership and IT governance.

    Everything’s Going Ops

    At this point, most large suites of digital services have adopted a complex hybrid and multi-cloud modality. And many teams have embraced their own unique form of DevOps to get a handle on the management difficulties of delivering software and maintaining reliability for microservices architectures. Different groups pulling in various directions create an ongoing battle of EverythingOps, explained Kukowski.

    You can think of it as separate Ops approaches emerging around different business functions. For example, FinOps is emerging to control spending, ITOps is about delivering services, DevOps is improving release fluidity and DevSecOps aims to bring security to the forefront of the release process. Lastly, BizDevOps is about increasing observability for business outcomes.

    At a high level, CIOs have committed to digital transformation to move quicker. Yet the problem now is “battles between different Ops can easily lead to broken IT governance,” said Kukowski. This tension introduces several false dichotomies, he explained. In addition to slowing agility, a lack of DevOps unity eschews reusability, contributing to waste and increased IT expenses, said Kukowski.

    Solution: Shifting Governance Left

    There is a belief in IT that true innovation will come if you only give engineers the keys to use anything they want. While that may be true, without better leadership with regard to the basics in this new cloud order, some things can get out of control very quickly, Kukowski warned. Instead, he argued that built-in guardrails for provisioning tools are necessary. “We have too many tools—we need better frameworks to tie this together.”

    Not only would increased cloud governance help tie together disparate tooling decisions, but Kukowski argued it also can reduce the cognitive load necessary to onboard each tool. “Shifting left will increase speed—it won’t slow people down,” he said. Contrary to popular belief, he said that shifting guardrails left can have the effect of increasing speed, which may greatly outweigh the negatives of shackling developers to a single approach.

    CloudBolt predicted siloed management will shift to organizational governance around tools in the coming year. But it’s not only the tooling decisions that require forethought, “it’s the leadership and cultural approach,” said Kukowski. A common area where greater leadership is required is in tagging workloads for applications, he said. “It’s not being done well anywhere.”

    Or, consider containerization. Containers have become fundamental to most modern cloud technology stacks, yet, the cognitive load required to deploy, configure and run a contain remains quite high, explained Kukowski. Do you allow every developer to custom-code each container, or do you enable a self-service repeatable provisioning process? If standard code is not being reused, you’re not getting speed or governance, he explained. Furthermore, with green IT initiatives becoming a priority, companies should be looking at ways to reuse processes and reduce waste where they can.

    Final Thoughts: Shift Left, Automate and Reuse

    For a cloud-native company, the emphasis is constantly on speed and growth. And engineers are absolutely driving this focus and the resulting innovation. Yet, this innovation can’t come at the expense of bringing all departments onto the same level of innovation. “Innovative stuff in the cloud can’t be separated from governance, ESG and security,” said Kukowski.

    A big hurdle of tomorrow’s cloud management will be identifying and uniting these islands of automation, these pockets of tools and preferences that differ between departments. It will also require a different frame of mind across divisions. “There’s a difference between solving the problem for the task and solving the problem for reuse and speed,” explained Kukowski. Delivering on reuse and speed will require less bespoke DevOps automation and better foundations for solving future problems, he said.

    “Shift left, automate and reuse—people shouldn’t have to touch it. That’s the thinking that needs to be applied to this new cloud order,” said Kukowski.

  • A Pandemic Side Effect: Rampant Technical Debt

    A Pandemic Side Effect: Rampant Technical Debt

    On average, about a quarter of all IT budgets are allocated to handling technical debt each year. And this figure is set to rise, as 68% of organizations say their technical debt will increase in 2022. As digital acceleration continues to drive more IT investment across divisions, the technical burden almost seems like an inevitable outcome of delivering more software and amplifying end-user experiences.

    Unsurprisingly, IT budgets are set to increase this year. This has been a stable trend throughout the pandemic as even companies that are not traditionally technical increased their reliance on software. Most teams quickly moved to remote working arrangements and acquired additional software tools to keep internal operations afloat. Not to mention the new complexities and array of tools emerging to support the microservices design paradigm.

    Although new tech adoption is vital to enable key initiatives across a company, unbridled technical debt can also inhibit digital transformation initiatives. Software AG recently released its Situation Report 2022, a survey that analyzed the state of technical debt among IT professionals in 2021 and technology adoption forecasts for 2022. Below, I’ll review the key points from the report and consider how organizations can respond.

    Increasing Reliance on Tech

    The growth in this area is astounding—78% of companies say their technical debt rose in the last year. A full 38% of companies reported taking on a great deal more; at the same time, IT budgets are set to increase for most organizations in 2022. On average, this increase is expected to be 37.1% higher than 2021 spending. It’s easy to correlate increasing technical debt with these IT budget increases.

    A large portion of this increased spending is related to digital transformation. This checks out; modernizing existing technology is a top priority for 19% of tech leaders. Other top priorities include exploring new technologies (17%) and decreasing technical debt (15%). Though managing technical debt is not ranked as the highest priority for IT leaders, it still ranks as a pressing concern due to its ability to hurt innovation. The report found that 69% of companies admitted it has caused their digital transformation initiatives to slow.

    Perceptions of Technical Debt

    So, what are the leading causes of technical debt? It turns out that most stems from recent modernization and digital transformation efforts. In fact, 39% said new digital products were a major cause. This is followed by infrastructure modernization (34%), data analysis and integration (32%) and new customer-facing services (27%).

    Interestingly, not all technical debt is bad. Often, IT professionals say the benefits justify the expense. For example, 86% say that increasing product or service launches justifies taking on tech debt.

    Technical debt is also a result of the remote nature of today’s hybrid work environment. During the pandemic, businesses made many concessions to enable remote work and adopted new collaboration and development tools to enable productivity. As a result, 88% reported the pandemic increased their awareness of technical debt. The majority, 83%, also said the pandemic has increased acceptance of technical debt, a sign that many of these changes are here to stay in the new normal of working life.

    Responding to Technical Debt

    Not all investments lead to the same degree of technical debt; some platforms are bulkier than others. Or, specific software vendor lock-in schemes can prevent lift-and-shift which can swell a technology ecosystem. So, what areas are companies investing in and which are the leading causes of technical debt?

    In 2021, companies invested heavily in cloud computing (60%), integration (43%), IoT (39%) and 5G (39%). Investment strategies will change slightly in 2022, with 5G becoming the top most-invested area at 45%. Cloud computing drops to 42% and edge computing is rising to 35%. Mapping where new investments will occur can help organizations anticipate the repercussions. Of these investments, 60% of IT professionals ranked 5G as a leading cause of technical debt. Other high producers include cloud computing, IoT and blockchain.

    In 2021, companies spent 24.8% of their IT budget handling technical debt. Managing it presents some significant challenges—32% say siloed data and processes are a key challenge when addressing technical debt. Limited budgets, regulations and a lack of internal alignment also get in the way. Clearly, resolving the issue requires both inter-department collaboration and a hefty budget.

    Final Thoughts

    Technical debt is, arguably, unavoidable as companies adopt cutting-edge technologies to remain relevant. And, taking on more technical debt is often intentional; for example, new DevOps tools are helping increase deployment fluidity and manage complex cloud-native architectures. Or, new SRE platforms bring observability and monitoring capabilities to improve application reliability. A newer programming language may be more performant, while a low-code abstraction layer complements an engineer’s workflow. In all these scenarios, the benefits often outweigh the added debt.

    Yet, much of it is unintentional and is slowing innovation down. This could be shadow APIs or forgotten rotting legacy infrastructure. Sluggish code and un-optimized routing between applications can also be a drag on resources. The reason these situations sound murky is because technical debt, itself, is difficult to quantify. And, only those addressing it with adequate visibility into their stack will have a clear indication of how much debt there really is.

    Though the current pandemic has indeed amplified technical debt, things will always evolve and new trends will replace the old. In the digital era, some degree of tech debt is inevitable within a large organization. The key will be effectively managing it over time to maintain a healthy, lean codebase and progressively release features that meet the demands of new customer experiences.

    The Software AG Situation Report 2022 surveyed 700 IT professionals on their software usage and annual predictions at the end of 2021. For more insights, you can read the full report without an email gate or paywall here.

  • How Are API Management and Service Mesh Different?

    How Are API Management and Service Mesh Different?

    The use of APIs and microservices is increasing—a recent study from F5 estimated that the industry is approaching 200 million total APIs. As organizations expand their use of APIs and microservices, they inevitably require some form of service management architecture. There are primarily two major design options for implementing this: An API management system or a service mesh.

    The key distinction between API management and service mesh can be framed using the four cardinal directions. API management is typically adopted for REST services that face external developers—referred to as north-south traffic. On the other hand, service mesh is more commonly used by internal suites of microservices to help control lateral, or east-west, communication. There are more nuances between each, as we’ll cover below, but this is essentially how they’re adopted in practice.

    However, this binary is becoming blurred as collections of enterprise digital applications expand and access to cross-company business domains begins to look and feel more like consuming a third-party SaaS. Furthermore, some suggest that with the right controls, service mesh can cater to external users in the same way API management does. What’s more, service mesh and API management are complementary—they can coexist within the same organization and even be deployed in the same projects.

    I recently chatted with Mark Cheshire, director of product at Red Hat, about the emergence of both API management and service mesh. According to Cheshire, both approaches accomplished similar goals but evolved very differently. Below, we’ll revisit why these two paradigms emerged and compare and contrast them. In a separate article, we’ll consider when it’s best to integrate them both.

    What is API Management?

    API management is most often associated with north-south traffic. By exposing your API to the outside world, you enable other developers to connect it to their systems and leverage your data and functionality like building blocks.

    API management arose in response to the rise of web APIs. API management is a broad concept encompassing many functions required to expose an API endpoint to the outside world. This can include authentication, authorization, request routing, documentation, rate limiting, monitoring, logging, monetization and life cycle management. A key goal is to make integration easier for the developer consumer. Architecturally, API management follows a gateway pattern. Consumers issue requests to the gateway which then performs routing to the underlying services.

    For APIs, the big catalyst was the emergence of Web 2.0, Cheshire explained. Numerous startups began consuming RESTful APIs to connect their web properties and partner ecosystems. Companies also found REST APIs to be a much more straightforward integration approach than SOAP and SOA. They also saw the ease-of-use benefits and adopted API-first designs within their internal infrastructure.

    As of 2021, over 70% of APIs are built for internal use. So, do all these services need API management? It depends. “If you communicate externally, you need API management,” explained Cheshire. But, it’s a grey area for services within an enterprise, he says, though inter-domain traffic often could benefit from API management.

    A diagram of API management capabilities, from 3scale
    A diagram of API management capabilities, from 3scale

    Strengths

    • Can enable north-south traffic
    • Protects the boundary of an application
    • Designed to ease external developer consumption
    • Potential for monetization, subscription plans
    • Typically plug-and-play; easier to support
    • Many solutions on the market
    • Developer experience and docs
    • Analytics and reporting

    Weaknesses

    • May be heavier than service mesh
    • Fewer open source options; vendor lock-in
    • Analysis paralysis in comparing solutions

    What is Service Mesh?

    Service mesh is most often associated with controlling east-west communication between internal services. Service meshes like Linkerd, Istio and Kuma make it easier to reimplement similar logic—such as networking, observability, traffic management, security policies and routing—across many microservices from a single control plane.

    Service mesh was born to address slightly different challenges than API management. “The transition from monolithic to microservices is all about east-west traffic,” Cheshire said. Companies found it slower to innovate monolithic applications and thus broke them up into microservices that communicate via APIs. But these microservices all needed similar operational capabilities, which would be cumbersome to repeat over and over for each service.

    “For one monolithic app blown into microservices, you may not even need a service mesh,” said Cheshire. “However, when you have hundreds or thousands, that’s when applying service mesh becomes critical.”

    Istio service mesh architecture
    Istio service mesh architecture

    Architecturally, service mesh is typically divided into a data plane and a control plane. The service mesh deploys a sidecar proxy that sits alongside each individual service. Istio, for example, utilizes the Envoy proxy. Other meshes like Linkerd use their own proxy.

    Strengths

    • Fits the microservices architecture paradigm well
    • Good for traffic routing; improving resiliency
    • NIST-recommended architecture for achieving zero-trust
    • Many open source options
    • Fits large microservices deployments

    Weaknesses

    • Requires a more hands-on approach
    • Potentially more maintenance to support
    • Unnecessary for a single service
    • Requires filter chain add-ons for north-south abilities

    Comparing API Management and Service Mesh

    API management is predominantly aligned with externalizing APIs to partners. When you do so, maintaining the contract with a consumer is more important to meet SLAs. Thus, API management commonly delivers life cycle management abilities, such as versioning. API management is necessary even if you are exposing a single API, as this service needs to be adequately documented and secured. That being said, these tenants can also be relevant to developers reusing software within an organization—there’s something to be said for adopting a product mindset when dealing with internal APIs.

    On the other hand, service mesh is more aligned with controlling internal networks. Microservices are rarely deployed alone; typically, multiple microservices make up an application. And, many more might exist throughout an organization for unique purposes. Service mesh thus helps maintain consistency throughout large networks of decoupled microservices. Service mesh is also a rapidly growing field, experiencing an explosion of new development and production use. A Cloud Native Computing Foundation (CNCF) report found that service mesh use rose in production by 50% in 2020 alone.

    To put the comparison another way, William Morgan, CEO, Buoyant described how API management is more about business logic, whereas service mesh handles intra-cluster operations:

    “API gateways typically handle ingress concerns, not intra-cluster concerns, and often deal with a certain amount of business logic, e.g. ‘User X is only allowed to make 30 requests a day’. By contrast, a service mesh like Linkerd is focused on operational concerns (e.g. instrument these calls and report their success rate; secure this communication between these pods) and purposefully avoids the mixing of business and operational logic; and focuses on communication between the services on the cluster.”

    Though service mesh and API management achieve slightly different aims, both layers make it easier to reuse web-based services. And, they certainly can be used in conjunction. For example, you could build API-first microservices with service mesh and simultaneously use API management to expose certain domains. In another article, we’ll consider when it’s best to integrate the two technologies.