devopsy.kinsta.cloud

Category: Application Performance Management/Monitoring

  • How SREs Benefit From Feature Flags

    How SREs Benefit From Feature Flags

    When you think of who uses feature flags, your mind most likely goes to software developers. In general, feature flags are closely associated with software engineering.

    But site reliability engineers (SREs), too, can benefit. SREs may not be the ones to create feature flags, but they should work closely with developers to ensure that the applications their teams support include feature flags.

    Here’s why feature flags benefit SREs, and what SREs can do to make sure their organizations take full advantage of them.

    What Is a Feature Flag?

    A feature flag—also sometimes called a feature toggle—is a technique in software engineering that allows developers to turn specific features on or off at application runtime. In other words, they let you control which features are available within your application when it’s deployed to production.

    Feature flags can be configured in such a way that they enable features to be toggled on or off for specific sets of users, rather than an entire set of end-users. This is useful if, for example, you want to turn on an experimental new feature for users who have agreed to test it without exposing that feature to your general user base.

    Usually, developers and SREs use feature flags to turn features on and off manually. But you could automate the process by, for example, writing a script that automatically shuts off a feature if your observability tools determine that the feature is causing a problem for users.

    Why Developers Love Them

    The main reason why feature flags are popular among developers is that they make it easy to implement and experiment with new features while keeping the development pipeline simple.

    Without them, developers would have to create a new application release every time they wanted to implement new features. If any of the features in that release turned out to cause a problem, they’d have to remove the features from the code or fix the buggy features, then deploy a new release. This approach is tedious and it leaves teams prone to delayed releases and application rollbacks.

    But with feature flags, developers can continuously add new features to an application without worrying that bugs within those features will lead to a rollback or a delayed release. If one feature has a problem, developers can simply turn it off using a feature flag, without having to do a new release.

    How SREs Can Benefit

    It’s not just developers whose lives are made easier—SREs, too, can leverage feature flags as a way of supporting the core SRE goal: Maximizing reliability.

    That’s because feature flags minimize the risk associated with fast-moving software development pipelines. They allow teams to experiment with new application functionality—which is important if they want to keep pushing out application enhancements to delight users—while at the same time providing a safeguard against buggy feature implementations.

    Imagine, for example, that you’re an SRE and you discover that the latest release of an application you support has a severe latency problem due to an issue with a new networking feature in the app. If that feature is controlled via a feature flag, you can remediate the issue very quickly by simply switching it off. You don’t have to wait for developers to fix the underlying code issue before you can solve the problem for your users.

    Limitations

    This doesn’t mean, of course, that feature flags are a cure-all against reliability problems.

    They won’t help you if the root cause of your reliability issue lies within your infrastructure rather than your software. If an app crashes because you misconfigured your Kubernetes clusters or you ran out of server capacity, for instance, you can’t simply turn the problem off with a feature flag.

    Note, too, that feature flags can only be applied to specific features. In other words, they can only control application functionality if developers deliberately created one to control that functionality before releasing the app. If you discover a bug that’s not managed by a feature flag, the only way to remediate the issue is to redeploy the application (or microservice, if the problem is limited to just one microservice).

    How SREs Can Encourage Feature Flag Use

    As an SRE, you most likely don’t have direct control over the software development process. You’ll therefore need to collaborate with developers to ensure that they build feature flags into their applications.

    Doing this is simple enough. Start by educating developers and making sure they understand the benefits. Be sure, too, to identify which features are most likely to create reliability issues, so that developers can prioritize creating feature flags for them. And consider factoring them into your incident response playbooks, which will ensure that incident response teams remember to take advantage of them when possible to remediate issues quickly.

  • Natural Language Query (NLQ) is Key for Self-Service BI

    Natural Language Query (NLQ) is Key for Self-Service BI

    Modern business intelligence (BI) and analytics software solutions offer more ways to explore data and find insights than ever before. However, ensuring everyone in the enterprise can use analytics independently, when they need it, remains a challenge.

    For analytics to be adopted across the full enterprise, your tools need to be tailored for all users.

    While many BI solutions today provide streamlined experiences, some data literacy and knowledge is required to fully understand them. Dashboards, for instance, are considered self-service BI tools, but they are often still too technical for the average person to use effectively.

    Today, only two out of five (40%) organizations report their people can analyze data without getting help from IT, according to Ventana Research. To better enable business users to interrogate their data independently, certain BI vendors now provide a natural language query (NLQ) capability that sits alongside their dashboards and reporting. Rather than having to search for answers in a traditional dashboard or chart, NLQ allows analytics users to ask questions of their data directly.

    An Overview of NLQ

    Natural language query is a self-service BI tool that provides the ability for an analytics user to ask a question using non-technical language and get an automatically generated answer instantly. Using data modeling and machine learning technologies to parse questions for key terms, NLQ scans related databases and generates a tailored report or chart that provides relevant insight.

    The integration and implementation of natural language query capability, as well as the complexity of the questions and the types of data supported, varies between vendors. The most common and traditional approach is search-based NLQ, which places a free text search bar somewhere within the user interface (typically on a dashboard) that allows BI users to begin typing a question and get a list of reports and charts generated quickly as potential answers.

    A smaller number of vendors also include support for voice commands in their NLQ solution.

    As NLQ increasingly becomes a part of BI ecosystems, however, some vendors have evolved their natural language offering to account for additional barriers: A user still has to know what question to ask, and how they can use it to avoid inaccurate or irrelevant results to their question.

    The new approach is called guided natural language query, or guided NLQ, which takes the concept of NLQ further by programming the solution itself to guide the user as they type a question, helping them structure it using pre-defined sequences and suggested prompts.

    This eliminates the burden of knowledge or any potential problems in accurately interpreting the semantics (language) of a user’s question, as the system itself assists them in building it, from start to finish. They don’t need prior technical knowledge or need to go to IT to get started finding answers from data.

    When employed correctly, guided NLQ can help make the use of analytics more pervasive and open up analytics for every type of business person in the enterprise—not just experts.

    Self-Service That Suits Every Enterprise User

    The aim of NLQ is to help workers not accustomed to traditional, structured BI tools find insights and information they need to inform and make business decisions, while also being a helpful tool for advanced users throughout the enterprise.

    For non-technical analytics users, NLQ is an easy way to get detailed answers from their data faster than exploring a dashboard or chart and use it as a springboard for their ad-hoc reporting.

    Similarly, advanced users can leverage NLQ, whether guided or traditional, to quickly generate a report or chart and further build upon it.

    Generated insights using NLQ can be added into existing analytic content, such as dashboards, reports and data stories, to enhance them with additional detail, relatively quickly and easily.

    Finally, NLQ can be embedded into the interfaces of your users’ workplace applications, to make analytics far more ubiquitous in their day-to-day workflows. They can ask a question, get an answer, and start exploring data much faster and more easily than via dashboards alone.

    Ultimately, having an awareness of your users’ varying self-service BI needs, and how the new wave of tools such as Guided NLQ can help to further open up analytics to more people, will help you ensure your analytics tools are adopted and usable by everyone throughout the enterprise.

  • Netlify Leverages GraphQL to Simplify Access to Backend Services

    Netlify Leverages GraphQL to Simplify Access to Backend Services

    Netlify this week added a Netlify Graph tool to its portfolio that makes backend services available to developers via a single click. The tool aims to simplify DevOps teams’ access to those services when building their applications.

    Netlify Graph is based on the GraphQL query language for application programming interfaces (APIs) and enables developers to integrate services such as GitHub, Salesforce and Stripe within frontend and web application development tools without any direct intervention required from a DevOps team.

    Netlify provides an application development framework based on JavaScript, application programming interfaces (APIs) and markup languages, otherwise known as a JAMstack, to isolate frontend and backend components of an application.

    Netlify CEO Matt Biilmann said Netlify Graph is based on the GraphQL technology the company gained when it acquired OneGraph. The overall goal is to make it simple for frontend developers to invoke backend services accessed via either GraphQL or REST APIs, he said. Services based on GraphQL can be especially complex for developers to invoke as they navigate the management of tokens and application secrets, Biilmann added.

    It’s not clear to what degree GraphQL APIs will supplant REST APIs. However, developers tend to prefer GraphQL APIs because they provide more granular control over what data is accessed. The challenge is the number of backend services that expose GraphQL APIs is still comparatively limited. As GraphQL becomes more widely adopted, the level of effort DevOps teams now need to expend to expose backend services should decline as tools such as Netlify Graph are employed by frontend developers.

    As a result, at least theoretically, the rate at which new applications are built and deployed to drive digital business transformation initiatives should increase. Most IT teams will not be replacing REST APIs with GraphQL-based APIs overnight, but the percentage of new applications that rely on GraphQL APIs will steadily increase. DevOps teams looking for easier ways to provide developers with simpler access to backend services now have an incentive to embrace GraphQL.

    Ultimately, Biilmann said, the goal is to enable developers to build applications without having to directly engage anyone on the IT team. Each backend service can be managed in isolation from the frontend of any application. It’s not clear how long it might take to achieve that goal, but as responsibility for the end-to-end management of the application experience continues to shift left toward developers, the need to surface backend systems-as-a-service becomes more pronounced. In fact, given the current focus on improving developer productivity, IT organizations might soon face a mandate to embrace GraphQL more broadly. After all, every minute that a developer spends on plumbing is time they’re not spending writing actual business logic.

    Naturally, it may take time for enterprise IT to be truly transformed into a set of services that developers can easily consume. But as progress continues to be made, that dream may finally be realized.

  • IBM Automation Insights: Improving Observability and App Performance

    IBM Automation Insights: Improving Observability and App Performance

    Observability offers deep visibility into the internal state of complex systems in order to quickly identify and troubleshoot performance issues to keep up with customer expectations and business requirements. Observability enables better application performance monitoring (APM). So, how can DevOps teams gain full observability into modern applications? 

    On Feb. 23, experts from IBM and GitLab will come together at the IBM Automation Insights virtual event to discuss how to effectively observe, manage and maintain applications across platforms and how to improve application performance.

    The virtual event has been curated to create interactive sessions, led by DevOps experts who will deep dive on the latest innovations, conduct demos and share approaches that tackle challenges.

    IBM Automation Insights features a speaker lineup of distinguished engineers, IT leaders and executives, including:

    • Cosmo Schillaci, DevOps business leader at IBM
    • Jennifer Velasquez, Americas DevOps leader at IBM
    • Laurel Dickson-Bull, product manager at IBM
    • Kurt Dusek, senior solutions architect at GitLab
    • Jamie Coleman, software engineer at IBM

    Attendees will learn how Instana and GitLab can enable DevOps teams to accelerate CI/CD pipelines and how to build applications easily using cloud-native Java applications and microservices locally, with containers, and for OpenShift/Kubernetes.

    There will be a cross-platform panel discussion in which attendees will get a chance to interact with the speakers, ask questions on any of the strategies presented and discuss several topics, including cybersecurity, optimization for observability, vulnerability tool proliferation and more.

    For more information and to register, please visit the IBM Automation Insights website.

  • IBM Shifts Mainframe App Testing to the Cloud

    IBM Shifts Mainframe App Testing to the Cloud

    IBM this week revealed it has added the ability to set up testing environments on the IBM Cloud for z/OS mainframe applications.

    In addition, on March 15, IBM will make available an IBM Z and cloud modernization stack to make it simpler to modernize mainframe applications by adding application programming interfaces (APIs) using a cloud-native set of DevOps tools that run in a Kubernetes environment.

    Tarun Chopra, vice president for the IBM Z Hybrid Cloud, said IBM Z as-a-service on IBM Cloud for development and test eliminates the need to dedicate mainframe capacity to an application testing environment. Instead, mainframe application testing can now occur in the IBM Cloud, he added.

    Currently available as a closed experimental today, IBM plans to make IBM Wazi as-a-Service (Wazi aaS) for z/OS generally available in the second half of this year. That offering will enable DevOps teams to self-provision an instance of a z/OS virtual server on a secure private space running on IBM Cloud.

    That hybrid approach means application testing no longer disrupts mission-critical applications running in production environments on a mainframe, added Chopra. The overall time required to gain access to a z/OS development and test environments will drop from days or weeks to six minutes or less, he said.

    At the same time, it will become easier to modernize mainframe applications by making them more accessible to other applications via a set of secure APIs. Many organizations today increase their technical debt by duplicating functions that already exist within mainframe environments simply because those functions are not easily accessible to developers. The IBM Z and cloud modernization stack makes it simpler to modernize mainframe applications to drive a digital business transformation initiative without increasing the overall technical debt an organization needs to service, noted Chopra.

    While many workloads over the years have been migrated off mainframes or simply retired, the applications that run on z/OS today are still among the most mission-critical that organizations run. A recent IBM Institute for Business Value study found 71% of respondents said mainframe-based applications are still central to their business strategy. Four out of five respondents also noted they needed to modernize those applications, as well.

    Modernizing mainframe applications is a less complicated endeavor than migrating them. The challenge is that building, deploying, maintaining and securing all the APIs required to do so represents a significant amount of effort. IBM said it is committed to simplifying the mainframe app modernization process.

    One way or another, mainframe applications are becoming part of the larger distributed computing environment. Most organizations that have invested in mainframe applications continue to run them in on-premises environments because of the unique transaction processing capabilities a mainframe provides. Mainframe applications that have been migrated to the cloud generally run more slowly than they did when running natively on a mainframe. There may still be circumstances where migrating a mainframe application to the cloud makes sense as part of a larger IT strategy. However, like every other application being accessed over a wide area network (WAN) there will always be performance tradeoffs to consider.

  • Dynatrace Advances Application Environments as Code

    Dynatrace Advances Application Environments as Code

    Dynatrace, at its online Perform 2022 event, launched a bevy of updates to its portfolio that make it easier for DevOps teams to programmatically embed observability and other software intelligence capabilities within applications.

    At the same time, Dynatrace also revealed that it has extended the reach of its observability platform to now include all the major serverless computing platforms.

    Additionally, the company extended its existing Application Security Module to provide the ability to detect vulnerabilities and block attacks against application runtimes in real-time using the framework defined by the Open Web Application Security Project (OWASP). That module is based on Dynatrace Smartscape topology software and a Davis artificial intelligence (AI) engine to provide real-time visibility into IT environments that enables DevSecOps teams to better prioritize which vulnerabilities to remediate first based on their severity.

    Dynatrace CEO Rick McConnell told conference attendees that all these capabilities are enabled by agent software that can be programmatically embedded within applications as code using application programming interfaces (APIs) and templates to automatically surface software intelligence. Rather than having to deploy multiple agents, McConnell said, IT teams can deploy a single agent to drive a wide range of analytics.

    That capability makes it easier for IT organizations to achieve service level objectives (SLOs) for performance, quality, and security at a time when more responsibility for the management of applications is shifting left toward developers, he added.

    It’s not quite clear how far responsibility for the management of applications after they are deployed is shifting left toward developers. It tends to vary widely by organization. The one thing that is certain is more applications are being instrumented. Adoption of application performance management (APM) platforms was previously limited by cost concerns and the challenges associated with embedding agent software in applications. As it becomes easier to instrument applications the data that can be collected and analyzed from logs, metrics and traces are now considerably greater. As such, APM platforms are now being positioned as observability platforms that can correlate analytics collected from applications and the underlying infrastructure upon which they depend.

    Much of that need for greater instrumentation is being driven by digital business transformation initiatives, noted McConnell. Many of those applications are being constructed using microservices that have dependencies on each other. A single agent makes it easier for IT teams to instrument cloud-native application environments, he noted.

    There is no shortage of options when it comes to instrumenting and observing applications. The issue now is turning all the data being collected into actionable intelligence. As a result, observability platforms are being imbued with AI engines that make it possible to analyze massive amounts of data. Armed with those insights, it then becomes possible for an existing IT team to manage a much more complex IT environment at scale. In fact, it’s now no longer a question of whether AI engines will be injected into DevOps workflows as much as it is to what degree.

  • Techstrong Research DevOps Radar Report 2022

    Techstrong Research DevOps Radar Report 2022

    You can’t control the pace of change; instead, consider that we are in a process of reinvention—both with regard to the way organizations use technology and how vendors position their offerings. At the same time, it’s never been more important for IT organizations to have complete visibility into the way technology is used from both a security and operations perspective. 

    This need to embrace reinvention while continuing to have broad and deep insight across all technology platforms led me to create a 2022 DevOps Radar Report. In this article, I highlight three near-term priorities that must be on every technology team’s radar. In addition, I’ll point out some long-range radar trends that should be on your agenda.

    Collision Warning! Your Current Radar View

    API Security

    APIs are the lifeblood of the cloud-native and app-based economies. Increasingly, poor API security across a variety of applications and systems puts your organization at risk. A compromised API or the use of unsupported or outdated APIs can prove to be an easy attack vector for cybercriminals. In addition, API vulnerabilities are calling into question how organizations approach application security and the usefulness of web application firewalls (WAF) and encryption protocols. Therefore, API security is rapidly becoming a top security priority for enterprise security teams and software professionals. 

    Failing to adequately manage the security of APIs creates a host of new challenges and questions that you must consider. For example, who is using which APIs? Are your APIs updated? Did your DevOps team evaluate the security of your APIs during their selection process? Have you formalized the way your organization is evaluating API security?

    It is increasingly clear that an API breach can lead to the downfall of your entire digital software strategy—but it doesn’t have to be this way.

    At Techstrong Research, we segment the market into “pure-play” API security vendors and larger security vendors with specific API security offerings. Our team will be publishing a Market Vista report on the API security landscape soon, so if you want to share your opinion or thoughts, please reach out.

    Below is a brief list of some of the vendors we are tracking in the API security market. Of course, expect consolidation—some of these pure-play vendors will likely be acquired in 2022.

    Pure-play API security vendors we’re tracking include:

    General security vendors with API security offerings that we’re tracking:

    Observability is Evolving to AIOps

    Modern applications are composed of containers and APIs that your organization can’t control, and into which visibility is difficult. Monitoring and application performance management (APM) solutions aren’t new; for many years log vendors built their businesses on collecting log files and sending alerts … so what’s changed?

    First, logs without context are meaningless. You need to understand upstream and downstream dependencies. Additionally, speaking for myself, I’m a mere mortal—if your teams are also made up of mere mortals, there is no way they can ingest, correlate and understand the tsunami of incoming DevOps log data quickly enough to take proactive measures.

    Besides the need to understand context, we are dealing with multiple cloud and on-premises platforms that each have their own logging protocols. Ultimately, the goal of observability and AIOps is to understand what’s happening across and among all these environments and technologies so you can detect and resolve issues as quickly as possible to keep your systems efficient and reliable and your customers happy. 

    Humans are the Biggest Security Vulnerability

    As an industry analyst, I see our clients pouring resources into cybersecurity efforts to prevent zero-day attacks and sophisticated advanced persistent threats (APTs). These threats, while real enough, are mostly external and may not pose as great a threat as many organizations believe. In fact, the vast majority of security vulnerabilities are caused by:

    • Well-meaning employees making innocent mistakes, including:
      • Falling victim to phishing attacks
      • Poor corporate password policies
      • Cloud misconfigurations and open ports
    • Technical debt—relying on older software and hardware that was never designed for a 100% mobile workforce, for example

    The first step in combating this problem is improving security awareness and building a cybersecurity culture. That starts with everyone in your organization understanding that security is a team sport and that it is a 24/7/365 responsibility. Even if you work from home; even when you’re cooking dinner while checking email on your phone, you need to have your “corporate security/Is it safe?” hat on.

    At the same time, machine learning, AI and automation have a role to play. There are emerging offerings that help enforce DevSecOps best practices and help security teams reduce the constant noise so they can focus on the most vital threats.

    Longview Radar: Plan Your Strategy Now

    To support emerging business models, what do DevOps leadership teams need to be thinking about from a long-term perspective?

    • Web3/Web 3.0 and the potential for change to the internet business model. But there is still a ton of opportunity with Web 2.0.
    • How will you support low-code/no-code? As a business model, vendors are selling directly to business units—it’s likely that your IT team has an existing solution to solve the business pain point(s).
    • Packaged software offerings that include blockchain, NFTs and other digital ledger technologies.
    • How to upskill teams, both from a technical perspective and a business perspective.
    • Providing developer teams with the tools they need while establishing guardrails to enforce governance and compliance requirements.
    • How to manage cloud costs in a predictable way no matter how many cloud vendors your organization uses.

    Conclusion: You Can’t Put the Brakes on Change

    Attempting to control change and the speed of reinvention is futile. You need to get comfortable riding the waves of industry change, or you’ll be swept away! Every IT leader needs to learn how to roll with and manage the ebb and flow of business, IT and personnel change.

    The best place to start is by building a successful culture within your organization. Although we often talk about technology-focused solutions to business problems, the first step on this journey doesn’t involve tech at all; it involves helping IT and business teams understand each other’s pain points and learn how they can each support a continual reinvention process.

    I will be tracking these incoming radar targets and other associated trends throughout the year and I look forward to your input. Do you think I missed something? Have a question or want to share what your organization is doing to support the need for continual reinvention? 

    Feel free to engage with me – @DDKirsch, LinkedIn or, of course, email. 

  • The Evolution of Incident Management

    The Evolution of Incident Management

    Have you ever thought about the history of incident management?

    If you’re an SRE, you might be so caught up in the day-to-day work of managing reliability and responding to incidents that you never take time to step back and reflect on the evolution of your role and your responsibilities. And that’s a shame because SREs didn’t invent incident management concepts and strategies on their own.

    On the contrary, the way SREs think about incident response, structure incident management teams and rank the priority of incidents owes much to incident management strategies developed in the offline world decades ago. To understand fully what it means to be an SRE today, you have to appreciate this deep history of incident management and response.

    So, let’s take a look at that history and examine where modern incident response concepts originated.

    Historical Problems in Incident Management

    Societies have always had incidents, of course. Fires, floods, infrastructure breakdowns and similar crises have been happening for millennia.

    For most of history, however, humans lacked an efficient, purposeful way to manage these sorts of incidents. Response efforts were ad hoc, and their effectiveness owed more than a little of their success to sheer luck.

    Particular challenges included:

    • Lack of effective, consistent communication between stakeholders.
    • Varying organizational structures which made it hard to identify leaders, coordinate response efforts and delegate tasks.
    • Inconsistent response strategies.
    • Differing approaches to assessing the priority of incidents.

    Historically, organizations may have been able to handle incidents well enough if the incidents required response from only one, small group. But the more stakeholders involved, the harder it was to respond quickly and effectively.

    Putting Out Fires: The Birth of the ICS

    Things began to change for the better when stakeholders started thinking about better ways to put out fires—literally.

    By the 1960s, fire chiefs in California realized that they were struggling to respond effectively to the wildfires that broke out every summer. Each year brought worse blazes than the last, with more land burned and more buildings lost. The Laguna fire of 1970 brought matters to a head and was the catalyst for a new approach to incident response for fire agencies.

    After assessing what was going wrong, the fire chiefs determined that it wasn’t a lack of equipment or personnel. It was poor coordination among the various firefighting agencies that responded to blazes. Lacking a clear chain of command and a systematic approach to firefighting, the agencies struggled to deploy their resources rapidly and, thus, effectively.

    To fix the problem, California fire chiefs developed what became known as the Incident Command System, or ICS. The ICS defined a hierarchy for incident response with an incident commander at the top. It also defined several categories of incident response processes including operations, planning, logistics and finance. And it established a consistent set of terms that stakeholders can use to describe their actions during incident response, which makes it easier to communicate clearly.

    Although the ICS was initially conceived to fight fires, it became the de facto standard for organizing incident response strategies of all types.

    From ICS to NIMS

    The history of incident response doesn’t end with the ICS. A new chapter began in the early 2000s when the U.S. federal government developed an even more comprehensive approach to incident management called the National Incident Management System, or NIMS.

    NIMS was born in the wake of the September 11, 2001, terrorist attacks, which underlined the importance of efficient communication not just between different agencies of the same type (like fire departments), but of entirely separate organizations. To achieve this, NIMS expanded upon the principles of the ICS.

    In addition to adopting most of the incident command principles and practices included in the ICS, NIMS included standards for coordinating the distribution of resources. It also embraced the concept of the emergency operations center, which is in some ways similar to a network operations center in the digital world.

    In some respects, NIMS resembled a compliance framework (although to be clear, that’s not what it is). It includes fourteen management principles, similar to compliance controls, that organizations must implement to manage incidents using a NIMS approach.

    Incident Management Today

    Obviously, putting out forest fires and responding to terrorist attacks is pretty different from dealing with data center failures or a buggy application deployment. ICS and NIMS weren’t designed for site reliability engineering or IT teams specifically.

    Still, the influence of ICS and NIMS on the way SREs think is clear enough. Terminology like “incident commander” comes from these frameworks. So do concepts like shared accountability for incident response processes and the importance of involving all stakeholders—not just technical teams—in incident response.

    ICS and NIMS may not be acronyms familiar to most SREs. But they should be, because they are the historical sources of the incident management philosophies that form the foundation for SRE work today and offer valuable lessons for any SRE on the job today.

  • How Stream Processing Goes Beyond Real-Time

    How Stream Processing Goes Beyond Real-Time

    How fast is real-time? According to Robert Miller’s 1968 seminal study on human-computer interaction, we perceive a response time of 100 milliseconds or less as instantaneous. This is much quicker than the average human reaction time, which one online test clocks at 273 milliseconds.

    Nowadays, customer-facing applications need to respond in as close to real-time as possible to win our attention and meet standard requirements. For example, 100 milliseconds is a golden rule for credit card processors to meet SLAs. Especially in our new remote-first economy, other areas like media, health care, retail and manufacturing need to take full advantage of real-time data processing to enhance digital business and stay relevant. Yet, leveling up with real-time requires some technical nuances that might be challenging to implement with the current developer skills gap.

    According to Manish Devgan, chief product officer at Hazelcast, the problem with most data management techniques is that data isn’t being processed in-stream at the edge. Instead, data gets stored and thrown into a data lake where it accumulates, unfulfilled. To make matters more complicated, decentralized data management means organizations may be overseeing many different data lakes, adding to the potential for unused data and increased latency.

    I recently met with Devgan to learn how in-memory stream processing goes beyond real-time data. According to Devgan, the secret sauce is processing incoming data in-stream while tying it to persistent storage to create context. By doing so, companies could unlock more innovative real-time experiences, he said.

    Use Cases For Real-Time

    Real-time presents many opportunities for business operations and end-user customer engagements. This is especially true in the financial sector. For example, Devgan explains how one bank was able to use real-time data at the moment of customer interaction at their ATMs. The system was able to fetch a history of the user account and extend an offer for a low line of credit, increasing their total loans by 400%. Capturing value at the moment of user intent is a golden opportunity, said Devgan.

    Capital One is another example of a financial institution leveraging advanced technology to improve the product experience. “They’re one of the leading banks in terms of technology,” said Devgan. Capital One’s threat detection system calls a machine learning model in real-time to detect and prevent fraud. Only by acting on incoming data in the moment of interaction can preventative responses like these be signaled.

    Other scenarios are also beginning to incorporate increased real-time analysis. “This split-second preemptive decision-making is important for many other use cases,” said Devgan. Physical retail might loop into customer loyalty programs at point-of-sale systems to increase in-store spending. Or, automobile manufacturers may want to use real-time processing to detect a problem with their robotic arms sooner rather than later.

    Real-time vs. Stream Processing

    When an event comes in, it usually has very little information. But when you look up a customer profile and connect it to a greater context, you can direct further actions. “Correlating data in motion with data at rest is where the magic happens,” Devgan. This distinction uncovers the difference between real-time data and stream processing.

    The real-time label tends to get thrown around a lot, and it’s quantified differently for different groups. Real-time might be five minutes for some applications but 10 milliseconds for others. Real-time could simply mean that a system accesses data at rest very quickly, delivering perceivably “real-time” results.

    On the other hand, stream processing incorporates data at rest and data in motion. It computes incoming, in-stream data and correlates it with data at rest. “You don’t let the data land — you process it as it comes in,” Devgan said.

    For years, enterprises have been collecting and storing exorbitant amounts of data. A survey by Seagate and IDC reveals that 43% of data collected is largely unleveraged and other studies estimate the amount of this “dark data” is over 50%. And, not acting upon data effectively makes it just another pile on top of the growing mountain of technical debt.

    There is a significant storage expense for holding on to too much data, whether on-premises or in the cloud. But, the paradigm is shifting a bit, said Devgan. Organizations don’t want to hold onto everything. The attitude is beginning to shift toward “garbage in, garbage out.”

    Technology Behind Real-Time Processing

    So, what sort of technology does stream processing require? Processing data at the edge requires some fundamental changes to design thinking. For one, since data must be computed in-stream, there’s more emphasis on in-memory processing. Devgan explained this as “pushing compute around instead of pushing data around.” To enable this, Devgan pointed to open-source projects like Jet that provide powerful distributed batch and stream processing.

    SQL has a storied history of acting upon data at rest. Now, Devgan said, he noticed the reemergence of SQL in a stream processing context. Though the industry is still defining how the SQL dialect will conform in a streaming world, he’s bullish on the prospect of SQL guiding this area. “Streaming SQL will be the lingua franca for streaming,” he insisted. We’re also in an early era of determining how to bridge the stateful and serverless worlds, Devgan added.

    But regardless of computing location, it will come down to the net customer experience outcomes. “People want to build their business applications and are less worried about the operational management of the system,” said Devgan. Whether it’s building real-time offers for e-commerce or implementing fraud detection, it will come down to the numbers. There is also a role for stream processing in health care to optimize physical resources and enhance patient experiences with real-time data. In all these scenarios, rapid access to data and processing at the edge will determine much of their success.

  • Heterogeneous Hardware Needs Universal Software

    Heterogeneous Hardware Needs Universal Software

    As hardware advances and diversifies, we’re entering what many see as a new golden age of computer architecture. However, the idea of an ever-expanding hardware landscape can be daunting for software developers, because while hardware diversity is wonderful for price and innovation, it can lead to spiraling software complexity on the back end. So many software tools are specific to the hardware they run on that a world where CPUs, GPUs, FPGAs, ASICs, DSPs and more are freely intermingled feels overwhelming; an invitation to interoperability headaches and spiraling tech overhead. The question on the minds of developers is how to navigate this constantly changing ecosystem to take advantage of the new world of hardware acceleration without driving themselves crazy in the process. 

    The challenge for developers is that, if we look historically, every time a new piece of world-changing silicon shows up, it often comes with a new software stack of tools. With heterogeneous hardware innovation moving at a nearly untrackable pace already, keeping up with the software end is beginning to feel untenable. In response, a new philosophy is emerging: We should be able to have a common set of tools, libraries, frameworks and compilers that can address a wide variety of accelerators in a “silicon agnostic” way. 

    Anyone doing DevOps work today should be asking, “Are the tools we are adopting ready for the heterogeneous hardware future?” Rather than using proprietary stacks exclusive to individual vendors’ concepts of how acceleration should work, we should be thinking about abstraction layers that allow us to move to open software stacks that work across multiple vendors and architectures. 

    This philosophy is central to a number of movements—oneAPI being one example—of asking for libraries, compilers and analysis tools that are more versatile. It’s incumbent on any future-conscious developer to analyze the tools they’re using and ask if they are preparing for the future or locking them into a single solution that excludes you from other innovations. 

    When it comes to navigating the changing developer landscape, it’s particularly important to look at the multiple-vendor piece of the puzzle. One of the signature strengths of computing and electronics is its ability to interoperate. Without compatibility between platforms, we end up locked into single solutions that don’t take advantage of the diverse landscape of innovation available to us. If systems are built from the ground up with mixing and matching in mind, we don’t have to change our software tools to adapt to new technologies and we can use the same languages, libraries and analysis tools we are used to. 

    It’s important we take on this work now because hardware diversity is not just at an all-time high, its momentum is only growing. In the past when the hardware landscape was more limited, switching to different toolsets for different CPUs was a feasible, if inelegant solution. Today’s computers, however, have computational capabilities from multiple vendors. If my tools don’t stretch across all of its ecosystems, I end up in a complex mess very quickly.  

    A particularly relevant example can be found in supercomputing, one of the most exciting and fast-moving areas of innovation in the developer community. What is the most exciting thing about supercomputing? It’s heterogeneous, allowing multiple architectures to take on specialized hardware acceleration for the different tasks for which they are best suited. The Aurora supercomputer, for example, is planned to achieve up to 2 exaflops of performance—a staggering number (the first supercomputer I helped design had 1 teraflop—or 2 millionths of Aurora’s performance). The amazing capabilities of these machines are only possible because of advancements in heterogeneous memory technologies, storage capabilities, sophisticated CPUs and GPUs all working together seamlessly.  

    The advancements we’re seeing in supercomputers, as always, will ultimately filter down into lower-level systems. In the near future, regardless of the type of work you do, enormous computing power is coming online. We need to make sure that our libraries and tools can take full advantage of it when it arrives.

    Two great places to learn more about activities to help software be open, multi-vendor and multi-architecture for heterogeneous programming are:  https://sycl.tech/ and https://www.oneapi.io. There are numerous online tutorials, a link to the SYCL book (free PDF download) and much more.