devopsy.kinsta.cloud

Category: DevOps and Open Technologies

Editorial Calendar for February 2016

  • Nixpkgs: How to Build Better, More Open Software

    Nixpkgs: How to Build Better, More Open Software

    The world we know today depends in fundamental ways on open source software. As providers of this software, package managers serve their respective ecosystems from their centralized, opaque hubs in ways that are sometimes difficult to reconcile with the open source nature of the software they’re serving. I want to talk about the extent to which that’s true and discuss Nixpkgs—the largest open source package repo in existence today—in this context.

    People sometimes focus on Nix the language or Nix the package manager, but it’s the Nixpkgs package collection (pronounced ‘nix packages’) that is core to the success of Nix. But it’s more than that: We can also understand Nixpkgs as a powerful tool to ensure the continued growth, transparency and resilience of open source software, first and foremost because it democratizes the process of curating this software and its build instructions. It is the living compendium of the efforts of thousands of contributors who both stand on the shoulders of giants (the software they’re packaging) and provide the ladder for others to climb further.

    A Purely Functional Grimoire

    The Nix package manager is different from other package managers in several ways, but one key aspect is that Nixpkgs contains only recipes to build packages, not the packages themselves. This is a fundamental idea to Nix: If we build a package reproducibly, we can separate the specification of a build from its built output. When we build a “recipe” for a package, we determine the output hash of that build–without actually building it. This means you don’t need a cache of build artifacts: Recipes are enough to rebuild your software world.

    Nixpkgs is just that: The big fat recipe book of open source. Anybody can view it, fork it, patch it and use Nix to inspect the largest set of free software recipes in the world. Anybody can create a cache that uses those build recipes (or your own!) to publish packages: The default cache is the official one run by the NixOS Foundation at cache.nixos.org.

    Derivation and Substitution

    So what happens, in practice, when I download a package from the Nix cache? When you query for a package, the CLI creates the specification (we call it a derivation) of the package to be built. This derivation includes a hash: The summary identifier of all source code, patches, build flags and hashes of the dependencies themselves.

    With that output hash in hand, Nix the package manager just asks whether that path exists in its cache. If it does, Nix fetches the contents of the package you asked for; if it doesn’t, Nix will build the package locally. We call this move substitution: Creating a derivation from a Nixpkgs expression gives you all the information you need about the output of the build, so the cache can substitute a binary with that output hash if we have it. Of course, if you end up having to build the package locally, it will result in the expected output hash anyway.

    Democratizing Package Curation

    The term ‘lingua franca’ refers to the dialect spoken by sailors across the Mediterranean in previous centuries, and it was the shared language with which one could communicate and do business from Morocco to Greece. Today we package software for independent islands, each with its own rules, languages and cultures and, until Nix, we had not found a way to “speak” across the many ports-of-call of our software archipelago. What this allowed for is packages being available on more platforms, which drew more developers in, which resulted in Nixpkgs becoming more complete, and so on in a positive feedback loop.

    Nix is a cross-platform, cross-architecture package manager. If we had to design the next-generation package manager, this would probably be at the top of the requirements list, yet few package managers enable this today. Each of the teams at RedHat or Debian maintains their own packaging format and package cache and has a fixed team of vetted contributors to keep packages secure and up-to-date. This results in few who effectively contribute to packaging open source projects and do so for one of many platforms.

    An open book of package recipes written in this lingua franca results in another surprising effect: Nixpkgs maintains more packages than anybody with an order of magnitude fewer maintainers [1]. It makes sense: Since it’s a common language, people can review the packaging code for packages they don’t necessarily know intimately, which in turn leads to contributors and reviewers cycling through updates faster and providing more up-to-date packages, which reinforces the previous feedback loop from earlier.

    Even if the rest of the Nix ecosystem stalled out, we’d still have this incredible resource in Nixpkgs to curate and share open source software; flox sources its packages exclusively from Nixpkgs for this (and many other) reasons. As of February 2023, there were around 6,000 contributors on the Nixpkgs repository and growing.

    (via (repology.org)[https://repology.org/repositories/graphs])

    Stay Unstable, my Friends

    Nixpkgs embodies a new model for package management and one that opens it up to more people to contribute. Providing cross-platform and cross-language packages is obviously attractive, but it’s the fact that it provides a lingua franca for software packaging that makes it a pillar of open source moving forward. Because Nixpkgs unifies a broader swath of software and makes it easy for anybody to contribute new packages and fix existing ones, it continues to grow and serve the open source community: A positive feedback loop propelling the existence of a more free and open world of software.

    References:

    Nixos.org

    https://reproducible-builds.org/

    https://wiki.debian.org/SecureApt

    https://rpm-packaging-guide.github.io/

  • Grafana Labs Reins in Cost of Storing Observability Data

    Grafana Labs Reins in Cost of Storing Observability Data

    Grafana Labs today made available an ability to customize what observability data is collected via its managed cloud service. This capability can help reduce storage costs in addition to enhancing the dashboard Grafana provides to make it simpler to track metrics usage.

    Wayne Jin, vice president of product marketing for Grafana Labs, said both capabilities would make it easier for DevOps teams to track and ultimately reduce the cost of storing the time-series data collected via its cloud platform.

    As more DevOps teams use an observability platform to continuously collect metrics like traces and log data, they are discovering that the cost of storing all that high-cardinality data can easily spin out of control.

    Grafana Labs is now making available an Adaptive Metrics capability that enables DevOps teams to fine-tune what data is collected and stored. However, in the event of an issue, DevOps teams can also turn back on all data the platform can collect to aid an investigation of the root cause of disruption, noted Jin.

    The Cardinality Management dashboards Grafana Labs made available last year for paid tiers of its cloud service are now being made available to users of the free tier of service as well, Jin said. This will further help reduce storage costs by making it easier to identify metrics that are being collected but not used by anyone on the DevOps team, he added.

    The Adaptive Metrics aggregation engine can then be applied to transform these metrics at ingestion into lower cardinality versions. Unused or partially used labels are stripped from incoming metrics, reducing the total count of time-series data collected. Adaptive Metrics also recommends aggregations based on an organization’s historic usage patterns, and DevOps teams can choose which aggregation rules to apply. Dashboards, alerts and historic queries are guaranteed to continue to work as they did before aggregation, with no rewrites needed.

    Based on results reported by early users, Grafana Labs reported that Grafana Cloud Adaptive Metrics could eliminate an estimated 20-50% of the time-series data collected with no perceived impact on observability.

    The Grafana Cloud service relies on an instance of open source Grafana Mimir to store data collected in a format originally defined for the open source Prometheus monitoring tool. While Prometheus is widely employed in Kubernetes environments, it’s also starting to gain traction as a monitoring platform in legacy monolithic environments. The amount of data being collected via that platform over time is steadily increasing. The Grafana Labs approach makes it possible for DevOps teams to reduce that data whenever there is no usage that needs to be investigated, for example.

    It’s still early days as far as adoption of observability is concerned, but in uncertain economic times, there’s generally a lot more sensitivity to the total cost of IT. Many organizations are especially focused on reducing the cost of storing data in the cloud to rein in monthly spending on cloud services. Regardless of the motivation, collecting and storing data that isn’t needed is never the best use of cloud storage resources that, from a cost perspective, are just as finite in the cloud.

  • CDF Report Reveals Lack of DevOps Maturity

    CDF Report Reveals Lack of DevOps Maturity

    A global survey of nearly 26,000 DevOps professionals published today by the Continuous Delivery Foundation (CDF) at the cdCon + GitOpsCon event found 84% of respondents are involved in some type of DevOps activity. However, the report also found just under half 49% are using a continuous integration (CI) or continuous delivery (CD) platform and only 22% are using CI and CD approaches to automate all building, testing and deployment of code in a production environment.

    The report, conducted in collaboration with SlashData, found the number of developers using CI or CD platforms only increased two percentage points year-over-year, with the use of self-hosted CI/CD tools dropping from 32% in the first quarter of 2022 to 23% in the same period this year.

    The survey also noted that only 7% of respondents this year described themselves as a DevOps engineer or specialist.

    Overall, medium-sized businesses have the highest involvement in DevOps practices (94%), followed by small businesses and large enterprises at just over 90%. More than three-quarters of developers (79%) are now using DevOps practices.

    The most common DevOps activity is monitoring software or infrastructure performance (47%), followed by application security testing and using CI to build and test code, tied at 37% each, and approving code deployments into production environments at 36%, the survey found.

    The average number of DevOps-related technologies used by developers (4.5) has, meanwhile, remained stable over the last year. The most widely used tools address issue tracking (39%), source code management (38%), test automation management (34%), collaboration and knowledge-sharing tools (32%) and application performance management/observability and agile project management tools tied at 31%.

    The survey also noted that the percentage of top performers—defined as those with lead times of less than one day—now stands at 15%. Survey results for top performers over the years continue to fluctuate between 13% and 17% of the survey total. On the plus side, 40% of developers have lead time changes of less than one week, the second-highest ranking for this survey.

    More than a third of all DevOps practitioners (34%), however, take longer than a week to restore service. Only 12% of DevOps practitioners take less than an hour to restore service.

    In general, the survey finds that organizations that use 10 or more DevOps tools are twice as likely to be high performers.

    Fatih Degirmenci, executive director for the CDF, told conference attendees the consortium, which now oversees nine projects, including the open source Jenkins CI/CD platform, continues to make progress as additional projects are added.

    However, while it’s clear that many organizations have adopted multiple DevOps tools and platforms, far too many of them still have a long DevOps journey ahead. Each organization will need to decide whether DevOps workflows need to be adopted to fit their needs, even as the pressure to build and deliver software faster continues to mount. In fact, at a time when more organizations than ever are dependent on software to drive mission-critical digital business transformation initiatives, time may be running out to adopt DevOps workflows at a deeper level if organizations hope to stay competitive.

  • GitHub Revamps Search Engine for Code Repository

    GitHub Revamps Search Engine for Code Repository

    GitHub today unveiled a new search engine it has developed to make it simpler and faster to browse and navigate the world’s largest code repository.

    A completely redesigned search interface now provides suggestions, completions and the ability to slice and dice results, while a redesigned code view tool has integrated search, browsing and code navigation.

    Mario Rodriguez, vice president of product for GitHub, said the popular code repository provider decided to create a new search engine to improve developer productivity. There are more than 200 million GitHub repositories that can now be more easily explored, he noted. The updated search engine also makes it much easier to search for specific vulnerabilities that might exist in a codebase, added Rodriguez.

    This latest addition to the GitHub portfolio is part of an ongoing effort to inject more intelligence into software development using tools such as GitHub Copilot that employ generative artificial intelligence (AI) to make developers more productive, he said.

    During uncertain economic times, there’s always going to be more focus on productivity. Organizations of all sizes are reluctant to hire additional full-time developers, so any effort to make existing developers more productive is welcome. The challenge is the DevOps teams that support those developers are now under increased pressure to reduce or eliminate as many bottlenecks as possible. In most instances, that will require increased investments in automation and AI to enable greater amounts of code moving through multiple pipelines to be managed at an unprecedented scale.

    In the meantime, instead of focusing on metrics such as recovery times, DevOps teams might want to focus more on how much they are improving developer productivity. After all, developers are an expensive asset, so in the age of digital business transformation, most business executives are squarely focused on increasing the rate at which applications are being built and deployed. In many cases, the amount of profit and revenue being generated by any business is now a direct reflection of the pace at which quality software is developed and deployed.

    One way or another, a new age of software development is dawning. Most of the innovations being brought forward are focused on issues that impact developers, but it’s only a matter of time before AI is also increasingly applied to DevOps workflows. As more organizations begin to realize how much of a bottleneck manual steps in DevOps processes have become, the willingness to invest in more advanced platforms should increase.

    The challenge, as always, is there are a lot more developers than software engineers, so the immediate focus is on developer productivity. That’s not necessarily a bad thing, but in the absence of any advances on the back end of the software development process, it is likely to aggravate an imbalance that most DevOps teams strive daily to correct.

  • FIDO/WebAuthn Passkeys is Inevitable: Get on the Train ¦ IBM CEO Hates WFH

    FIDO/WebAuthn Passkeys is Inevitable: Get on the Train ¦ IBM CEO Hates WFH

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The Passkeys authentication standard gets a huge boost, and IBM’s Arvind Krishna wants workers back in the office. (more…)

  • Survey: Most Open Source Software Maintainers Want to Get Paid

    Survey: Most Open Source Software Maintainers Want to Get Paid

    A survey of 339 maintainers of open source software projects found 60% of them classified themselves as unpaid hobbyists versus only 13% who said they earn most or all of their income from maintaining projects. Slightly less than a quarter (23%) described themselves as semi-professionals, earning some of their income from maintaining projects. Overall, more than three-quarters of unpaid maintainers (77%) would prefer to be paid for their efforts.

    The survey was conducted by Tidelift, a provider of a platform for managing open source software use, and found there is a clear correlation between getting paid and the amount of time spent actually maintaining a project. A full 81% of professional maintainers spend more than 20 hours per week maintaining their projects, compared to 27% of semi-professional maintainers and only 7% of unpaid hobbyist maintainers, the survey finds.

    In fact, paid maintainers are much more likely to have reproducible and verifiable build processes (77%), formal backward compatibility policy (71%), a security disclosure plan (69%), provide fixes and recommendations for vulnerabilities (69%) and have a defined dependency management process (57%).

    Unfortunately, the survey also made it clear that more than half of maintainers of open source software projects (52%) are not even aware of emerging frameworks to better secure software.

    On the plus side, however, the survey also found that among maintainers that are aware of those frameworks 43% have already begun work to align with them or plan to start within the next year. Alas, 39% said they have no plans to align to these industry standards, while another 19% are still on the fence. Well over one-third of maintainers (38%) who do not plan to align their projects with industry standards said they just don’t have the time, while 37% won’t do it because they are not being paid for the work.

    Nearly half of maintainers (47%) want to be paid for undertaking the work needed to align their projects with the security frameworks, with 54% of maintainers noting they would appreciate help that would enable them to better understand these frameworks and how they apply to their project.

    Tidelift CEO Donald Fischer said as more organizations are concerned about the security of the open source software used across software supply chains, the survey makes it clear that maintainers need some sort of financial compensation to make it worth their time and effort to address vulnerabilities in a timely manner. That’s especially critical, especially as legislation is pending that would hold software developers more liable for vulnerabilities in software. In the absence of being paid to more proactively address vulnerabilities, many open source software maintainers may abandon projects altogether because of liability concerns.

    It’s not clear to how enterprise IT organizations are evaluating which open source projects developers can use based on the level of support made available. What is clear is that not all open source software is created and supported equally well. Of course, many of the organizations benefitting from open source software could be doing a lot more—not just in terms of financial support, but also helping with everything from identifying vulnerabilities and testing patches to simply writing better documentation.

  • Breaking Down Silos: Applying Open Source Practices in the Workplace

    Breaking Down Silos: Applying Open Source Practices in the Workplace

    DevOps plays a prominent role in mainstream technology and companies’ ability to deliver results. While the definition of DevOps encompasses the complete range of characteristics it covers, in practice, current tech culture tends to focus solely on the tooling aspect, neglecting the equally essential focus on the people and process critical in delivering value to end users. While cultures are difficult to change and redefine, a transformation is needed to fully capitalize on the potential DevOps has to accelerate software development.

    Conway’s law–the concept that organizations ship software based on their organizational structure–outlines the need for visibility. This law revolves around the idea that in order for a software product to successfully operate, its developers must have established open pathways of communication to ensure each component works as a whole rather than as individual pieces. Unfortunately, today, many organizations still run into problems with how their teams operate when individuals own limited pieces of development.

    Take a retail platform, for example. In the development of the platform, there will likely be several sub-teams that focus on individual components of the design. Perhaps there’s a team for the shopping cart service, another for the catalog service, a team for the underlying data platform and one for user account services. The list goes on. Typically, those teams do not have visibility into each other’s work and operate in their own silo. As soon as teams start thinking about ‘their’ component or ‘their’ microservice, a dangerous tunnel-vision takes hold. In some cases, there may be standup meetings across teams for coordination efforts, but this takes time, energy and steals focus from the important task: Delivering value to end users.

    Alternatively, think about your favorite open source project. The inner workings were likely  designed asynchronously, openly, remotely and at a global scale. Given the remote nature of open source innovation, developer teams don’t need to be co-located in the same workspace. While physically operating separately, open source developers have mastered alternative methods to operate with transparency in such a way that external contributions aren’t just welcome, but encouraged.

    This same idea applies to proprietary software development, or InnerSource, a modern engineering approach that applies open source practices inside your organization to proprietary code. While a technological shift in practice, InnerSourcing is also a cultural revolution toward collaboration.

    Companies of all types can benefit from implementing InnerSource practices. Some enterprise organizations that operate using InnerSource methodologies today include 3M, Ford, KPMG, Mercedes-Benz, Nubank, Spotify and more.    

    A critical component in making this change is prioritizing team visibility. If other teams in your organization are unable to see what you’re working on, you’re inherently (intentionally or not) creating silos and limiting efficient collaboration. In practice, there will be scenarios where company-wide visibility may not be feasible due to external regulations, highly secure environments or sensitive projects; however, these moments of limited project visibility should be the exception rather than the default.

    When approaching InnerSource through an open source mindset, you will find that many DevOps principles still hold true. Here are a few strategies for implementing a collaborative open source culture when developing proprietary code:

    1. Use existing resources: When problem solving, using the existing materials available within your organization is a great place to start—it is possible someone has already tackled the issue you are addressing. If you find the right fit, you can reuse prior work and, if it has not been started, this is an opportunity to add value to your organization by creating a project that could help others in the future. When building upon someone’s work or starting a project, prioritizing visibility by providing an explanation of any technical requirements of the change will allow future project maintainers to determine whether it’s in the project scope, already a proposed change, on the backlog, or not a prioritized piece of work. This use of existing resources will help develop a collaborative culture over time.
    2. Documentation of work: Documentation is critical. Keeping track of the expected standards of the project and which tools or dependencies are required to build the project for local testing will help others in your organization know what to contribute. When in the early stages of contributing code to a project, documenting proposed changes without directly impacting the production version will allow you to receive feedback early on from the project maintainer about the direction you should take as they will be able to see your steps clearly outlined. To contribute code to a project, you can propose changes as a pull request, which doesn’t impact the production version but means you can get valuable feedback on the direction the project should take. Handling contributions by a pull request also ensures that branch protection rules apply and continuous integration practices are required.

    While implementing these practices may not result in immediate changes, over time they will result in a significant cultural shift. And while cultural transformations require continued conscious efforts, embracing modern software engineering practices–learned from the open source community–will enable teams to break barriers for organizational collaboration.

  • Open Source Communities Need More Than Funding From Big Tech

    Open Source Communities Need More Than Funding From Big Tech

    In recent years, major players like Microsoft, Amazon and Google have increased their involvement in open source projects, providing both financial and technical support. More consistent contributions from big tech—whether in the form of people or resources—is exactly what the open source community needs. However, underdeveloped collaboration guidelines can lead to friction between large corporations and open source developers, which ultimately harms both parties. Friction jeopardizes tech companies’ ability to rely on open source for operations while leaving the open source community without the support it needs. To make their relationship more sustainable, both parties need to cultivate a greater level of trust, develop clear guidelines for collaboration, and prioritize community.

    Big Tech Contributions: Moving From Cash to Collaboration

    For years, major tech companies have used open source solutions in their tech stacks. But many open source developers—who receive little to no money for their work—have voiced concerns about companies that take more from the open source community than they give.

    In many ways, big tech has answered the call. For example, Amazon recently passed IBM as the fifth-largest contributor to open source projects. Companies like IBM and Amazon have a vested interest in supporting these projects—if open source projects are understaffed and underfunded, the quality of the solutions suffers.

    But many tech companies are grappling with an unstable financial environment, which makes hefty donations more difficult to justify. As a result, support from big tech often involves donations of their developers’ time. However, this is also challenging as tech companies like Twitter and Google continue to lay off the developers who were keeping many open source projects afloat.

    Yet, even before recent layoffs, collaborations between big tech and open source teams were strained. There’s a natural power imbalance between a corporation like Amazon and the open source community, which includes members who contribute to these projects in their free time, sometimes for little or no pay. Big tech developers must juggle the interests of the open source team they’re collaborating with and their own employer—with little guidance on how to do so.

    A lack of transparency and communication from big tech also puts open source developers in a tricky situation. Many developers need financial and personnel support but may not want to relinquish control of their solutions to an outside corporation.

    To alleviate this tension, big tech needs to establish clear, mutually beneficial guidelines for open source collaboration that prioritizes sustainability.

    Reconciling the Fraught History Between Big Tech and Open Source

    Past tensions between big tech and the open source community can affect collaboration if left unaddressed. For example, Microsoft has become one of the biggest contributors to the Linux kernel, but many open source teams still remember former Microsoft CEO Steve Ballmer calling Linux a “cancer” in 2001.

    To overcome this tension, big tech companies need to clarify their intentions from the beginning of a project. They should disclose their reasons for selecting the project, the developers who will work on that project and how their contributions will look. Open source maintainers also have a role to play here. They must properly vet these companies and inquire about their prior open source experience and strategies before welcoming them into the fold.

    One way to reduce the risk of power imbalances or other tension is by collaborating through a project like the Linux kernel. Google, IBM and many other corporations contribute toward Linux kernel, and this distribution of contributors creates a much more balanced playing field. Rather than one company assuming operational control, multiple companies play a small part in keeping the Linux kernel up and running.

    The Benefits of an Open Source Program Office

    While these guardrails certainly help, big tech companies also need to formalize their open source contribution processes. At Aiven, we’ve done this through our open source program office (OSPO), which helps us manage our relationships with the open source communities we rely on. OSPOs ensure organizations remain compliant with open source licensing and provide a foundation to expand their involvement with the community over time.

    When developing an OSPO, tech companies should be intentional about how they measure success. The most obvious metrics relate to the work developers do—for example, the number of features delivered, the number of fixes and the number of projects they contributed to. But OSPOs should also measure how developers contribute to the community itself, not just the projects within it. For example, do developers review work from other developers in the community? Do they attend conferences, write blog posts and share knowledge with other developers?

    Big tech leaders must place as much emphasis on community-building as they do the number of contributions developers make. If they don’t evaluate developers based on their engagement with the community, the developer is less likely to focus their efforts there, injecting toxicity into the communities they’re trying to support.

    Open Source Thrives on Collaboration. Let’s Keep it That Way

    The challenge of reducing the stress and burnout of open source developers can’t be solved solely through financial means. No amount of money will reduce the amount of work it takes to maintain an open source project.

    To make collaborations between open source teams and big tech sustainable, both sides need to prioritize supporting the community above all else and develop guidelines that support this mission. More sustainable collaborations will ensure that the open source community gets the support it needs while big tech continues to reap the rewards of the community’s innovations.

  • Linux 6.3: What’s New ¦ AWS Layoffs are a Worry

    Linux 6.3: What’s New ¦ AWS Layoffs are a Worry

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: A new Linux kernel drops and layoffs at Amazon Web Services point to trouble. (more…)

  • InfluxData Makes Processing Observability Data at Scale More Efficient

    InfluxData Makes Processing Observability Data at Scale More Efficient

    InfluxData today made available an update to its open source time series database that can now analyze metric, event and trace data in a single datastore with unlimited cardinality in terms of how they are aggregated.

    The company is now making available a single-tenant instance of InfluxDB as a managed service alongside its already-existing multi-tenant cloud service that is based on a serverless architecture.

    Finally, InfluxData also announced that later this year it will make available InfluxDB 3.0 Clustered and InfluxDB 3.0 Edge to provide a curated version of the database that organizations can deploy themselves where they best see fit.

    InfluxData CEO Evan Kaplan said InfluxDB has been developed to support a wide range of emerging applications that require access to time series data, including the observability platforms core to DevOps workflows that require visibility into metrics, events and trace data.

    Over the past three years, InfluxDB has been revamped to run on a columnar engine, dubbed IOx, that leverages the open source Apache Arrow memory format and written in the Rust programming language. Kaplan said that approach makes it possible to continuously ingest, transform and analyze hundreds of millions of time series data points per second without limitations.

    At the same time, InfluxDB takes advantage of high compression object storage to reduce the total cost of storing all that data. It also provides interoperability with Open Data Architecture (ODA) to integrate with data lakes based on open source platforms such as DataFusion, Flight SQL and Parquet that are being advanced by the Apache Software Foundation.

    The arrival of version 3.0 of InfluxDB comes as many DevOps teams are starting to struggle with the amount of observability data being generated. DevOps teams want to be able to move beyond monitoring a set of pre-defined metrics and query data in a way that enables them to surface anomalies indicative of a potential issue before there is a major disruption to an application service.

    One of the major challenges today is the tradeoff between how much data is collected and analyzed versus the cost of processing and storing it. InfluxDB provides a mechanism for analyzing high cardinality data involving metrics, events and traces cost-effectively.

    Observability is, of course, only one of several use cases for a time series database capable of processing data in near-real-time. As organizations embrace digital business transformation initiatives, they need to be able to process data in near-real-time at the point where it is being created and consumed. Those applications won’t necessarily replace existing applications based on batch-oriented processing but, over time, they create two distinct classes of applications that process data in fundamentally different ways. The challenge, as always, will be defining the DevOps workflows required to manage applications running on multiple distinct types of architectures.