devopsy.kinsta.cloud

Tag: data

  • Sumo Logic Eliminates Ingest Fees for Observability Log Data

    Sumo Logic Eliminates Ingest Fees for Observability Log Data

    Sumo Logic this week announced it will no longer charge a fee for ingesting log data into its observability platform to encourage DevOps teams to apply analytics more deeply.

    Michael Cucchi, vice president of product marketing for Sumo Logic, said the Sumo Logic Flex Licensing plan eliminates what amounts to a tax for using an observability platform. Flex Licensing is being made available immediately to new customers and will be offered to existing Sumo Logic customers later this year. There are no hidden monthly charges, feature restrictions, performance tradeoffs or user limitations being applied under the plan.

    The licensing plan only applies to log data, but DevOps teams will be able to ingest all structured, semi-structured and unstructured data for a lower cost that Sumo Logic is working to store more efficiently, said Cucchi. The overall goal is to make available a disruptive approach to licensing that lowers the total cost of observability, he added.

    Storing log data is going to be increasingly critical as organizations begin to apply multiple types of artificial intelligence (AI) models to further automate IT operations, he added. Those AI models promise to make observability platforms more accessible because algorithms will be able to surface issues that need to be addressed. Historically, the value of an observability platform has been tied to how proficient a DevOps team became when using its query language.

    IT teams that have adopted observability platforms have been, to varying degrees, limiting the amount of data they collect and retain to keep storage costs from exploding. The challenge is that as DevOps teams deploy more microservices-based applications, the amount of log data generated has exploded. Determining the root cause of any issue involving those applications can be problematic if log data isn’t readily available. Log data is, after all, the atomic unit for observability, noted Cucchi.

    In general, IT environments are becoming too complex for humans to manage without the aid of observability platforms augmented by AI models, noted Cucchi. Each application environment is fairly unique, so it is pivotal for those AI models to be exposed to as much data as possible to ensure optimal results.

    It’s not clear whether observability platforms might one day replace the need for many of the monitoring tools that IT teams rely on today to track a set of pre-defined metrics. However, one thing that is certain is that observability platforms, at the very least, present an opportunity to rationalize some of those tools. The biggest challenge is finding ways to fund the acquisition of an observability platform in the first place, which then creates an opportunity to consolidate those tools later.

    Given the complexity of highly distributed computing platforms, it’s only a matter of time before most IT teams have some type of observability capability in place. Of course, observability has always been a core DevOps tenet. The trouble is that the tools being used to achieve it are legacy monitoring platforms that don’t provide the level of depth required to successfully manage modern application environments.

    Image source: Joshua Sortino via Unsplash.

  • JFrog Extends MLOps Integration Efforts via Qwak Alliance

    JFrog Extends MLOps Integration Efforts via Qwak Alliance

    JFrog today announced it has integrated its DevSecOps platform with a managed machine learning operations (MLOps) platform from Qwak to advance collaboration between teams building and deploying multiple classes of software artifacts.

    This alliance comes on the heels of a similar Amazon Sagemaker alliance announced last month that also integrated a managed service for building artificial intelligence (AI) models provided by Amazon Web Services (AWS) with the JFrog Software Supply Chain Platform.

    Both machine learning operations (MLOps) platforms provide data science teams with a complete stack of curated tools required for building AI models from the ground up versus, alternatively, customizing AI models that were previously built.

    Gal Marder, executive vice president for strategy for JFrog, said integration with the Qwak platform makes it possible to manage the software artifacts created by MLOps teams alongside the rest of the software artifacts a DevSecOps team is already managing. That approach makes it possible to also detect and block the use of malicious ML models in addition to ensuring models comply with company policies and regulatory requirements, he noted.

    As it becomes apparent that more AI models will be directly embedded within applications, the need to integrate DevOps workflows with machine learning operations (MLOps) platforms used to create AI models is becoming more pronounced, said Marder.

    JFrog Software Supply Chain Platform can be used to provide both data scientists and developers with a single source of truth for securely managing software artifacts using a common repository to foster greater collaboration between teams that, in most cases, are still determining how best to collaborate, he added.

    The challenge is data scient teams typically train and deploy AI models every few months, while DevSecOps teams are often updating applications multiple times a month. Data science and DevSecOps teams, as a result, generally have distinct cultures, but in the long term, DevOps and MLOps workflows will eventually merge, said Marder.

    Most organizations are still trying to determine how best to operationalize AI using their own data. In the term, AI models are being invoked via application programming interfaces (APIs). Still, it’s also now only a matter of time before more AI models are embedded in applications to improve overall performance. The challenge is that AI models can’t be updated the same way other software artifacts are patched, so managing versions of AI models will require a different set of controls as one model replaces another. JFrog, for example, has developed versioning capabilities that can be applied to AI models in the context of a DevOps workflow.

    There is, of course, already no shortage of MLOps platforms, so DevOps teams should expect to see a wave of alliances being formed between the providers of these platforms. Less clear is to what degree those alliances might lead to mergers and acquisitions among the providers of those platforms.

    One way or another, however, AI models are coming to DevSecOps workflows. The only issue to be resolved is how best to manage their deployment alongside all the other types of software artifacts already moving through existing DevOps pipelines.

  • Blueprints for a Secure, Future-Proof Hybrid Cloud

    Blueprints for a Secure, Future-Proof Hybrid Cloud

    For today’s organizations, hybrid environments are a given. Whether for legacy reasons, regulatory requirements, or other specific needs, some of an enterprise’s critical data is likely to reside on, or in some cases, revert to, on-premises. However, other equally crucial data might be more efficiently and cost-effectively suited for storing and accessing in the cloud. Hence, hybrid.

    Despite their logic and many benefits, hybrid environments bring new levels of complexity. And they bring greater vulnerability, with ever more devious cyber threats probing the multitude of network access points. More than 80% of breaches now involve data stored in hybrid environments, according to IBM’s Cost of a Data Breach 2023 Report.

    That’s why, to avoid data loss and maintain a comprehensive level of protection across the entire environment, organizations need a blueprint for adopting and evolving their hybrid systems. The right solution enables data protection across workloads, providing the hybrid enterprise with the flexibility to protect data wherever it is today — and wherever it might be in the future.

    A growing multitude of organizations in various industries have successfully implemented blueprints of this sort, built on the following foundational considerations:

    Cost-optimization: A solution that can span a hybrid environment — SaaS, software and on-premises — and reduce infrastructure to manage and save administrative time.

    Air-gapped security: Backup of the most critical data in cloud locations that can’t be accessed through whatever network or networks might be the focus of a cyberattack.

    Fast recovery: Flexible, performance-optimized storage combined with the ability to send an air-gapped copy to the cloud for ransomware protection.

    Broad coverage: An integrated set of software, appliance and SaaS offerings that span hybrid environments — and can evolve with the organization.

    Simple management: Single-pane-of-glass dashboards for monitoring the hybrid environment while reducing management overhead.

    Reliability for peace of mind: Leveraging industry-leading, trusted platform solutions rather than unproven point products.

    Now’s the time to see it in action.

    Ensuring an Insurer’s Data Backup

    In Israel, Harel Group is the country’s top insurance and financial services provider, managing a large volume of data and investments. As such, a critical priority for the company is to ensure all that data is 100% backed up and can be restored without any issues.

    Harel Group were using a secure solution to reliably protect more than 1.5PB of data in its on-premises environment. More recently, though, it has sought the flexibility and efficiencies of moving part of its infrastructure to the cloud. However, stringent industry regulatory requirements meant Harel could store sensitive data in the public cloud only if it could be backed up and stored as efficiently, securely and reliably as the on-premises data. What they needed was a SaaS solution.

    “A SaaS solution takes care of data availability, but not the backup operation of the data,” said David Ben-Eli, system IT infrastructure manager for Harel Group. “We knew from day one that in taking on Microsoft 365, we would need a SaaS solution with good restore functionality that satisfies our business model and strategy.”

    And so, Harel designed a blueprint based on a fully managed SaaS platform. The initial stage involved protecting approximately 1,000 of its employees’ mailboxes, which will soon cover more than 5,000 mailboxes.

    The SaaS solution also enabled Harel to move its on-premises storage securely to the cloud, which has the dual advantage of reducing infrastructure costs and air-gapping its backups to mitigate the risk of ransomware attacks. “We have peace of mind that our data is secure,” Ben-Eli said. And the entire cost of the SaaS solution “is less than the cost of hiring more people.”

    Backing up and protecting a company’s data in a hybrid environment is challenging enough. To be a vendor providing those capabilities to other companies comes with additional challenges.

    A Vendor Puts its Data in Order

    Evolutio, based in Madrid, is a leading enabler of cloud services that provides more than 5,600 kilometers of deployed networks with 6,100 virtual machines across three data centers. Its goal is to support customers’ digital transformations and foster their agility, flexibility and innovation capacity to optimize their business value.

    To meet Evolutio’s needs, Alain Rodriguez, head of services management office, had to find an easy-to-use solution that consumed minimal resources to manage backups and security for Microsoft 365. His blueprint aimed to protect critical data before anything could go wrong.

    So, Evolutio chose a single platform to back up and restore data for internal systems. By implementing a fully hosted SaaS-based solution, Evolutio no longer needs to worry about purchasing and maintaining equipment yearly. There is also no upfront investment required and no additional costs for support from the vendor. As a result, Evolutio’s business can grow on demand.

    Resolving a Global Data Sprawl

    Woodward, Inc. is an American designer, manufacturer and service provider of control systems and components for aircraft engines, industrial engines and power systems. The company has been growing significantly across dozens of global networking sites, with expansion that quadrupled the size of data and backed-up servers.

    Virtual machine sprawl ate into company resources while complicating backup operations requiring standardized procedures. Slow legacy servers and apps made it impossible to meet recovery point objective (RPO) and recovery time objective (RTO) targets. And the absence of a coherent, comprehensive disaster recovery plan worsened things.

    Topping this off was increased backup storage costs via disk and tape and a need for central management or cross-department visibility into data protection and backup policy compliance. There was no blueprint, in other words.

    After trying unsuccessfully to address its needs with various disparate solutions, Woodward decided to replace everything with a comprehensive cloud-based backup and recovery platform. The company switched all global sites to this solution. As a result, Woodward has reduced the overhead of managing their data protection, can consistently enforce backup, restore and retention policies and has unprecedented comprehensive visibility and reporting across the end-to-end environment.

    “I used to cringe when I received a restore request since backups would frequently fail,’’ said Loren Leveque, senior manager of IT and system networking. With the cloud-based platform, he said, “I am ready to support the backup, restore and archive needs of our company.”

    Supporting Dozens of Manufacturing Plants

    Linamar Corporation is a diversified global manufacturing company and world leader in designing, developing and producing highly engineered products. Because the Canadian company supports 61 plants globally, a cloud approach became necessary to streamline efficiency in backup and recovery.

    Linamar had recently shifted from Microsoft Exchange on-premises to cloud-based Microsoft 365 for its 7,000+ users worldwide, and the company’s tape-based backup system didn’t support it. The built-in Microsoft 365 data protection was a start, but it couldn’t handle the rapidly expanding demands imposed by Linamar’s global network.

    By moving to a cloud-based backup and recovery platform supporting Microsoft 365, Linamar saved the costs and administrative time previously needed for managing tapes. And backup is no longer a full-time job – it’s more like “when you need it, set it up,” said Kristian Smith, Linamar’s manager of technical services and support. Restores can now be handled 100% within the team and take hours to execute, not weeks.

    Bringing it All Together

    The examples might be as varied and numerous as enterprises worldwide with crucial data to protect. But these and any other success stories all have at least one thing in common: They took a blueprint approach to ensuring that their hybrid environment is secure and sufficiently future-proof.

  • Scrimmage for Control: On-Premises Vs. SaaS in the Observability Super Bowl of 2024

    Scrimmage for Control: On-Premises Vs. SaaS in the Observability Super Bowl of 2024

    Observability means data. It encompasses a set of practices and tools designed to track the performance and health of software applications, providing insights into the “why” behind system behaviors, not just the “what.” At its core are data pillars we’ve learned how to work with and hopefully get the most out of—metrics (quantitative data on system performance), logs (recorded events within the system) and traces (detailed views of request paths through the system) that offer a comprehensive view of the system’s operation in real-time. SaaS is where the industry has been turning towards for the past decade. It’s easy to operate and upgrade and takes away maintenance and manpower requirements.

    But, using a SaaS solution where data is stored on the vendor’s infrastructure can be compared to playing a football game at your opponent’s home field. Your team does not own the field and must play along with conditions that are outside their comfort zone. While your team can still strategize, execute plays and score points, the control over the playing environment, including the surface, lighting and even some rules, lies with the owner of the stadium (in this case, the SaaS vendor).

    You don’t get the home-field advantage and the comfort of your familiar surroundings, and you also can’t control the audience’s shouts and cheers in the stands across the arena.

    Just as a visiting team must trust the host’s management of the stadium and adapt to its conditions, organizations using SaaS solutions entrust their operational data to third-party vendors. They must trust its data protection policies and privacy agreements to ensure that the organization’s data is handled securely and in compliance with relevant regulations.

    And even more importantly, they have to adhere to its hosting costs.

    Let’s Leverage the Home-Field Advantage

    The alternative is clear: While playing on your own field, you, the coach—hell, the entire team—know this field by heart, and you know best how to keep and enforce your own rules.

    Just as a football team controls the ball and decides on the play, having on-premises data means a company or organization keeps its data within its infrastructure, offering direct control over its storage, management and security. The organization decides how to deploy, use and protect this data and even how to manage the tradeoff of data costs.

    The downsides? Playing on your home field at all times means you’re running slower than the world out there. You’re not quick to update your software to the newest common practices and you’re tasked with maintaining another stack—your observability stack—on top of building your product, which serves your users.

    There’s Another Way. Play the Data on the Scrimmage Line

    In football, the scrimmage line is the imaginary line across the field where the ball is placed at the start of each play. This line serves as the point of engagement between the two opposing teams, with specific rules governing how players can position themselves and how the play begins.

    What if we can keep the data on-premise, in the home field, a little like having the ball on our side of the scrimmage line, within our team’s possession and control? We have the ball, and we decide and strategize how to advance the ball or score from the position behind the scrimmage line, but we can still interact with the outside world (or the opposing team) under a set of rules to which we both adhere.

    This is exactly where observability solutions are heading: A combination of the advantages of SaaS—easy to update and maintain value, with an easy way to distribute the solution inside the organization, alongside the advantages of on-premise—data privacy, data control and full transparent and controllable data costs.

    I call this approach inCloud. It is a modern on-premises approach—data in your cloud environment, protected and controlled by the user, but still governed, managed and served with a guaranteed SLA just like the best SaaS offerings out there.

    Just like the scrimmage line in football that provides a clear boundary and set of rules for how the game is played, inCloud acts in a similar way. Data is kept on-premises, allowing the organization full control over retention, privacy and data integrations. And on the other hand, inCloud defines a set of strict rules under which a control plane is allowed to operate and pass the scrimmage line, or past the organization’s cloud borderline, to manage and access the data like a SaaS solution does best.

    So when we sit down to watch the Super Bowl this Sunday—let’s use that halftime show to think about where we’ll be taking our organization’s observability capabilities in 2024. Will the data be in our hands, crossing that invisible scrimmage line? Or will we lose to our opponent’s home-field advantage?

    May the best team win.

  • How Can DevOps Teams Use Customer Intelligence Data?

    How Can DevOps Teams Use Customer Intelligence Data?

    Customer intelligence involves collecting and analyzing data about customers’ behaviors, preferences and needs. It helps businesses understand their customers better and tailor their products and services to meet customer demands. The data is gathered from various sources, such as customer feedback, web analytics, social media, purchase history and software usage metrics. The insights drawn from customer intelligence can be used to inform business strategy, product development and marketing efforts.

    The goal of customer intelligence is to create a detailed understanding of the customer’s needs, preferences and habits. This information can be used to predict future behavior, improve customer service and drive sales. For instance, if a software company knows that a particular customer segment mainly accesses their software on mobile devices, the organization can improve their mobile support or create dedicated mobile applications for that customer segment.

    How Can DevOps Teams Use Customer Intelligence Data?

    There are several ways that a DevOps team can leverage the data gathered from customer intelligence.

    Prioritizing Feature Development Based on Customer Feedback

    When developing new features, it’s useful to analyze customer feedback to understand what features customers find most valuable and prioritize those for development. For instance, if feedback suggests that customers want a more intuitive user interface, the team can prioritize UI improvements.

    By focusing on the features that customers find most valuable, DevOps teams can improve the usability and functionality of their products. This can lead to higher customer satisfaction and loyalty and, ultimately, increased sales. Additionally, prioritizing feature development based on customer feedback can help DevOps teams avoid wasting time and resources on features that customers don’t value.

    Tailoring UX and UI Design Based on Customer Preferences and Behaviors

    By analyzing data on how customers interact with their products, teams can identify pain points and areas for improvement. For example, if data shows that customers are having difficulty navigating the website, the team can redesign the navigation to make it more user-friendly. 

    Alternatively, if customers abandon their shopping carts at a certain point in the checkout process, the team can investigate and make necessary improvements.

    Using Customer Data to Drive Personalization and Customization Features

    Personalization is about tailoring the customer experience to the individual user, while customization is about giving customers the ability to tailor the product or service to their specific needs.

    For instance, by analyzing data on a customer’s browsing and purchase history, a team could develop a feature that recommends products based on the customer’s past behavior. Or, if a customer frequently purchases a certain type of product, the team could create a feature that allows the customer to customize that product to their liking.

    Leveraging Predictive Analytics to Forecast Future Customer Needs and Trends

    Predictive analytics involves using historical data to predict future events. In the context of DevOps, this could mean using customer data to forecast future customer needs and trends.

    For example, if data shows a growing interest in eco-friendly products, the team could anticipate this trend and start developing more eco-friendly products. Predictive analytics can also help teams identify potential issues before they become problems, allowing them to address those issues and improve the customer experience proactively.

    Best Practices for Effectively Using Customer Intelligence in DevOps

    Here are some best practices that DevOps teams can use to make the most of their customer intelligence data.

    Integrate Customer Feedback Early and Often

    Customer feedback from surveys, social media, and customer service interactions provides invaluable insights into what customers need, their pain points, and their preferences.

    By integrating this feedback into the DevOps process, teams can better understand customer needs and develop products or services that address those needs effectively. Early integration of customer feedback allows teams to pivot and make changes more swiftly, thus saving time and resources.

    It’s also important to obtain feedback regularly. Customers’ tastes and preferences evolve continuously, and their feedback reflects those changes. Feeding frequent customer intelligence into the development process ensures the DevOps team’s work stays aligned with the shifting customer landscape.

    Leverage Analytics Tools

    Analytics tools can help DevOps teams sift through large volumes of customer data and extract actionable insights. These insights can then be used to inform decision-making and drive innovation.

    Some tools excel at data visualization, helping teams understand complex datasets through graphs and charts. Other tools excel at predictive analytics, helping teams forecast future trends based on historical data.

    Choosing the right analytics tool depends on the specific needs of the DevOps team and the nature of the customer intelligence data they have at their disposal. The key is to leverage its capabilities fully to extract the most value from the customer intelligence data.

    Ensure Data Quality and Relevance

    Not all data is created equal, and if it is poor quality or irrelevant, it can lead to misguided decisions and wasted resources.

    Data quality refers to the accuracy, completeness, consistency, and reliability of the data. DevOps teams should implement robust data quality checks to ensure that the customer intelligence data they’re using is of high quality.

    Data relevance, on the other hand, refers to the data’s applicability to the task at hand. Not all customer intelligence data is relevant to every DevOps project. Teams need to carefully select the data that is most relevant to their specific project and ignore the rest.

    Balance Quantitative with Qualitative Data

    While quantitative data provides hard numbers and concrete facts, qualitative data offers deeper insights into customer attitudes, perceptions and behaviors.

    Quantitative data can help DevOps teams identify trends, measure performance, and track progress over time. However, it often fails to explain why certain trends are occurring or why performance is changing. 

    It also can help DevOps teams understand the reasons behind the numbers. It can provide insights into why customers behave the way they do and what they truly think and feel about a product or service. Balancing quantitative with qualitative data gives DevOps teams a more holistic view of the customer landscape, enabling them to make more informed decisions.

    Cross-Functional Collaboration

    Customer intelligence data is valuable for all teams across the organization, not just DevOps. By collaborating with other teams, such as marketing, sales and customer service, the DevOps team can gain additional insights and perspectives. 

    Cross-functional collaboration also fosters a culture of data-driven decision-making across the organization. When all teams use customer intelligence to inform their work, the organization as a whole becomes more customer-centric, agile and innovative.

    Conclusion

    Successfully using customer intelligence data in software development allows teams to remain responsive to customer needs and increase the value of their products. DevOps teams need to integrate customer feedback early and often, leverage analytics tools, ensure data quality and relevance, balance quantitative with qualitative data,and foster cross-functional collaboration. By following these best practices, DevOps teams can use customer intelligence data to drive innovation and create products and services that truly resonate with customers.

    Image source: Josh Sortino via Unsplash  

  • IBM Adds Lower-Cost Linux Mainframe to Portfolio

    IBM Adds Lower-Cost Linux Mainframe to Portfolio

    IBM today made its most aggressive move yet in its quest to make mainframes running Linux more affordable, launching an entry-level platform starting at $135,000.

    The IBM LinuxONE 4 Express is a pre-configured rack system that is based on the same 5.2Ghz IBM Telum processor that IBM uses in its Z16 series of mainframes.

    In addition, the platform supports IBM Secure Execution, which is designed to thwart any attempt to decrypt data in the future using quantum computers.

    IBM claims organizations that move Linux workloads from x86 servers to an IBM LinuxONE 4 Express can save over 52% on their total cost of ownership over five years.

    The overall goal is to make it simpler for organizations to consolidate workloads that otherwise would need to be deployed across a distributed computing environment, she added. That approach ensures higher levels of availability for workloads that are increasingly processing massive amounts of data using two levels of physical and two levels of virtual cache in a way that consumes less energy, noted Tina Tarquinio, vice president of product Management for IBM Z and LinuxONE.

    The IBM LinuxONE 4 Express also includes an integrated accelerator for running artificial intelligence (AI) inference engines using more than 6 TFLOPs compute capacity per chip.

    IBM expects that as more organizations process sensitive data to drive AI applications, there will be increased interest in deploying applications in more secure on-premises IT environments, said Tarquinio. Mainframes provide a more cost-effective way to achieve that goal at a time when DevOps teams are embracing methodologies such as platform engineering to centralize the management of workloads, she added.

    At the same time, increased reliance on automation is making it simpler for IT teams to rely on multiple types of platforms to optimally process workloads. That capability enables IT teams to make platform decisions that are based on fit-for-purpose best principles more easily, noted Tarquinio.

    It’s not clear how many organizations will now be able to afford a mainframe when the initial cost of an x86 server is still much lower. But in the face of ongoing economic uncertainty, it seems more organizations are willing to consider all their options. The issue is that adding additional X86 servers is often viewed as the path of least resistance to increasing compute capacity.

    Naturally, each IT team will need to decide what path forward makes the most sense for them, but as workloads continue to become more data-intensive, it’s clear that infrastructure requirements are changing. The more data that needs to be processed, the more challenging it becomes to maintain service level agreements that are based on maintaining specific levels of application performance, noted Tarquinio.

    The challenge now is determining where best to run those workloads across an increasingly hybrid cloud computing environment that continues to offer more options as IT platforms evolve and expand.

  • Dynatrace Extends Data Reach and Scope of Observability Platform

    Dynatrace Extends Data Reach and Scope of Observability Platform

    At its Perform 2024 event today, Dynatrace unfurled a Dynatrace OpenPipeline that makes it possible to apply analytics to multiple types of data sources in real time.

    The company also unveiled a Data Observability offering that can be used to vet the quality and lineage of data being exposed to the Davis artificial intelligence (AI) engine at the core of the Dynatrace observability platform to reduce false positives that otherwise might generate an alert in addition to helping reduce the volume of data that might need to be stored.

    Finally, Dynatrace announced it is starting today extending the reach of its observability platform to the large language models (LLMs) that are used to create generative AI platforms. That capability will, for example, make it simpler to monitor the consumption of tokens that are used to provide access to these models.

    Steve Tack, senior vice president for product for Dynatrace, said Dynatrace OpenPipeline will enable organizations to streamline the collection of data in a way that will enable them to apply observability more broadly by applying stream processing algorithms to petabytes of data.

    Scheduled to be available in 90 days, the Dynatrace OpenPipeline capability enables IT teams to ingest and route observability, security, and business events data from any source and in any format at the point of ingestion, including unstructured data that is automatically converted into a usable format. That data can then be enriched to enable deeper analytics.

    It also provides IT teams with more control over which data they analyze, store or exclude from analytics, which in turn helps reduce the total cost of observability, noted Tack.

    Finally, it presents IT teams with the ability to apply more customizable security and privacy controls to how that data is employed, he added.

    Dynatrace is pursuing a multimodal approach to AI that spans predictive, causal and generative models. Collectively, that approach makes it simpler to identify the root cause of issues, identify anomalies that might disrupt services and streamline incident management by, for example, providing summarizations of events in a natural language format.

    It’s not clear how broadly observability will be applied beyond the data collected to manage DevOps workflows and other IT processes, but it’s clear there is a high correlation between IT events and business outcomes as organizations become more dependent on software. As AI becomes more pervasively employed, it should become simpler to apply analytics to a much wider range of types of data to surface the relationship between IT events and business processes. In effect, Dynatrace is making it simpler to apply best data engineering practices to collect, manage and analyze that data, noted Tack.

    In the meantime, DevOps teams should revisit how their pipelines are currently constructed to make it simpler to capture all relevant data. As critical as telemetry data collected from the DevOps platform is to ensure application development and delivery occurs as fast as possible, that data is only one factor in a larger equation. The challenge and the opportunity now is determining how best to apply AI to correlate it all.

  • 2024 Infrastructure Tech Predictions

    2024 Infrastructure Tech Predictions

    Ganesh Srinivasan, partner at Venrock, co-authored this article.

    2023 was a rollercoaster like none other; from the death of the modern data stack sprawl to the birth of generative AI, we are only at the beginning of a new era in the ‘art of the possible.’ We guarantee 2024 won’t be a disappointment.

    With a new year approaching, it’s the perfect time for us to examine what we anticipate being the biggest developments in the year ahead. Here is what we think is going to happen in 2024:

    1. OpenAI’s Reign Challenged

    With the emerging learnings in core neural net architectures that led to the transformer and dominance by OpenAI, it is likely that their imminent release of GPT5 will be surpassed in specific performance benchmarks by a new entrant on the backs of more efficient architectures, improved multimodal capabilities, better contextual understanding of the world and enhanced transfer learning. These new models will be built on emerging research in spatial networks, graph structures and combinations of various neural networks that will lead to more efficient, versatile and powerful capabilities.

    2. Apple: The New Leader in Generative AI

    One of the most important players in the generative AI space is only starting to show their cards. 2024 will be the year Apple launches its first set of generative AI capabilities, unlocking the true potential of an AI-on-the-edge, closed architecture with full access to your personal data – showing that Apple is actually the most important company in the generative AI race.

    3. Building for Client-First

    The last decade has reflected a shift away from fat clients to server-side rendering and compute. But the world is changing back to the client. Mobile-first experiences will be required to work in offline mode. Real-time experiences require ultra-low latency transactions. Running LLMs will increasingly be required to run on the device to increase performance and reduce costs.

    4. Death of Data Infrastructure Sprawl

    The rapid growth of the data infrastructure needs of enterprises has led to an increasing sprawl of point solutions, from data catalogs, data governance, reverse extract, transform, load, and airflow alternatives to vector databases and yet another lakehouse. The pendulum will swing back to unified platforms and fewer silos to bring down the total cost of ownership and operating overhead going into 2024.

    5. Approaching the AI Winter

    Generative AI in 2023 could be best characterized as the ‘art of the possible,’ with 2024 being the true test to see if prototypes convert into production use cases. With the peak of the hype cycle likely here, 2024 will experience the stage of disillusionment where enterprises discover where generative AI can create margin-positive impact and where the costs outweigh the benefits.

    6. The Misinformation Threat

    While image and video diffusion models have unlocked a new era for digital creation and artistic expression, there’s no doubt that its dark side has not yet taken its toll. With a presidential election in the wings, diffusion models as a machine for political disinformation will emerge to become the next major disinformation weapon of choice.

    7. AI’s Real-World Breakthrough

    Coming out of the ‘field of dreams’ era for AI, 2024 will represent a breakthrough for commercial use cases in AI, particularly in the physical world. Using AI for physical world modalities will unlock our ability to change and engage with the machines and devices around us, from understanding machine telemetry for problem detection to distilling insights across thousands of real-time sensor feeds.

    8. S3: The Growing Threat to Snowflake

    AWS S3 is about to become the biggest competitor to Snowflake. With a rapidly growing ecosystem of tools emerging as the compute layers on top of S3 leverage their ultra-low costs, high availability, Availability Zone support, and new single-digit millisecond latency service, everyone’s ‘Snowflake cost problem’ is about to have a new set of attractive alternatives.

    9. The Rise of Flink and Operational Analytics

    If business analytics drove the first wave of the data infrastructure evolution, operational analytics will drive the next wave, with Apache Flink at the center of it. Enterprises will begin en masse adopting real-time and streaming capabilities to begin using data to drive operational decision making. This will create new value in areas such as logistics, financial transactions, inventory management and fraud detection.

    10. The Developer AI Stack

    Going into 2024, AI’s most significant impact in the digital world will be unlocking developer productivity. From code generation to debugging to build management, AI will unlock a new level of developer productivity that we haven’t seen since the advent of the cloud.

    If you agree, disagree or would like to share your own 2024 predictions with us, please reach out on X at @ethanjb and @gan3sh.

  • KubeCon 2023: Elevating Observability

    KubeCon 2023: Elevating Observability

    Speaker 1: This is Techstrong TV.

    Alan Shimel: Hey everyone. We’re live back here at KubeCon. I hope you’ve been enjoying our coverage today, Mitchell Ashley, our CTO and a GM of Techstrong Research. And I have been kind of splitting the duties. Just we’re that booked with people. I don’t know, there’s over 10,000 people here. I’m going to guess there’s 12 to 15,000 people.

    Alex Burkhardt: Yeah, there’s a lot of people.

    Alan Shimel: … At the event. Maybe more I’ll know more. I’ll find out tonight. But anyway, let me introduce you to our next guest. His name is Alex Burkart. Alex is with Observe Inc. Alex, welcome to Techstrong TV. Thanks for being here with us at KubeCon.

    Alex Burkhardt: Yeah, thanks Alan. I appreciate you guys taking some time to talk to me today.

    Alan Shimel: Absolutely. So let’s start with this. Let’s start with Alex.

    Alex Burkhardt: Yeah.

    Alan Shimel: What do you do at Observe?

    Alex Burkhardt: Yeah, so at Observe I run our technical customer facing team. We call it field engineering. And we work with customers from the first interaction when they’re in the sales cycle to figure out if observes a great solution for them, all the way to success and renewal and helping them realize value from their investment in our partnership.

    Alan Shimel: Okay. I mean, so you’re at that role, but let’s tell a little bit about more your personal journey.

    Alex Burkhardt: Oh yeah. So I started as a computer scientist actually in the Navy.

    Alan Shimel: Really?

    Alex Burkhardt: Yeah. I went to the Naval Academy, got a computer science degree, and they said, that’s great, go drive some ships.

    Alan Shimel: Such is life in the Navy, what do you expect?

    Alex Burkhardt: And then my journey brought me to Silicon Valley after I did some cybersecurity work for the Navy. And what I love more importantly than anything else is helping people find a way to use technology to solve a problem that makes their life better.

    Alan Shimel: Excellent. Now, when did you come to Observe?

    Alex Burkhardt: I’ve been at Observe for more than three years now. Yeah, I was employee 28. I think if my numbers are right, pre customers, in fact.

    Alan Shimel: Really early.

    Alex Burkhardt: I think when I started, we signed our first customer.

    Alan Shimel: Good for, good for you. Good for them. So we don’t have to assume, I know for a fact there are people out here who don’t know, Observe from, have never observed Observe. Tell them. How would you describe what Observe is and does to our audience?

    Alex Burkhardt: That’s a great question, Alan. So at Observe, what we realized is observability is a data problem. So we started by building a data company. What we wanted to figure out is can we bring a solution to market that allows customers to send any data they’d like to correlate it and interact with it in a way that allows them to get more value out of it more quickly. And what we’ve done is we’ve brought a solution to market that unifies the observability tooling into one platform that allows you to send logs, metrics, traces, and any other type of data structure or unstructured that you’d like to keep in your observable universe into one tool, one data store.

    Alan Shimel: Love it. And before we jump into news and stuff, for people who want to just say, hey, I’d like to just see what their website is, what’s the URL?

    Alex Burkhardt: Yeah, our website is www.observeinc.com.

    Alan Shimel: So it’s observeinc.com.

    Alex Burkhardt: Observeinc.com. We are Observe Inc technically. So observeinc.com.

    Alan Shimel: Makes sense to me. All right, we’ve got all that out of the way, Alex, let’s jump in. Now, you guys recently announced some new product feature releases. Why don’t we jump in right there and see what’s that about?

    Alex Burkhardt: Our last release was called Hubble, and Hubble is significant for a couple of reasons. One, naming after the Telescope, one of the things that we want to do is help people elevate their observability. And that came in two specific forms.

    Alan Shimel: I get it. I just got to ask, is the next release called Web?

    Alex Burkhardt: We have considered other satellites, and in fact, we had Erica on our launch. She was actually a NASA scientist.

    Alan Shimel: Very cool. All right.

    Alex Burkhardt: One of the things that we know is very important to our customers is if we bring a solution to market and it’s very, very powerful, is it easy to use and are we taking advantage of the latest and greatest technology that’s emerging in the market? On the ease of use side, one of the things that this Hubble launch and we did, we call it a launch included, was the recognition that when we introduced our logs in our metrics and our tracing explorer, we’ve seen an enormous increase in users engaging in the platform. And we supercharge that by adding one additional layer, which is how we leverage GPT or AI in our platform. And that happens in four specific locations. We have a chat bot that allows you to get guidance on how to use the tool, will actually allow the AI to write any of our query language for you if you’d like to do that. We call that copilot.

    We also have two cool functionalities in terms of making your data more actionable. You can click on a field and observe and ask Ollie or Ollie GPT specifically to explain what an error means and it will actually return an answer that helps you stop having to copy and paste into Google and do outside investigations. The last thing that’s most interesting is we leverage GPT to allow users to separate out their data and do those hard parsing and Reg X challenges that no one likes writing Reg Xs and anyone that has written them knows that regx101.com is their friend, and we wanted to kind of bring that into the product and make that easy for customers to use.

    Alan Shimel: Very cool. Very cool. Now this was released not here at the show, it was released earlier in the month.

    Alex Burkhardt: October 4th.

    Alan Shimel: What’s been the feedback?

    Alex Burkhardt: Well, we’ve more than doubled our monthly active users since we’ve released that.

    Alan Shimel: Really?

    Alex Burkhardt: Yeah.

    Alan Shimel: Since October 4th?

    Alex Burkhardt: Technically since late September. But yes.

    Alan Shimel: Wow.

    Alex Burkhardt: It’s really simple. We listen to our customers and we really care about what they say and how they experience ourselves on our platform. And when we say we’re focusing on ease of use, we mean it.

    Alan Shimel: So the entrepreneur in me just has to ask you.

    Alex Burkhardt: Okay.

    Alan Shimel: What were you waiting for?

    Alex Burkhardt: It’s very easy to make very quick, bad decisions. It’s a lot harder to take a data-driven approach to how to build a product better. And when we approach any problem at Observe, and it’s built into our culture as we first of all figure out this is an important problem, how can I measure the impact I can make on this problem? And that’s how we measured usability. Now for AI, we kind of knew that it was going to be something special and it could help. We’re still figuring out exactly how to measure the impact there, but every decision we make at Observe is grounded in what’s best for our customers and how do we help make the market better.

    Alan Shimel: So how much of this phenomenal growth in such a short period of time do you think is just attributed to riding the AI wave, if you will?

    Alex Burkhardt: That increase in active user growth actually wasn’t tied to the AI release. It was more to the ease of use functionality that we brought. And I think that’s important for any product, but more specifically, one as powerful as ours. When you’re solving big data problems, people get exposed to big data. And one of the things we thought very deeply about is how do we take the experience that a user has to have to get to the outcome they want and narrow that down and give them an obvious path to go through so that they don’t have to deal with all of the problems if they just want to come in and check their logs and see what’s not working. Sometimes it’s the simplest question of, hey, is it easy to do this in the tool? And when we decided ourselves that it wasn’t easy enough, then we started making some really serious changes and that was backed by every single one of our customers agreeing with us back in the active user growth.

    Alan Shimel: Sure. That’s fine. That’s phenomenal, man. Alex, I can’t look for those out here. This is impressive. Let’s talk a little bit about KubeCon here now. I mean, it’s only day one of the expo floor open and everything. What are you hearing from the crowd, from the attendees?

    Alex Burkhardt: Yeah, that’s a great question. This is my second KubeCon. I was new to it last year when we were in Detroit, and I really love this conference, Alan.

    Alan Shimel: Yeah, it’s great. It’s a great community.

    Alex Burkhardt: It’s a great community. It’s a great conference. And frankly, when people come up to our booth and want to talk to us, they want to talk about what we do. There’s not a lot of tire kickers. Everyone that’s here seems to be operators, people that are experienced and steepen the problem. And I love whenever someone comes up and talks to us. I think the other thing I’ve seen in this particular venue specifically, the room is packed. It’s crowded, so crowded that I think that we’re going to have to have a bigger footprint next year.

    Alan Shimel: Observers or the whole?

    Alex Burkhardt: Both.

    Alan Shimel: Yeah. Well, the problem is there’s a pretty big convention center. I mean, short of breaking into sort of a two different exhibit halls, which RSA used to do that. Now it’s one big continuous one. It’s hard. The other thing I will tell you, so we do the North American and the European. So in Amsterdam in the spring, they actually said, look, I think it was 12,000 was the number. At 12,000 they stopped selling tickets. They did a waiting list and a lot of people didn’t get to the show. They were on the waiting list. As anti-business as that sounds, I wouldn’t be surprised if they did something similar.

    Alex Burkhardt: Yeah, that makes sense. I mean, the quality of conversations and the quality of attendee keep the quality super high. And you asked about some of the conversations we’re having when people come by. It’s a variety of things and a couple of themes that I see. I’m curious if other people that have come on the show have said the same, but people continue to talk to us about saving money. They want to do more with less.

    Alan Shimel: That’s the big thing today.

    Alex Burkhardt: [inaudible 00:09:23].

    And I think they also want to figure out, do we need all of the tools? Is there a way we can both get some hard dollar savings and some soft dollar savings. By compressing some tooling and aligning teams we’re also seeing a lot of push and the DevOps teams are getting more involved in security conversations as well. Which I guess-

    Alan Shimel: No, no, that there’s no doubt.

    Alex Burkhardt: As a former security practitioner, if you had asked me five years ago if I thought these two markets would converge, I would not have said yes. Now I think it’s inevitable.

    Alan Shimel: I think it’s already happened, right? So there’s a lot of convergence going on. There’s security with DevOps into what we call DevSecOps. But there’s also, there was a time, and I’ve been going to these conferences now before Covid, so I don’t know, four or five years. There was a time where cloud native was very developer-centric, not as much centric, but it was very distinct from DevOps. There was DevOps and there was cloud native and then there was security. So you look at a company like ours, right? We have devopsy.kinsta.cloud, cloud native now, security boulevard.

    And I always knew there was an inherent overlap between them, but we’ve seen much more than an overlap now. We’ve seen a conversion around this with security being a primary concern of people. And you don’t do cloud native today without doing DevOps.

    Alex Burkhardt: Agreed.

    Alan Shimel: And everything getting pushed out today, especially Greenfield, not Brownfield stuff. It’s predominantly probably 85% or better cloud native.

    Alex Burkhardt: Yeah, I think that’s another thing I’ve seen too as well. Many, many customers are adopting the cloud native terminology, setting goals in those languages and words.

    Alan Shimel: Now, multi-threaded, no doubt. Container based, the whole cloud native thing. And now what I think we’re starting to see and we see, look, you’re naming your company’s Observe. It’s not just the architecture and development being cloud native, but the cloud native aspect of the operations of ops. Forget dev, forget dev ops, ops.

    Alex Burkhardt: It even touches revenue. So one of the things that we see very often is do the nature of how our platform can process data. We take technical telemetry from a customer’s application. We also take business data. Sometimes customers ask us, how can you Observe, help us give our customers a better experience? Well, one of the ways is we can tell you when a particular customer is hurting, when a customer has a challenge or help you reframe all these very deep technical jargon and things that are happening into the context of how does this impact the customer’s experience? How am I able to get proactive? Is what I’m doing in my infrastructure plan having a net positive impact on my customers, on my users?

    And those are questions that previously companies I don’t think they thought they could answer. And when they start working with us, they start seeing a path to that. And I love that. We have some customers even completely away from the DevOps side, that send us shipping data just because they want to know, can I make a support dashboard for my low level support employees so that they can be more efficient with their time and our data? It’s such a cool convergence. I agree with you. I love that word convergence.

    Alan Shimel: It really is. Let’s talk a little bit about observability though too. So over the last two years, so there was a KubeCon in LA during around Covid time that was a little dicey. Then there was the one in Europe, then got canceled. Then there was Detroit, then Amsterdam here. We’ve really seen the observability market just explode, right?

    Alex Burkhardt: Yeah.

    Alan Shimel: Prometheus. What’s the other, I’m drawing a blank. The other big observability open source. OpenTelemetry.

    Alex Burkhardt: OpenTelemetry. Yes. Everyone-

    Alan Shimel: I think of the second and third biggest projects of the gazillion projects in CNCF now. Right?

    Alex Burkhardt: That’s such a good point. I remember when OpenTelemetry was OpenTracing and it was still kind of a fractured project. If you asked us three years ago if we thought that open telemetry would be the way, we weren’t super sure a year into that. So two years ago we were like, yeah, this is the direction. It’s solid.

    Alan Shimel: No doubt about it. You look and you can’t see this at home guys because all you’re seeing is here, but there’s a-

    Alex Burkhardt: All our beautiful faces.

    Alan Shimel: … Big floor out there, but so much of it is based now on this. Really good stuff, really good stuff.

    Alex Burkhardt: I think the other thing, observability as a vernacular is kind of emerging even at the higher level in the business. By definition observability is, can I tell something about a system by looking at its outputs? And one of the things as we’re moving closer to cloud native is more and more systems have outputs and that kind of builds on that data problem I mentioned and why we were founded. So I look forward to seeing how this problem space changes in the next year.

    Alan Shimel: Oh, I think we’re just, look, when you start, you guys are already doing it with gen AI and stuff. You start adding gen AI to this stuff, we start putting more focus on it. We’re in for an interesting times. What’s the Irish proverb? May you live in interesting times.

    Alex Burkhardt: Yes.

    Alan Shimel: Definitely interesting times. Anyway, Alex, thanks for joining us here live at KubeCon. This is Observe Inc. So let me again observeinc.com. Check it out. Hey, they’re doing amazing things. This new release, something that gets twice as many new visitors right? In a couple of weeks, a month. It’s got to doing something right.

    Alex Burkhardt: Yeah.

    Alan Shimel: All right, we’re going to take a break. We’re live at KubeCon in Chicago. I think we have one or two more videos before we wrap for today. We’ll be right back.