Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Instead of AI aids for programming, what about for debugging? The pseudonymous “BioBootloader” says he’s persuaded GPT-4 to make his code self heal. (more…)
IT operations teams, site reliability engineers (SREs) and service providers are on a mission to scale across geographies, expand their digital services and create new experiences for customers. Amid this drive, their backend IT systems are getting more complex. This hinders visibility into applications and makes monitoring and troubleshooting cumbersome. With heightened competition in the market, businesses cannot afford to take hits on revenue or customer experience scores due to issues stemming from their IT stack, such as unknown outages, increased incident management times and data replication.
Infusing artificial intelligence capabilities into IT operations (AIOps) can raise efficiency through automation, improve security and make a significant impact in warding off downtime. AIOps uses machine learning (ML) to enhance IT operations such as performance monitoring, event correlation and analysis. With AIOps, IT teams, SREs and service providers can predict and prevent outages before they occur and resolve them automatically. Moreover, AIOps uses ML to analyze current and past data to make precise predictions, reducing costs and speeding up return on investment (ROI).
Launching AIOps requires a unique approach depending on your organization, its capabilities and needs. This article provides a three-step implementation strategy to help your organization detect incidents before they impact users, automate responses and prevent recurring issues.
Fig: AIOps implementation approach
With the advanced ability to collect and analyze IT operational data, AIOps is an asset for a range of actions and solutions. When implemented effectively, AIOps can enable IT teams, SRES and service providers to significantly reduce application outage costs by over 60% within a span of 24 months.
The AIOps platform can be leveraged by IT teams, SREs and service providers for data gathering, analysis and generation of useful insights. It is designed to enhance operational efficiency, offer predictive alerts, reduce mean-time-to-identify (MTTI) and mean-time-to-repair (MTTR) as well as prevent service outages.
Fig: AIOps platform
Organizations contemplating AIOps as a strategy should first evaluate their current maturity and then chart out a path to make the best use of AIOps, which is automated resolution. There are three levels of maturity in AIOps adoption.
Level 1: Dashboard
Level 2: Alarm & Recommend
Level 3: Automation
Conduct an AIOps workshop with key stakeholders to narrow down the list of apps and prioritize them for evaluation. Apply specific criteria to prioritize applications that meet the required onboarding standards.
Recommendations
To initiate application onboarding, monitoring tools must be set up to collect data for ML models. Predictions from the models can be integrated into visual dashboards for analysis.
Fig: Application onboarding
Recommendations
By implementing AIOps, service providers can successfully reduce the MTTI and MTTR, attaining a 63% reduction in outage costs for applications within 24 months, a 90% reduction in false-positive alerts and a 400+ hours reduction in downtime per annum.
Ramesh Ram (Assistant Vice President), Sreedhar S K (Associate Director), Sakthivel D (Director), and Rohit Karthikeyan (Manager) also contributed to this article.
Artificial intelligence (AI) is revolutionizing the way we develop and test software, especially with the advent of AI chatbots. These intelligent agents have the potential to streamline and automate continuous testing while improving software quality. However, software and QA managers must understand the urgency, opportunities, pitfalls and best practices associated with integrating AI chatbots into continuous testing processes. This blog explains crucial aspects to help your team harness the full potential of AI-driven continuous testing.
• Increasing complexity of software applications: Modern software applications are becoming increasingly complex, which makes manual testing time-consuming, error-prone and costly. AI chatbots can analyze vast amounts of data in real-time, identify patterns and learn from the testing process, making them invaluable for continuous testing.
• Fast-paced development and market demands: Today’s competitive market demands faster release cycles and shorter time-to-market. AI chatbots can help you accelerate testing and development cycles, allowing your team to focus on delivering high-quality software promptly.
• The growing importance of customer experience: AI chatbots can simulate user interactions and predict user behavior, helping to improve user experiences by identifying potential issues before they become real problems.
• Improved test coverage and accuracy: AI chatbots can execute a large number of tests simultaneously, leading to better test coverage and increased accuracy.
• Enhanced defect detection: AI chatbots can identify defects early in the development process, reducing the overall cost and time spent fixing them.
• Efficient resource allocation: By automating repetitive tasks and reducing human intervention, AI chatbots enable teams to allocate resources more effectively, leading to increased productivity.
• Over-reliance on AI: AI chatbots are powerful tools, but they should not replace human testers completely. Human intuition, creativity and critical thinking are essential for effective testing.
• Insufficient training data: To work effectively, AI chatbots require quality training data. Ensure that your chatbot is trained with diverse and representative datasets to minimize the risk of biased or inaccurate results.
• Integration challenges: Integrating AI chatbots with existing systems and processes can be challenging. Ensure that your team has a clear understanding of the integration requirements and potential issues that may arise.
• Define clear objectives: Establish clear goals and expectations for your AI chatbot to ensure that it aligns with your team’s overall testing strategy.
• Choose the right AI chatbot platform: Consider factors like cost, scalability, ease of integration and support for programming languages when selecting an AI chatbot platform.
• Continuously monitor and refine: Continuously monitor your AI chatbot’s performance and make necessary adjustments to improve its effectiveness and efficiency.
• Foster collaboration: Encourage collaboration between human testers and AI chatbots to leverage the strengths of both.
• Improved test coverage and accuracy: AI chatbots can execute a large number of tests simultaneously, leading to better test coverage and increased accuracy. By automating repetitive and complex tasks, AI chatbots can effectively handle edge cases and efficiently test various scenarios, thereby reducing the likelihood of undetected issues in the software.
• Enhanced defect detection and early issue identification: AI chatbots can quickly identify defects and potential issues early in the development process, reducing the overall cost and time spent on fixing them. By leveraging machine learning and pattern recognition, AI chatbots can predict potential problems and bring them to the attention of developers, allowing for proactive resolution and improved software quality.
• Efficient resource allocation and increased productivity: By automating repetitive tasks and reducing human intervention, AI chatbots enable teams to allocate resources more effectively. Human testers can focus on higher-level tasks that require creativity, critical thinking, and domain expertise. This leads to increased productivity, as the team can concentrate on delivering high-quality software and improving user experiences.
AI chatbots are transforming the landscape of continuous testing, offering unprecedented opportunities to improve software quality and speed up development cycles. By understanding the urgency, opportunities, pitfalls and best practices associated with integrating AI chatbots into continuous testing processes, software managers can effectively harness the potential of these intelligent agents and drive their teams toward success.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Google forces apps to make deleting users’ data easier, and the RISC-V drumbeat grows louder. (more…)
New Relic is preparing to extend the capabilities of its observability platform by making use of both additional machine learning algorithms and the ChatGPT generative artificial intelligence (AI) platform.
Peter Pezaris, senior vice president for strategy and experience at New Relic, said during an interview on an episode of TechStrong TV that a prototype of an AI model created using machine learning algorithms will automatically notify developers when code in a production environment is not performing well—right within their integrated development environment (IDE). The notification will then take developers to the precise line of code identified as the root cause of the issue, he said.
DevOps teams will need to add two lines of code to applications running in production environments to take advantage of this capability. New Relic also plans to use a similar approach to integrate with generative AI platforms such as ChatGPT to create suggestions for fixing code, he added. ChatGPT now makes it possible to create entire programs in ways that output HTML and JavaScript code, so DevOps teams should expect to be able to use the platform to create natural language queries that invoke the proprietary query language New Relic makes available for its observability platform, he noted. Conversely, ChatGPT will make it possible to explain how a piece of code functions in plain language, Pezaris added.
Previously, New Relic employed a similar approach to integrate its observability platform with machine learning operations (MLOps) platforms used to build and deploy AI models. This was part of a larger effort to integrate MLOps and DevOps workflows, added Pezaris.
These AI extensions to the New Relic platform are part of a larger effort to extend the reach of the New Relic One observability platform further left toward developers. Most recently, New Relic added a free CodeStream module to its observability platform to give developers access to metrics and telemetry data that will enable them to write higher-quality code faster.
That approach eliminates the need to wait for IT operations teams to surface issues that usually don’t manifest until long after the code in question was originally developed. The overall goal is to provide development teams with frictionless access to observability data at every stage of the software development life cycle to reduce mean-time-to-detection (MTTD) and mean-time-to-resolution (MTTR) of issues.
Organizations should also be able to reduce technical debt faster by making it easier to identify issues that, for example, adversely impact application performance.
In general, most organizations are still in the early stages of achieving full-stack observability. A recent New Relic survey found only 27% of respondents have achieved full-stack observability, and only 5% claimed they have a mature observability practice in place. A third (33%) of respondents also said they still primarily detect outages manually or based on complaints, the survey found.
On the plus side, the survey also found nearly three-quarters of respondents said C-suite executives in their organization are advocates of observability, and more than three-quarters of respondents (78%) saw observability as a key enabler for achieving core business goals. However, more than half (52%) of respondents said they experienced high-business-impact outages once per week or more and 29% said they take more than an hour to resolve those outages.
The hope is, of course, that by augmenting DevOps teams with AI capabilities, the number of those outages will be dramatically reduced in the months and years ahead.
Tabnine has extended its alliance with Google Cloud to advance the adoption of generative artificial intelligence (AI) to automate the writing and testing of code.
The generative AI platform provider has already developed its own large language model that is hosted on Google Cloud. Tabnine is now also committing to leveraging the large language model that Google is developing to automate software development life cycle processes.
Brandon Jung, vice president of ecosystems for Tabnine, said it’s apparent that application development and deployment teams will be making use of multiple large language models across SDLC workflows. Not all those large language models will be developed by one single platform provider, he noted.
Rather than being dogmatic about large language models, Tabnine plans to make it possible to invoke multiple models via application programming interfaces (APIs) that are integrated with development environments, he added.
The Tabnine platform supports multiple programming languages, including Python, Java and JavaScript and is designed to be integrated with integrated development environments (IDEs) such as Visual Studio Code and Jetbrains. The company also previously integrated its code completion tool with the GitLab continuous integration/continuous delivery (CI/CD) platform. The overall goal is to make it easier for developers to automatically write code based on custom models using approved source code hosted in a secure private repository.
Generative AI creates a large language model that assesses the probability of what the next line of code will be based on what has preceded it. It’s not likely DevOps teams will be replaced as generative AI is extended across workflows, but the overall size of those teams might shrink as it becomes possible to do more with fewer people. At the same time, the barrier to DevOps adoption will also fall as AI platforms make it simpler for more organizations to embrace DevOps best practices.
The challenge will be making sure the data collected to train AI models is of a high enough quality to ensure the desired outcome. Organizations will need to find a way to apply generative AI to data models that have been validated by DevOps teams.
One way or another, it’s only a matter of time before generative AI capabilities are applied more broadly. Platforms such as OpenAI’s ChatGPT are only the tip of an iceberg that impacts almost every manual process, including software development and deployment. The issue will be determining how quickly those innovations will become practical enough to employ.
In the meantime, it is already apparent generative AI platforms are having a significant impact on the rate at which code can be developed. Inevitably, that means the amount of code moving through DevOps pipelines at any one time should increase significantly. DevOps teams should expect generative AI technologies to be applied both before and after application code is built and deployed, noted Jung.
In the meantime, the biggest issue is, arguably, keeping up with a rate of generative AI innovation that is occurring faster than many organizations can absorb.
Postman today made generally available a visual tool infused with artificial intelligence (AI) that makes it simpler to build software by invoking application programming interfaces (APIs) found on its network.
Shamasis Bhattacharya, chief software architect for Postman API, said Postman Flows makes it possible to use a low-code tool to build software using tens of thousands of APIs that can be found on the Postman API network.
The goal is to reduce the amount of toil required to build applications of moderate to medium complexity in a way that doesn’t require developers to have deep programming expertise, he added.
Instead, developers are presented with “blocks” that they can drag and drop to create an application and deploy the associated workflows. Postman Flows leverages AI to manipulate data returned by APIs using a natural language interface that reduces the level of technical acumen required to create those workflows.
That approach provides developers with a reactive experience that is embedded within a visual tool, noted Bhattacharya.
Many developers still have a natural bias toward writing code, but as demand for applications continues to increase, Postman Flows provides a means to significantly increase developer productivity, said Bhattacharya. There may still be instances where writing code is required, but as developers save more time using Postman Flows, many of them will switch over to a visual tool to build software faster by essentially gluing blocks together, he added.
There’s naturally a lot of focus these days on developer productivity during uncertain economic times. More organizations than ever are dependent on software, but their ability to hire armies of developers is limited. Low-code tools have emerged to provide a means for accelerating application development, but Postman is now looking to apply that concept directly to the network of curated APIs that it provides to application developers.
The one thing that is certain is that as the pace at which applications are being built increases, the number of them running in production environments will steadily increase. DevOps teams will find themselves maintaining a wide range of applications that were constructed using both procedural code and low-code platforms.
In the meantime, IT organizations may want to start evaluating which types of applications lend themselves to being built by development teams that may not require as much experience writing actual code. Much of the application development backlog organizations are struggling with today could be reduced using tools that automate a wide range of development tasks. That should free up developers with more advanced skills to work on more complex applications. That approach might also reduce developer turnover as much of the fatigue associated with building more routine applications is reduced.
Regardless of how applications are built, there will be a need to manage them in production environments. DevOps teams may soon be able to focus more of their time and expertise on optimizing application performance in production environments versus continuing to shepherd massive amounts of code through continuous integration/continuous delivery (CI/CD) pipelines.
Hewlett-Packard Enterprise (HPE) this week revealed it is acquiring OpsRamp to gain access to a platform for embedding artificial intelligence in IT operations (AIOps).
Latha Vishnubhotla, chief platform officer in the office of the CTO for HPE, said the company would embed the machine learning technology developed by OpsRamp across its entire portfolio in addition to leveraging it to automate IT processes the company manages via its GreenLake service for HPE customers.
Ultimately, HPE will be applying machine learning across the entire IT stack, including applications, to reduce the total cost of IT, she said.
The acquisition of OpsRamp is expected to close in the third quarter of this year. Terms of the deal were not announced, but it comes at a time when more organizations are getting comfortable with AIOps.
A recent OpsRamp survey polled 138 IT leaders working for enterprise IT organizations and 127 representing managed service providers (MSPs) that generate at least $25 million in revenue and have more than 500 employees. The survey found the primary reason organizations are embracing AIOps is to improve service and application availability and performance (60%), followed closely by automation of operations (58%) and processes (54%).
The survey found that the greatest IT operations challenge for enterprises in 2023 was automating as many operations as possible (66%), followed by driving greater efficiency and productivity of IT operations teams and processes (61%). Application-to-infrastructure dependency mapping is the top incident management challenge (64%), the survey found.
The primary operational benefit cited was reducing open incident tickets (65%), followed by reducing mean-time-to-discovery (MTTD) and mean-time-to-remediation (MTTR), tied at 56%, and automation of tedious tasks (52%).
In general, organizations are choosing one of two AIOps paths, said Vishnubhotla. Some are waiting to evaluate the recommendations made by the AIOps platform before implementing, while others are looking to completely automate processes so they can focus more of their time and effort on application development and deployment, she said. In much the same way a driver of an autonomous vehicle learns to trust the AI over time, there is a similar transition occurring with AIOps, noted Vishnubhotla.
The challenge is that it takes an AIOps platform a few months to learn an IT environment. The more dynamic that IT environment is, the longer that process can take. At the very least, however, AIOps platforms have the ability to automate many low-level IT tasks today that conspire to increase the amount of toil DevOps teams would otherwise encounter.
It’s difficult to determine how pervasive AIOps may be within IT environments, but there is no doubt as organizations look for ways to manage IT at scale, there will be increased reliance on automation. Most organizations can’t afford to continue to hire armies of IT personnel, especially as the global economic outlook remains uncertain.
In fact, one of the benefits of an AIOps platform is that, unlike an employee, it does not take any days off or suddenly decide to take another job. It also, for better or worse, doesn’t forget anything once it’s been learned. As such, it’s now only a question the degree to which AIOps platforms will be trusted over time to manage increasingly more complicated IT tasks.
Data is getting a facelift in 2023 courtesy of event-driven architecture (EDA). These four technologies are set to transform DevOps and IT operations over the next year–whether it be how companies make use of data or the customer experience they offer.
Applied observability has been recognized by Gartner as a key 2023 strategic tech trend at its most recent Orlando Symposium. It ensures that the right data is delivered at the right time for rapid action based on confirmed stakeholder actions rather than intentions. “Observable” data include key digitized artifacts that appear when any stakeholder takes any kind of action. Applied observability feeds these observable artifacts back in a highly orchestrated and integrated approach to accelerate organizational decision-making and allows the business owner to track how long an action took to fully process at every point in the workflow.
But unearthing these insights requires an event-driven approach to software architecture. Distributed tracing, for example, is a method of tracking application requests as they flow from front-end devices to back-end services and databases. By definition, and in contrast to traditional tracing, distributed tracing can be visualized to show a searchable, graphical picture of when, where and how a single event flowed through an enterprise, regardless of the number of hops the workflow took to fully process.
Consider the potential traceability and end-to-end observability benefits across a payment ecosystem underpinned by EDA. Embedding distributed tracing into an event mesh emits trace events in OpenTelemetry format so banks can collect, visualize and analyze them in any compatible tool. This empowers them to not only confirm that a given message was published but easily understand exactly when and by whom, where it went, down to individual hops, who received it and when … or why not.
Gartner and IDC identified both hyper and intelligent automation in their yearly technology trends for 2023. Hyperautomation moves automation up a level, adding more intelligence to automation and using a broader set of tools so that previously un-automatable tasks can be automated. Hyperautomation initiatives come in many different shapes and sizes and are being seen across a wide range of industries, from banking and insurance to manufacturing and health care.
But hyperautomation requires an organization to be underpinned by an EDA and–more specifically, an event mesh–to maximize success. An event mesh is an interconnected network of event brokers that allows data to be pushed in real-time to parts of the organization where the data is needed. It does this dynamically, meaning that new event types can be added at any time and interest in events can be registered to allow a seamless interchange of data for the applications that are interested in using it.
Take the aviation industry as an example. An event mesh streams information such as flight routes, delays, cancellations and mileage accruals between applications, connected devices and people anywhere in the world, instantly. With an event mesh, information about events can be continuously streamed to multiple systems; filtered so each system only receives the data it needs. This ultimately enhances the customer experience as passengers, pilots and crews are notified in real-time when something relevant occurs on each and every flight.
The API economy has exploded. Research from MIT Initiative on the Digital Economy showed that over a four-year period, businesses using APIs saw 12.7% more growth in market capitalization compared to those that did not adopt APIs.
But the API economy is changing as event-driven architecture and asynchronous event-driven APIs become increasingly important to companies striving to operate in real-time. Using both synchronous and asynchronous methods can result in an application environment in which system resources are most effectively used.
AsyncAPI and RESTful APIs will be the next evolution of the API economy. API management platforms will need to adapt quickly to this new reality. As a prime example, Forrester recently spotlighted a global biotech company that is blazing a trail in the API-driven area. As the case study demonstrates, the organization understood that EDA is a complement to RESTful APIs–another tool in their digital business toolbox, filling in the gaps of a REST-only approach.
As a result, they planned a unified event + REST approach from the outset. Both APIs and events are managed as enabling digital products, with the governance and life cycle of both unified as one process. A platform team builds a unified API + event platform for app dev teams rather than building each as a separate siloed platform. The results include faster speed to market and new business opportunities. The company also embraces FAIR data principles: Data that is findable, accessible, interoperable and reusable. A digital marketplace consisting of both events and APIs makes the data findable and machine-readable via Open API Spec (OAS) and AsyncAPI metadata. Data can then be accessed on demand via APIs and events push data in real-time.
Driven by technologies including digital twins, augmented reality and virtual reality, McKinsey estimates the metaverse will be worth $5 trillion by 2030. Data in real-time will be the vital common denominator to link the digital and physical divide and optimize the Metaverse. McKinsey lists real-time data as a vital element to facilitate adoption of the metaverse in several key industries including retail, banking/financial services and transportation.
The metaverse cannot be static. It needs to be in real-time, in motion. It needs an EDA and a real-time event mesh to support it.
For example, if an avatar visits a retailer in the metaverse, the retailer will need to keep track of countless user actions such as how long someone spent looking at a particular product, which products got the most interest, etc. The metaverse will produce a significant amount of data that companies will be interested in analyzing in real-time. This data can be used to make decisions from real-time adjustments to inventory to incentivizing commerce through real-time offers/discounts/coupons and even dynamically pricing products.
Together, these four predictions have the potential to revolutionize company operations, communications and decision-making–but to take advantage of their full potential and reap success, these four trends will need EDA by their side.
In recent years, vector embeddings have emerged as a promising technology to power information retrieval projects. Vectors use mathematics to represent words and encapsulate meaning. Combined with deep learning algorithms, you can build powerful AI search applications. However, recent advances have shown that another kind of technology – neural hashes – can outperform vectors while avoiding the tradeoffs that people experience with vector search.
Neural hashing’s superiority over vectors will soon make it accessible, affordable and the primary method for delivering AI-powered search for e-commerce, enterprise search and other areas where search data is constantly growing and changing. Even as early as 2023, neural hashing will be the primary method for delivering AI-powered search.
Most of us are familiar with the basics of keyword search technology, which most search engines still use. When someone submits a query, the keyword search engine will perform a simple lookup and display results by relevance. Keyword algorithms employ techniques such as natural language processing (NLP), lemmatization, typo tolerance, synonym libraries and more to perform the lookups.
Keyword search engines work well for the most part but can fall short when it comes to long-tail queries, i.e., those longer and more specific queries which represent half or more of search queries. Despite its powerful keyword technology, you can never write enough rules or stuff your site with enough keywords to cover every edge case. Being able to improve long-tail query results, however, can be a huge benefit to an organization, which is why more companies are beginning to look for new technologies to complement their keyword search engines.
Semantic, or AI-powered search, can understand concepts. Semantic search goes beyond matching to understanding. For example, if you searched “something for a headache” on a website, a keyword search engine would look for items tagged with those specific words, turning up results only if they can match some or all of the text. But a semantic search engine would understand the concepts being described and offer solutions—such as pain medications.
Semantic search has traditionally been employed using vector embeddings. You can use vectors for all kinds of objects, too—images, videos, audio, etc. Vector-powered search engines can be incredibly smart, but processing and maintenance make them too expensive to run in production for most enterprise use cases. This is true, especially for certain industries where data is dynamic. In retail and e-commerce, for example, that inability to pivot and scale quickly is a real detriment, as brands need to update data that is unpredictable (e.g., user-generated content) and constantly in flux (many online sellers update their search databases hourly).
The solution is neural hashing, which refers to the process of compressing vectors with neural networks (deep learning algorithms which mimic the human brain) into hashes or binary vectors. These hashes retain 96% or more of the vector information while using a fraction of the storage space. A semantic search engine built using neural hashing is nearly or just as accurate as vector search but about 500 times faster.
When it comes to AI-powered search, neural hashing will eliminate the tradeoffs between speed, accuracy and costs. It can deliver results in single-digit milliseconds with no additional overhead. As good as it is, it’s even better when combined with a keyword search. We can use hybrid search—both technologies—simultaneously; most queries are a combination of keywords and concepts.
For example, what if rather than looking for a type of product, a shopper is searching for help with getting rid of a pesky symptom? Or, has a question-style query? As search data indicates, people are often myopically focused on a problem that they desperately want to solve, such as:
● Leaky faucet
● Stress headache
● Stained carpet
They only know they have an urgent problem. Of course, they’re looking for a solution, but they typically have no idea what particular product they need in order to get relief. They’re still using some keywords, however, like “faucet,” “stained” or “carpet.”
Each query can be scored by both the hashing and keyword engines to deliver a superior result for every query. Hybrid search can be combined with traditional search engine features like natural language processing (NLP), autocomplete, learning-to-rank algorithms, filters and more to optimize the end-user experience.
When it comes to search, speedy, accurate results are the name of the game. In order to make searches as intuitive, fast, and scalable as possible while still remaining cost-effective, search and discovery platforms should explore what hybrid search powered by neural hashing has to offer.
It has been shown that neural hashing can make highly intelligent AI fast and cost-effective at scale and without the need to bring on highly specialized engineers for every project. We appear to be at another turning point in the world of artificial intelligence, and neural hashing is going to be the driving force behind many future innovations.