devopsy.kinsta.cloud

Category: AI

Broadcom Extends AIOps Platform Reach

Broadcom today extended the reach of its artificial intelligence for IT operations (AIOps) platform to provide full-stack observability via integrations with 15 third-party data sources including Dynatrace, Datadog, AppDynamics, PagerDuty, Splunk, ServiceNow, SolarWinds and Microsoft Systems Center Operations Manager.

In addition, Version 2.0 of AIOps from Broadcom now captures data at the chip level to enable IT teams to triage network congestion and packet loss in real-time.

The latest release also adds visualizations and dashboards for different IT personas along with a recommendation engine that enables IT teams to automate tasks once the root cause of an issue is surfaced by the platform.

Sreenivasan Rajagopal, director of products and strategy for Broadcom Enterprise Software, said the new approach allows IT teams to decide what level of automation to apply based on the complexity of the problem and potential risk to the business.

Collectively, these capabilities are intended to make it easier to align people and processes with business outcomes by creating a continuous feedback loop, he said, noting it takes about 90 days for the machine learning algorithm embedded within AIOps from Broadcom to start delivering on that promise.

Other capabilities included as part of the latest release are additional machine learning topology and correlation capabilities, causal graphs and intelligent situational analysis for automatic root cause analysis, noise mitigation, alarm clustering and anomaly detection. Broadcom has also added a library of machine learning algorithms, natural language processing and an ontological graphical topology to enable situational correlation.

Finally, Broadcom has added capacity analytics tools that can be applied to specific groups of services that IT teams are collectively managing and maintaining.

Rajagopal said it’s now only a matter of time before AI is applied more broadly to enable IT teams to become more agile. AIOps won’t replace IT staff as much as it will simply augment the capabilities of the IT staff, he noted.

AIOps also will prove vital as older members of the IT team retire, Rajagopal added. Newer members of the IT staff will be able to become productive much faster if they can rely on insights and recommendations surfaced by an AIOps platform.

It’s not clear to what degree AIOps platforms will exist as a separate entity apart from traditional IT services management (ITSM) platforms that are also being infused with machine learning platforms. Broadcom is making a case for a distinct new IT operations platform that relies on data to align people, processes and platforms. The single biggest obstacle in terms of achieving IT agility is not the tools as much as it is people in different roles who can’t empathize with one another because there is no common source of data that every stakeholder can access, said Rajagopal.

Regardless of the AIOps path chosen, it’s clear that many of the existing tools IT teams rely on to manage IT environments are slowly becoming obsolete. In fact, there may come a time soon when IT teams won’t even consider an IT management platform that doesn’t have some level of AI capability.

December 1, 2020
Building Your Own AIOps Platform Is a Bad Idea
It takes considerable expertise to not only build, integrate and maintain an AIOps platform

In the wake of the economic downturn brought on by the COVID-19 pandemic, investments in digital business transformation have accelerated. The applications that drive those processes are not only highly distributed, but they also operate at a level of scale that no IT team can manage using legacy approaches to managing IT. It can take weeks to discover the root cause of any issue.

Enter AIOps. Machine learning algorithms make it possible to reduce the time it takes to resolve any issue and enable IT teams to continuously optimize IT environments at any scale. Many aspects of AIOps, however, are still largely unexplored. Rather than opting for proven platforms, some IT teams are building custom solutions in-house.

I have experienced firsthand several large enterprises that have embarked on this risky journey, including one Fortune 500 company that asked a partner to help develop a solution and ultimately deployed a commercial product. Enterprise IT teams go down this road to solve a specific tactical problem, such as alert noise reduction. In many cases, all the time and effort put into a do-it-yourself (DIY) project simply winds up being wasted.

Why DIY Platforms Usually Fail

Given the prevalence of open source AI tools and frameworks such as TensorFlow, Theano or the Microsoft Cognitive Toolkit (CNTK), it can be tempting to build your own custom AIOps platform. It takes considerable expertise, however, to not only build an AIOps platform but also integrate it into an enterprise and maintain it.

Here are the leading reasons why in-house developed AI projects are risky:
- You’ll need a properly constructed data lake: AIOps platforms require access to data residing in multiple technology silos in real-time. IT teams that build their own AIOps platforms need to make sure they are gathering all the right log data, metrics and traces alongside data collected from IT service and incident management platforms. These comprehensive data sets are needed to train whatever machine learning framework is in place, which is often selected at random. Invariably, that means building or buying a costly big data platform to create a data lake to store all that data. A poorly constructed AIOps platform will be worse than the proverbial disease it’s meant to cure because the insights don’t accurately reflect what’s actually occurring in the IT environment. Do you have the funds for this and experienced data science experts onboard to get this right?
- Designing AI-enhanced workflows is unlike other workflows: Getting the data is just the beginning. Determining how the system behaves and affects existing workflows is the next step. IT teams must decide to what degree they merely want the AIOps platform to passively surface recommendations based on what’s observed versus automatically resolving issues based on defined parameters.
- Deployment is complex: After developing a few AIOps algorithms to produce meaningful results, the next step is to determine how to deploy it in a resilient and performant architecture. What other systems does it need to integrate with and how will results be monitored and viewed?
- Monitoring user impact is critical: How will end users interact with the algorithm and what is the ideal UI/UX and workflow? How will feedback be provided by end users for improvement and adoption success?
- AIOps support and maintenance is not a project, but a team. Ultimately, an internal IT team would need to build the equivalent of a product which needs ongoing maintenance and support. The total cost of the custom platform starts to rise as the bulk of the IT team could wind up spending most of their time managing the AIOps platform instead of making continual improvements. Even if the IT team has the expertise required to build an AIOps platform, there’s no guarantee those individuals will always be available to maintain and update it. Very few IT professionals spend their entire career at one organization.
- Keeping pace with marketplace innovation: Finally, AIOps as a field is still relatively nascent and the startup community has hundreds of millions of dollars in VC backing to support R&D. Advances are being made at a rate most internal IT teams can’t keep up with, let alone evaluate and vet on their own.
How to Safely Encourage AIOps Exploration

There’s no substitute for knowing where an organization needs to go and how to get there. A commercial AIOps platform incorporates all the best practices that have been defined by legions of IT experts, along with these benefits:
- Faster time to value: You can embark on the AIOps journey much sooner. A commercial AIOps platform will begin surfacing insights in a matter of weeks. It will take an internal IT team months to build an equivalent platform with no guarantee of success. Time is better spent on user adoption and adding and refining use cases for business benefit.
- Seasoned experts: A commercial platform provides immediate access to not only a proven framework but also, AIOps experts who can troubleshoot and optimize issues quickly. There’s almost no AIOps challenge they haven’t seen before.
How to Measure ROI From AIOps

Savvy organizations that invest in AIOps are primarily betting on a better way to manage IT that will enable them to accomplish more as a business. The real value proposition of any AIOps platform is that it enables an existing IT team to do more not by just eliminating rote tasks but also making it possible to deploy more applications reliably without adding IT staff. It’s worth remembering that the cost of labor continues to be the single biggest IT expense.

The return on investment from an AIOps platform can be easily calculated by measuring:
- The number of incidents resolved in a given period of time.
- The size of the IT operations/incident management staff before and after an AIOps platform is deployed.
Arguably, however, those savings pale in comparison to the opportunity costs of using advanced technologies which can increase operational intelligence and deliver new business value in the form of excellent user experiences and high-performing digital services. IT organizations that don’t embrace AIOps will be unable to compete with faster, more nimble rivals that have modernized their IT processes. The easier, more affordable and least risky way to go about this is by deploying a solution already proven and in market.
November 20, 2020
OpsRamp Makes AIOps Platform Simpler to Configure

OpsRamp today took the wraps off an update to its namesake artificial intelligence platform for IT operations (AIOps) that makes it easier for IT organizations to declaratively configure the software-as-a-service (SaaS) platform.

Ciaran Byrne, vice president of product management for OpsRamp, said the goal is to enable IT teams to embrace AIOps without having to hire a small army of consultants.

The OpsRamp Fall 2020 Release adds a revamped user interface that provides access to a set of wizards that discover the assets that make up a specific IT environment and subsequently automate the configuration process.

OpsRamp has also added curated dashboards that enable IT teams to launch queries using the Prometheus Query Language (PromQL), developed for the open source monitoring tool being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

The latest edition of the AIOps platform also adds monitoring capabilities for containerized workloads and Kubernetes clusters. In addition, IT teams can monitor other cloud services including AWS ECS, Azure Functions, Azure Hyperscale (PostgreSQL) and Azure SQL Managed Instance.

OpsRamp has also added IT process automation capabilities that can be triggered by alerts, updates to resources or on a recurring schedule. An OpsQ bot now also recommends automation workflows. Alternatively, IT teams can require manager approval before a process is launched.

In general, Byrne said the goal is to improve the mean time to value for an AIOps platform managed by a team of data scientists who have extensive experience applying machine learning algorithms to IT processes.

Efforts to automate IT have increased sharply now that most IT staff are working from home to help combat the COVID-19 pandemic. However, few organizations are able to set up and configure an AIOps platform in an on-premises IT environment on their own, especially now that outside consultants are no longer being invited to spend weeks working on projects, Byrne said, noting a SaaS-based approach makes it possible for IT organizations to take advantage of an AIOps platform while continuing to work from home.

In general, the downturn in the economy has increased interest in AIOps because IT staff ranks are either remaining the same or shrinking even as IT environments continue to become more complex. Longer-term, Byrne said it’s not clear that hiring Level 1 and 2 IT administrators will be sustainable as AIOps platforms continue to automate processes.

DevOps teams, of course, are especially keen to automate as many IT processes as possible, which is why there is a high correlation between organizations that have embraced DevOps and early adopters of AIOps.

It may still be a while before AIOps is widely employed, but it’s now only a matter of time before machine learning algorithms are pervasive across IT. The challenge now is figuring out where the handoff between man and machine for managing IT ultimately will lie.

November 19, 2020
Want To Deliver Better Software Faster? Use Automation

Despite the evident evolution of the software industry, many enterprises continue deploying testing practices that require intensive time and effort, while also facing increased pressure to do more and work faster. I like to call this phenomenon “scattered islands,” as it describes the disconnect in maturity and capability across the testing landscape. The key differentiator here is automation, the impact of which can be found throughout all the phases of software testing, from test planning to defect analysis repair and feedback loop. Even so, the industry has just begun unlocking the true potential of automation in testing.

AI-driven automation holds the potential to completely transform software testing from an end-to-end testing lifecycle perspective.

Gap Between Knowledge and Action

The shortage of test automation is one of the biggest barriers when it comes to releasing new software to the market. In fact, most software testing is still done manually. Enterprises are embracing Agile and DevOps approaches for faster software release cycles to meet customer expectations. However, the bulk of the testing life cycle—especially the planning and design stages—continue to be manual and inefficient. Undetected software quality issues translate into production. The reduced speed-to-market is costing businesses trillions of dollars.

Strangely, the software industry has yet to embrace automation applications. According to a study by Kobiton, more than half (55%) of respondents believe automating testing would be an asset to software quality. And yet, most aren’t doing it. In that same study, 76% of respondents say they manually conduct most of their testing; 73% are running at least 100 tests prior to each software release manually—not automatically.

How Software Testing Is Adopting Automation

Organizations are starting to deeply consider how to adopt assisted and automated practices across the testing life cycle. That spans from test creation to data management and beyond. Automation is quickly becoming critical for effectively meeting customer requirements, because it offers important benefits: faster time-to-market, increased efficiency and improved quality.

Now, most organizations are applying automation little by little to improve productivity and stay at par with the demands of development and release cycles. For instance, they may apply automation to test execution or defect management. However, this stilted approach is inadequate for today’s scale.

Some have speculated that organizations may be leery of full-scale automation due to the looming fear that it would cost the testers their jobs. This is a misconception. Automation isn’t taking jobs, but it is changing them. Testing will become more about predictive rather than preventive methods. With the agile development environment, these methods are being implemented starting with the planning phase.

By conducting testing through automation tools, organizations can address issues faster. And that means testers will be able to upgrade their skills. With their deep domain expertise, they’ll be eligible for other business roles and not just IT roles.

Start Small and Expand

Ultimately, the goal with automation is to become autonomous. Therefore, it’s important to understand that not all tools are created equal. Many startups have entered the market with test automation tools, some of which make lofty promises about improved speed. But there is no tool that automates test scenario creation or provides intelligent test selection. What organizations can do is identify areas where it makes the most sense to start introducing automation and begin to implement it gradually.

It’s key to remember that organizational support is imperative to successful deployment. Without it, projects aren’t likely to succeed. It’s easy to get stuck in the status quo because “that’s always how we’ve done it.” To move to automation there must be a desire for change. Being the change agent in the organization and finding support before embarking on this journey is crucial.

Jumping Into the Automation Pool

AI is rapidly becoming core to every industry and function, including the software industry. AI-based automation enables enterprises to take their software to market faster and reap unparalleled business benefits that come with agility, speed and reliability in their software release cycles. Software testing is only one application of this type of automation. Test automation not only relieves testers of the burden of repetitive tasks but can also perform additional testing that might not otherwise get done because it is difficult to do manually.

Although the numbers clearly indicate that bugs and rework incur higher costs for organizations to maintain the status quo, many still hesitate to embrace the innovation. Most organizations agree that automated testing would be of tremendous benefit, and the hesitation indicates that early adopters will have a competitive advantage. With well-informed strategy and integration, test automation can enable you to deliver better software, faster and transform your business exponentially.

November 9, 2020
Red Hat Automates More Operating System Tasks

Red Hat today announced an update to Red Hat Enterprise Linux (RHEL) that automates a wide range of manual tasks.

Scott McBrien, a principal product manager for Red Hat, said RHEL 8.3, available in a few weeks, expands the Red Hat System Roles capability that provides a prescriptive and automated means of managing operating system configurations. Red Hat System Role now addresses kernel and log settings, identity management, certificate management and Network-Bound Disk Encryption (NBDE), as well as deployments of SAP HANA databases and SAP NetWeaver middleware. Red Hat has also updated Tuned, a set of pre-configured performance profiles it makes available.

In addition, RHEL 8.3 provides an update to Application Streams, a set of developer frameworks, databases, container tools and other resources that now adds support for Node.js 14, Ruby 2.7 and other current versions of tools and languages. Red Hat is now committed to streaming support for updates to the frameworks include in Application Streams versus only making updates available each time the entire platform is updated, said McBrien. That approach is intended to make it possible for developers to embrace new capabilities faster while maintaining support for previous versions of frameworks, said McBrien.

Red Hat is also furthering its effort to encourage organizations that have standardized on RHEL to embrace its tools for building and managing container applications. Updated container images for Buildah, a tool for building container images and Skopeo, a tool for moving container images across storage platform, have been added along with version 2.0 of Podman, a tool for managing containers and images that adds a REST application programming interface (API) that is compatible with Docker Container Engine.

Red Hat is making the cases for a more modular approach to deploying tools for building and maintaining containers as an alternative to a more monolithic approach pursued by Docker Inc., said McBrien.

Finally, Red Hat has added Secure Content Automation Protocol (SCAP) profiles for the Center for Internet Security (CIS) Benchmark and the Health Insurance Portability and Accountability Act (HIPAA).

RHEL and the Red Hat OpenShift application development and deployment platform are at the core of the hybrid cloud computing strategy being advanced by Red Hat and its parent company, IBM. According to a Red Hat Enterprise Open Source Report, 63% of organizations surveyed have a hybrid cloud architecture today, with more than half of those who do not have one in place planning to implement one in the next two years.

Beyond bundling developer tools with RHEL to reduce the number of IT vendors, an organization needs to engage, McBrien said Red Hat is committed to automating as many low-level IT tasks as possible. IT teams should be able to deploy operating systems in the cloud just as easily as they might in an on-premises IT environment, he noted.

It may be a little while yet before every manual IT task becomes automated, but with each new update to the operating system the number of those rote tasks that IT administrators need to devote time to continues to diminish.

October 29, 2020
Going Through a Cloud Migration? AIOps Can Help

AIOps can help ease many of the challenges associated with cloud migration

As businesses seek to compete in the digital economy and better engage with customers through new digital experiences, they have become increasingly reliant on cloud as a critical component of their digital transformation strategies. This has never been more true than during the COVID-19 pandemic, when businesses across all industries are aiming to provide the best digital experiences possible in our new, heavily remote world.

Gartner validates this reliance on cloud in a recently published report, which states that although IT spending is set to decline in 2020 due to the impact of COVID-19, the pandemic’s increase in remote work has set public cloud services to grow 19% this year.

There’s no doubt that cloud migration can be challenging, but properly preparing and utilizing tools such as AIOps can help initiate this process and can provide ongoing visibility.

The Ingredient for a Successful Cloud Migration

AIOps helps identify and resolve issues before they disrupt the end user during a cloud migration, arming technologists with insights and providing visibility within multi-cloud and containerized environments as companies manage their complex platforms.

A surge of data can be managed in the cloud but can be difficult to manage without having visibility across the platform. AIOps offers an overview and provides updates and optimization suggestions, which helps businesses be competitive and have control.

The AIOps and Cloud Migration Process

AIOps exists to help with big IT challenges and there is a suggested path businesses can use pre-, during and post-migration.

Pre-Migration

Before the migration begins, you should measure the existing performance of your applications as a baseline. You can then define performance goals and find bottlenecks you should avoid during your cloud migration. You must evaluate the potential impact of your cloud migration on your IT environment, which AIOps can help with. With robust visibility, you can see results on applications that power your business.

During Migration

Once you have migrated your workloads to the cloud, you can utilize machine learning and AI to check which resources might be underused. AIOps on the cloud will provide insight on overused resources, so you can identify instances that are best for your company’s needs.

Post-Migration

Being able to monitor your services on the cloud and end user performance gives full visibility into what you are getting from your cloud provider. Needs can change over time, so a cloud migration strategy that incorporates AIOps can identify what can be altered. This functionality helps make it clear which applications could benefit from being moved.

The reliance on cloud support is only increasing, so the sooner you incorporate AIOps into your IT environment, the faster you can have confidence your services will deliver improved performance for the end user. Why not get the most out of your investment in the cloud by using AIOps to support your business for the long term?

AppDynamics is sponsoring KubeCon + CloudNativeCon 2020 North America, taking place Nov. 17-20. To learn more about how AppDynamics is supporting businesses in their cloud migration journeys, register for our webinar during the show here.

October 29, 2020
Tricentis Applies Visual AI to Improve Application Testing

Tricentis this week announced it has added visual test design and automation technology based on artificial intelligence (AI) to its Tosca platform.

Dave Colwell, vice president of AI and machine learning at Tricentis, said Tosca 14 automatically recognizes and identifies visual user interface elements and controls across any form factor to identify the most appropriate tests to run as an application is being developed. That approach enables testing to be shifted further left during the application development process as successive iterations of an application are developed, he said.

While AI has been incorporated into other AI tools, Colwell said Tricentis is applying the same types of machine learning algorithms that are employed to enable self-driving cars to detect objects to the testing of controls included on an application screen.

Colwell said it’s now only a matter of time before most developers routinely expect testing capabilities based on visual AI technologies to be included in testing tools. Rather than having to continually encounter the same design issues, visual AI will enable developers to craft applications that end users will employ much faster.

Given the large numbers of applications that many end users haven’t embraced because of user experience issues, there’s clearly a base of developers who could use some advice from AI tools that have been trained to identify common problems that have affected previous generations of applications. Tricentis is now not only committing to providing testing tools but also curating the content required to train AI models.

It may take a while for developers to fully trust those recommendations, but the number of potential issues that could be averted earlier in the application development life cycle has significant implications for the productivity of DevOps teams. In theory, at least, the number of applications successfully moving through the build process should increase. The amount of testing required after an application is created should also decline as well.

Less clear is to what degree the shifting of testing of visual elements of an application might have on dedicated application testing teams. Arguably, IT organizations should be able to devote more resources to building versus testing applications.

Whatever the outcome, AI is not going away. Providers of testing platforms are now effectively engaged in an algorithms arms race. In fact, it’s becoming increasingly apparent AI will be applied with greater frequency across the entire application development and deployment life cycle. It may take a while for IT organizations to adjust to that new reality, but the fact remains that organizations that don’t incorporate AI into their DevOps processes will be at a distinct disadvantage compared to those that do. Members of DevOps teams would be well-advised to start assessing what manual processes today are most likely to be automated by algorithms tomorrow.

In the meantime, the overall quality of the user experience should also soon improve—assuming, of course, the AI models being created to test applications are indeed being trained properly.

October 23, 2020
Harness to Apply AI to DevOps

Harness, at its {Unscripted} 2020 conference today, announced its plans in the fourth quarter to make available as a beta a module that leverages machine learning algorithms to optimize build and test cycles on the Harness Continuous Integration (CI) Enterprise platform.

At the same time, Harness is adding a beta of a Continuous Features module to enable DevOps teams to employ feature flagging to simplify the application development process. A visual dashboard for feature verification and analytics monitors all flags, including active, inactive, live and unrequired flags, as well as customer issue tickets, to simplify the management of new capabilities being added to an application.

Harness is also updating its continuous delivery (CD) platform to include a revamped user interface, GitOps and pipelines-as-code processes, bi-directional synchronization and conflict management, and templates to standardize deployments across teams.

Finally, Harness is adding a Harness Next Generation Continuous Verification tool to provide visibility into the impact changes have on IT environments. This tool can be deployed standalone or used in conjunction with any CD platform and is intended to make it easier for DevOps teams to discover the root cause of issues that typically stem from recent changes made to an otherwise stable IT environment.

Harness CEO Jyoti Bansal said artificial intelligence (AI) in the form of machine learning algorithms are poised to play a major role in taking DevOps automation to the next level. The Harness AI module can reduce test cycle time by up to 75% by correlating and isolating tests to changed code rather than requiring all tests to be executed with every change. The Harness approach means only relevant test cycles are run, he said.

Continuous Integration Enterprise will also be able to identify gaps in test plans by highlighting code changes that are not covered by existing test cycles. Library caching and container image size optimization will also be added in the future to make builds more efficient.

Harness also announced its intent to apply machine learning algorithms more broadly. Algorithms will enable DevOps teams to control, optimize and accelerate the flow of software pipelines rather than requiring engineers to manually verify builds, tests, deployments and releases. Harness will also apply data science techniques to observe the quality, performance, cost and reliability of code as it flows through each pipeline step and, if need be, automatically abort or roll back the delivery of software.

As each build is committed, the opportunity to collect even more metadata about the workflow process only increases, Bansal added.

Obviously, Harness is not the only provider of a CI/CD platform looking to apply machine learning algorithms. Thanks to machine learning algorithms, the rate at which applications are developed and deployed could increase by several orders of magnitude in the months ahead. The issue now is to what degree the rest of the business will absorb that potential rate of change.

October 22, 2020
ServiceNow Partners With IBM on AIOps

ServiceNow and IBM this week announced that the Watson artificial intelligence for IT operations (AIOps) platform from IBM will be integrated with the IT service management (ITSM) platform from ServiceNow.

Pablo Stern, senior vice president for IT workflow products for ServiceNow, said once that capability becomes available later this year on the Now platform, IT teams will become on average 20% more productive. This is good timing, as most teams currently are struggling to adapt to the increased demand for IT support from employees working from home and in the office in the wake of COVID-19 pandemic.

Mike Gilfix, vice president of cloud and integration and chief product officer for Cloud Paks at IBM, said IT teams will also be able to proactively identify anomalies in IT environments 60% faster.

IT teams interact with IBM Watson AIOps via a chatbox interface such as Slack. The platform will surface recommendations based on ServiceNow data and enable IT teams to query IBM Watson AIOps to provide a faster mean time to resolution for IT incidents, said Gilfix. That capability is provided via a configuration management database from ServiceNow that tracks the operational state of all business services in real-time.

IBM Watson AIOps, which can be deployed in the cloud or on-premises, also can surface additional context the next time similar issues arise, in addition to being able to record actions and insights for auditing purposes. IT teams can also feed data from other systems into IBM Watson AIOps to automate a wider range of processes.

AIOps platforms won’t be replacing IT professionals anytime soon. However, IT teams that are not able to access AIOps platforms will find themselves working harder than their peers. As IT environments become more complex, the number of potential issues that can adversely impact end user experience are moving beyond the capabilities of an IT team’s ability to track manually.

AIOps platforms, meanwhile, continue to advance in sophistication. Given the structured nature of IT environments, IBM Watson AIOps is now able to have a significant impact on productivity with 24 hours of being deployed, Gilfix said. As part of this launch, IBM is expanding its global ServiceNow services offering to include additional capabilities that provide advisory, implementation and managed services on the Now Platform. IBM also has formed an AIOps Elite Team that will help prepare organizations to transition to AIOps free of charge.

IT may take a while still for IT teams to adjust culturally to AI, but there are many rote tasks that many IT professionals would like to see eliminated. Over time those tasks conspire to increase the level of drudgery IT professionals experience as they perform the same repetitive tasks. In theory, at least, AI should reduce overall IT staff fatigue in a way that could reduce turnover while at the same time freeing up resources to tackle more complex IT challenges.

It may take some time for IT teams to trust the recommendations surfaced by an AIOps platform, but the more those platforms learn about the IT environment, the more accurate those suggestions become.

October 16, 2020
Has the Time Come for IT Predictive Analytics?
Predictive analytics technologies have become critical to compete in manufacturing (predicting machine failure), banking (predicting fraud), e-commerce (predicting buying behavior) as well as to address horizontal use cases such as cybersecurity breach prevention and sales forecasting. Using data to predict and prevent IT outages and issues is also a growing best practice—especially as advances in monitoring software have made it easier to deliver analytics in a timely manner.

The process of rapidly collecting IT systems data, including by integrating data from various monitoring tools in use, has become rudimentary. The magic is in the algorithms, which can deliver actionable intelligence. IT predictive analytics, once known as IT operations analytics (ITOA), is still nascent in many organizations, but it’s far more streamlined than it used to be when one needed to export data sets to specialized analytics tools such as Tableau or Microsoft PowerBI. Data analysts and then data scientists were necessary to interact with these advanced tools and deliver valuable predictions—a process that could take weeks.

Today, IT professionals can use modern IT operations monitoring and management systems for collecting and integrating the data, normalizing it and then analyzing it in real-time. That’s making predictive analytics in IT far more user-friendly, affordable and useful for all organizations, not just the largest companies with the biggest budgets.

Advances in predictive analytics technology, aided by machine learning and AI, are creating more proactive workflows for IT operations teams. For instance:
- Storage usage. It is not a happy moment when IT finds out that the expensive storage array is down. It’s not enough to know after the fact that storage volumes were steadily climbing out of the normal pattern. By knowing ahead of time that a trend is heading out of the acceptable range, IT can research the cause and fix it before any damage is done. With cloud storage, risks include slow connectivity if network bandwidth is insufficient or security breaches if the cloud provider’s security mechanisms are not sufficient for your use case. Predictive analytics can alert on these possibilities in time to take corrective action.
- Cloud-to-cloud performance comparison. An important aspect of predictive analytics is the ability to make correlations and show relationships between multiple data points. IT leaders are interested in learning about an application’s performance across the entire IT environment, which may include comparing historical performance across multiple clouds to predict weaknesses in one service over the other and mitigate them proactively. This use of predictive analysis could also be valuable in helping companies choose one cloud provider over another for particular applications and workloads.
- Container optimization. Making correlations is especially valuable when it comes to managing performance in a microservices environment. Collecting metrics for a containerized application isn’t useful by looking at each container per se, but it is useful when looking across the container environment to see if there are common metrics dangerously trending the wrong way across many or all containers.
A Few Tenets for Predictive Analytics in IT Operations

Regardless of the tools that you choose to create a predictive analytics practice in your organization, there are several things you must get right first. Here are six recommendations to consider:
1. Centralize the data into one or two systems at the most. This allows for data cleansing, normalization and a single (ideally) system of record for all other users and integrations.
2. Invest in first-class data visualization. As business units now interact with IT management systems to see departmental usage and troubleshoot minor issues, tools should be user-friendly for people who are not data analysts. Dashboards and charts can tell the story simply and explain the correlations and conclusions which the software delivers.
3. Make it actionable. There is no use in creating predictive data if it’s not automatically sent to the right individuals and/or shared with other systems to execute actions when needed. Think about it the Amazon way: We all know that people aren’t sitting behind monitors sending millions of individualized recommendations to customers every day. Your IT operations portfolio should behave the same way: automated, intelligent, helpful and always adapting to new information.
4. Correlate data at the speed of IT change. Before there was such a thing as virtualization and cloud infrastructure, there weren’t many devices to monitor and things didn’t change much day-to-day. You could actually “set and forget” data center infrastructure components for weeks. Now, of course, changes are happening behind the scenes every minute without our knowledge. Yet your monitoring and change management tools need to know about it. By using standard deviation for change detection monitoring—four-hour increments are typical—you can ensure that your team always has an updated understanding of infrastructure status as well as the relationships between the various systems and components.
5. Avoid the black box. Enrich your data lake continuously with event data and allow users to see the IT operations platform working in real-time. With machine learning and artificial intelligence, there’s always a measure of doubt and a margin of error. This uncertainty can be counteracted by sending more diverse data sets so that the user can make her own conclusions. As well, when the machine makes an error in judgment, train it to avoid making the same mistake again.
6. Update internal IT skills. You’ve heard it before. In the age of big data, IT organizations need data science expertise and skills. It’s important to acquire or develop these skills so that your team can combine machine intelligence with human insight and make better decisions. That is where AI-driven analytics holds the most promise.
We are just getting started on this new age of predictive analytics in IT operations. Before long, organizations will move from predicting downtime and serious application issues to driving infrastructure optimization and cost savings. IT leaders will be able to answer with confidence, how much cloud infrastructure needs to be provisioned or de-provisioned over time. They will understand with better clarity common application scaling patterns, as well as overall capacity management needs. As predictive analytics takes hold, IT leaders will need to determine how to reorganize employees who previously spent hours scanning data on screens into more strategic roles.
October 15, 2020