devopsy.kinsta.cloud

Tag: artificial intelligence

With Platform, Dotscience Applies DevOps to AI

Dotscience today launched a namesake platform for building and deploying artificial intelligence (AI) models based on a set of best DevOps practices.

Company CEO Luke Marsden said that as organizations realize that AI models need to be trained and updated continuously, the need for a DevOps platform that accelerates that process will become more apparent. Today most AI models are trained over an extended amount of time and then deployed within an application environment. Over time, however, either more data becomes available or organizations determine the machine and deep learning algorithms originally used to create the AI model need to be updated or replaced. Whatever the underlying reason for replacing an AI model, a platform that addresses all the aspects of the AI model life cycle, including testing, reproducibility, accountability, collaboration and continuous delivery, is required, he said.

To address those requirements, the Dotscience platform facilitates concurrent collaboration across developer and operations teams, version control of the model creation process, tracking of provenance records in real-time, exploring and optimizing hyper-parameters when training a model and tracking workflows across multiple open source tools.

Most of the teams that build AI models have backgrounds in data science versus application development. As such, Marsden noted most of them have had little to no exposure to DevOps practices. In that absence of those processes, teams building AI models now routinely encounter issues such as having to navigate siloed data and technical debt, which all conspire to extend the time required to build AI model, Marsden said. In addition, teams building AI models need to keep track of not only versions, but also runs of their code that tie together input data with models and corresponding hyperparameters and metrics, he added.

In the absence of a DevOps platform such as Dotscience, it becomes challenging for organizations to document what changes were made to an AI model when, Marsden said. That governance issue has become especially problematic when it comes to AI models because organizations are coming under increased regulatory pressure to document how the AI models they are employing are built and updated.

Dotscience’s “The State of Development and Operations of AI Applications” report also published today identifies the top three challenges with AI workloads are duplicating work (33%), rewriting a model after a team member leaves (27.6%) and justifying its value (27%). Based on a survey of 500 industry professionals, the report also finds 52% of respondents track provenance manually using tools such as spreadsheets, while 27% don’t track provenance at all but think it is important. In total, the survey finds 63% of businesses report they are spending between $500,000 and $10 million on their AI efforts.

As more organizations rely on DevOps processes to build and deploy applications, there’s no doubt that teams building AI models that need to be inserted into those applications will have to fall in line with best DevOps practices. That challenge now is finding a way to extend those DevOps processes all the way back to the building of the AI models themselves. Only then is the AI model building and deployment process likely to become agile enough to stay relevant to the pace of change now occurring across digital business processes.

— Mike Vizard

July 30, 2019
DevOps Deeper Dive: BMC Software Bets on AIOps Transformation
As BMC tests the AIOps waters, the company seeks to find a balance in serving both the DevOps and ITIL populations

It’s clear in the enterprise IT space that artificial intelligence (AI) is poised to play a significant role in IT operations, a phenomenon known as AIOps. Often with this level of IT disruption, new entrants funded by venture capitalists flood the segment, leaving incumbents to either fend off the new rivals, cede ground or, in many cases, put an end to the threat by acquiring one or more those startup companies that pose a threat to their existence.

In the case of BMC Software, it has decided to stand its ground by aggressively investing in AIOps research and development. Most recently, BMC extended the AIOps capabilities of its existing portfolio of TrueSight IT management software through additional advanced analytics to identify patterns and anomalies within an IT environment. Other examples of BMC employing machine learning algorithms to automate IT environments tools include the ability to surface continuous recommendations for cost optimizations and automated event remediation workflows that promise to reduce the mean time to repair (MTTR).

BMC has been making a case for AIOps as far back as 2017, but it’s only now that many organizations have enough confidence in AI to start embracing AIOps beyond the pilot stage. The challenge organizations face with AIOps are both technical and cultural: It requires a significant amount of time and effort to train and tune an AI model of each customer environment.

[su_box title=”BMC Software Milestones”]
- Dec. 2016: Peter Leav succeeded Bob Beauchamp as president and chief executive officer. Beauchamp remains chairman of the board of directors.
- Oct. 2017: BMC Launches First AIOps Initiative for TrueSight
- Nov. 2017: BMC Refinances Debt
- Oct. 2018: BMC Software Goes Private After Being Acquired by Investors
- Feb. 2019: BMC Partners with IBM on AI
- April 2019: Bob Beauchamp resumes role as CEO
- June 2019: BMC Extends AIOps Capabilities of TrueSight
[/su_box]

In addition, the effectiveness of any AIOps model varies based on the complexity of the IT environment. Organizations also should be concerned about the potential impact any flawed AI model might have on IT operations—it’s one thing for a human to make a mistake, but an AI model making a mistake at a level of scale could be catastrophic.

From a cultural perspective, not every organization is yet comfortable with those AI models on current staffing, which can make some members of the IT staff less than comfortable with the whole concept.

BMC CEO Bob Beauchamp

It is early days for AIOps in DevOps processes. But the increased complexity of modern IT environments based on cloud-native technologies will force the issue within most organizations. BMC’s challenge is that while its management platform is employed widely in IT environments relying on ITIL-based frameworks, its footprint is comparatively nascent within organizations that have embraced best DevOps practices.

Bill Talbot, vice president of solutions marketing for BMC, said one of the company’s advantages is access to the massive amounts of data required to train an AI model effectively. Most startup companies in this space don’t have nearly the same depth in terms of customer base, he said. That puts BMC in a better position to employ big data analytics to identify patterns and trends across multiple customers without having to know what data came from any one specific customer, unless the customer elects to share its data with BMC.

BMC makes it a point to caution customers about rivals that make AIOps claims about their platforms but don’t actually solve a specific problem—a process known as “AIOps washing.” An AIOps platform not only needs to leverage Big Data analytics and machine learning, but it also provides the code and functionality needed to customize specific IT operations use cases, including capabilities such as automated event remediation and service resolution, optimized cost and capacity management and proactive vulnerability mitigation.

BMC Software today stands at a crossroads. The company recently brought back former longtime CEO Bob Beauchamp to run the company day to day and has made no secret of the fact it is looking for additional investors. Making a pivot to AIOps requires access to massive amounts of funding for incumbents and startups alike.

The next BMC act undoubtedly will involve employing AIOps across both DevOps and ITIL-based environments running on-premises and in the cloud. By any measure, that effort represents one of the most ambitious transformations ever attempted by any IT vendor.

— Mike Vizard
July 29, 2019
Mendix Injects More AI to Accelerate Low-Code Development

The rate at which low-code applications are built and deployed will accelerate as developers begin to rely more on artificial intelligence (AI) to write code. The summer 2019 release of the Mendix low-code platform includes a revamped instance of Mendix Assist, which employs machine learning algorithms to make coding recommendations. Those algorithms have been trained using data drawn from 5 million application models employed across 15 industries over the last 10 years, and Mendix claims this version of Mendix Assist improves the accuracy rate of the recommendations being made to 95%.

Sheryl Koenigsberg, director of product and solutions marketing for Mendix, said Mendix Assist will enable new developers to write better code faster, while experienced developers will be able to employ Mendix Assist to address routine issues that often get overlooked faster. Mendix Assist not only advises developers on what to do next but also generates recommended actions in previous logical steps based on whatever new code is introduced in the application.

To make these capabilities more accessible, the summer release of Mendix adds Mendix Studio and Studio Pro, a set of integrated development environments designed to be accessible to both citizen and professional developers, to the low-code platform.

In addition, Mendix is adding support for native mobile application development by adding support for React, a set of JavaScript libraries for building user interfaces for mobile applications, and including a native JavaScript editor. Those capabilities eliminate the need to rely on emulation software when building mobile computing applications, said Koenigsberg. Because the Mendix IDEs integrate low-code abstraction across iOS, Android, Java and Objective-C programming models, it also becomes easier for mobile developers to build reusable components. Mendix has also included more than 50 pre-built connectors to mobile device capabilities, such as camera, geolocation, biometric authentication and Bluetooth enablement, along with support for conversational chatbots and voice interfaces such as Alexa and Siri.

Longer-term, Koenigsberg observed that it’s conceivable the AI capabilities being included in application development tools eventually will share insights with AI models being incorporated into continuous integration/continuous deployment (CI/CD) platforms. Proposed changes to an application could, for example, might be assessed from a performance perspective before any lines of code are written.

In the meantime, the rate at which low-code applications will be moving through CI/CD platforms is about to increase substantially as developers augment their own skills by relying more on recommendations surfaced by AI engines. The quality of those applications should also improve as AI engines check, for example, to make sure all the appropriate compliance and cybersecurity controls are in place.

It’s not likely AI engines will replace the need for developers anytime soon. However, it is apparent all the little things that routinely conspire to slow application development and delivery processes increasingly will be automated. That should make it easier for a larger number of organizations to embrace best DevOps processes. After all, the primary issue with DevOps adoption these days is not necessarily appreciating the value of the concept, but rather managing pipelines that still tend to get clogged every time an application fails to pass a gate because a developer forgot to do one thing or another.

— Mike Vizard

July 26, 2019
How IoT Security Risks Derailing Robotics

Driven by advances in technologies such as artificial intelligence and deep learning, the mobile robots market is projected to grow to $54.1 billion by 2023 from $18.7 billion last year, according to Markets and Markets. But for robotics to progress, security for the internet of things (IoT) needs to be addressed fast, before the reputation of the entire industry suffers a permanent setback.

Every industry segment has a journey to maturity that involves ramping up how seriously it takes security. For robotics, that time has come. Failing to start robotics projects on the right software foundations will see confidence continue to drop, impeding the rate at which new solutions are created.

The future of robotics rests firmly on the shoulders of the IoT. Robots act as one part of intelligent ecosystems—they depend on IoT to link various sensors and smart meters, and pass data to and from third parties.

The self-driving car, for example, is a robot orchestrated by various smaller devices and smart sensors. In fact, robots are being created to tackle every conceivable problem. Take the Google-funded RangerBot, an underwater machine designed to track down a species of starfish responsible for coral reef destruction, or Small Robot Company, a startup tackling farming deficiencies with bots that autonomously feed, seed and weed arable crops.

However, while IoT is an established concept and connected devices are now mainstream, its full potential has not been realized in part because of security concerns. Ninety percent of consumers lack confidence in connected devices, according to a recent survey. These concerns are more than valid, as nearly half of companies are unable to detect when a breach occurs, and only 15% of budgets are earmarked for IoT security.

With IoT security a lingering weak spot, one major attack within robotics, could have a domino effect on the entire industry. Add to this the verticals that robots are beginning to infiltrate—health care and agriculture, for example—and the threat becomes immediate and very real.

Rising levels of sophistication within robotics go hand in hand with more targeted and damaging attacks. For example, telesurgery uses robotics to help surgeons perform procedures remotely. A malware bug in this scenario could mean the downing of tools, threatening the patient’s life.

Researchers at Brown University proved how easy it is to hack robots. The industry simply will not be sustainable without the backing of a secure, connected IoT network.

Robot manufacturers, therefore, must build with a security-by-design mindset. This begins by selecting a robust operating system from the outset—secure now, but also ready for future market demands.

Hackers are constantly evolving their activities and businesses must be flexible in their approach to security, shedding the old hardware-centric view of IoT security.

Software no longer can end when a device is shipped. It must align to the lifespan of a robot and be able to update whenever there is a potential flaw. The world of mobile took many years to get to grips with this. Robotics and IoT developers should learn from their mistakes.

One way developers can safely build and secure software is through snaps, containerized software packages on an open platform for building and publishing applications to an audience of millions.

If a security vulnerability is discovered in the libraries used by an application, the app publisher is notified so the app can be rebuilt quickly with the supplied fix and pushed out. This allowsr developers to stay focused on innovation while ensuring the longevity of the robotics hardware.

It remains unclear where the onus lies for IoT security, with nobody holding anyone else to account. Market constraints often prevent device makers from putting more budget than what is necessary into design security, when there is so much pressure to innovate ahead of competitors.

These issues need to be solved. Robotics has too much potential to let security concerns stunt their growth.

— Tom Canning

July 25, 2019
Micro Focus Lays DevOps Foundation for RPA

Micro Focus has announced the general availability of Micro Focus Robotic Process Automation (RPA), which sets the stage for employing DevOps processes to construct highly automated business processes.

Thanks to the rise of artificial intelligence (AI) and digital business transformation initiatives, interest in RPA has increased significantly. However, rather than merely using RPA to build bots to automate a narrow set of static processes, RPA is now leveraging machine and deep learning algorithms to optimize a wider range of dynamic business processes and adjust more easily to changes within an application.

Travis Greene, senior director for IT operations management ay Micro Focus, said new capabilities such as object recognition technology included within Micro Focus RPA enables the platform to detect and automatically adapt to changes to the user interface.

Other new capabilities now include the ability to visually or programmatically create workflows using a tool that records screen actions, the ability to more easily combine graphical and programmatic workflows spanning multiple platforms and the ability to deploy hundreds of thousands of bots as needed. A centralized dashboard manages the bots, all of which have unique IDs and encrypted, role-based credentials.

As RPA and other forms of AI continue to be integrated and embedded with applications, Greene said it’s a matter of time before DevOps practices are extended to RPA platforms. Organizations will seek to train bots continuously as more data is made available, which in turn will drive updates to applications that are dependent on those bots. In most cases, depending on the complexity of the task, the bots will be built using a mixture of application programming interfaces (APIs), command-line interfaces (CLIs) and graphical tools. Organizations will need to define the processes to determine how and when each organization will opt to invoke one interface instead of another, either in isolation or combination, added Greene.

Historically, organizations have been able to apply RPA to static processes with mixed success. With the rise of AI, the scope of those efforts has become more ambitious. Business leaders are too inclined to believe that just about every business process can be automated using a combination of AI and RPA. However, AI and RPA still are confined to relatively narrow use cases involving highly predictable outcomes.

That’s changing, however, and RPA and AI are becoming more pervasive across the enterprise. DevOps teams will need to find ways to update both their applications and RPA/AI models without breaking one or the other. Most of the current RPA/AI models are highly sensitive to any change made to the application environment they are monitoring. And, unfortunately, the data scientists who are largely responsible for constructing those RPA/AI models don’t yet tend to have much of an appreciation for best DevOps processes. There will, however, come a day soon when these two worlds will collide.

— Mike Vizard

July 23, 2019
3 Examples of AI at Work in DataOps

Artificial intelligence (AI) is making all the difference between innovators and laggards in the global marketplace. Yet, implementing a state-of-the-art DataOps operation involves a long-term commitment to putting in place the right people, processes and tools that will deliver results.

In this post, we look at three organizations that are doing cutting-edge work in the field of DataOps. We look at the specific strategies they use and the results they’ve seen as they navigate the uncharted waters of DataOps.

Uber: End-to-End DataOps

Uber is very vocal about the way it leverages AI across its apps and services globally. The company runs all its machine learning (ML) models on a platform called Michelangelo and use many other proprietary tools such as Manifold, Horovod and Piper to optimize and scale these operations.

Michelangelo helps manage DataOps in a way similar to DevOps by encouraging iterative development of models and democratizing the access to data and tools across teams. On top of this platform, Uber uses other tools to help data teams.

Uber ran its models in Tensorflow and realized model training was taking too long because of the large data sets it was dealing with. Taking a page from Facebook’s data team, the company realized the power of distributed training. This would enable Uber to run data operations in parallel on hundreds of distributed GPUs, rather than on a few GPUs. Yet, managing all this would be a challenge, which is why Uber built a tool called Horovod. Now under the auspices of The Linux Foundation’s Deep Learning community, Horovod supports additional frameworks such as PySpark, Petastorm and Apache MXNet. Additionally, using mixed precision training, Horovod is able to optimize the use of memory when running models, and in doing so, it enables running larger data models or running numerous mini-batches of data in parallel.

Manifold is a visualization tool that helps to gauge the performance of various models against each other. Rather than use a simplistic, two-dimensional axis to plot performance data, Manifold groups large quantities of model performance data into subsets that avoid duplicating the important indicators of performance. This reduces noise and helps speed analysis of the data. The lesson: Look beyond the trees to view the entire forest when it comes to metrics.

By adopting an end-to-end approach to DataOps and leveraging specialized tools at every step of the pipeline, Uber is pioneering the use of AI in its large-scale consumer applications.

Netflix: Offline First, Then Online

Content recommendation is key to the user experience in the Netflix app. Highly relevant content makes the experience personal and drives higher engagement. However, making this happen behind the scenes involves working with large quantities of data and overcoming obstacles every step of the way.

Netflix uses Spark to power content personalization. While the company has existing models in production, it always is testing new models and hypotheses to improve on existing ones. However, every new model needs to be tested against an older model and should be proven to be better before being implemented. Netflix does this model testing offline rather than online in the live app. The company makes a copy of historical data and performs offline feature generation on this data. The features are stored in a Hive store in AWS S3, and subsequent feature generation and model training is done in Spark. Once a new model is ready, it is run in parallel with a live model and, if it performs better, it is deployed.

Key to the success of this operation is the logging of historical facts from online to offline. This includes Netflix-specific facts such as video, user and computation facts. Once this historical data is stored, a data worker is able to run simulation tests against it using the newly developed models. These simulation tests are a great way to test models without breaking things and save time by failing fast. Once tested and compared with live models, Netflix goes on to establish different SLAs for different microservice apps that these models serve. All this effort results in a highly relevant recommendation engine and a very personalized experience for Netflix users.

Airbnb: Tapping Into Related Data Stores

Airbnb uses AI to improve the search rankings and relevance of its “AirBnB experiences” offering. The company started simply, only recording users’ daily searches, clicks and bookings. Soon, it created features on top of this raw data, including features such as experience duration, price of experience and user reviews, which would help build models. Airbnb used a ranking model to give weight to experiences that had a better conversion rate, better user reviews and a lower price than those with just a higher booking volume. This improved experience bookings by 13%.

Going further, Airbnb tapped into the other user data it had—homes booked by Airbnb users—to further personalize the ranking of experiences. This is where things get interesting, as AirBnB is able to show experiences that are more likely to match a user’s interests. This is only possible by looking beyond a single data store and looking for relevant data that may exist elsewhere in the organization. This personalization further improved experience booking by 7.9%.

Similar to Uber, Airbnb uses a combination of offline and online scoring for models. While the historical data is great to get started with and run basic simulation tests, online scoring unlocks additional features such as user location, search filters and login status that wouldn’t otherwise be available offline.

Conclusion

AI is changing everything, and DataOps is no exception. By helping companies to make sense of their data more quickly and with less reliance on human effort, AI drives faster, more consistent results.

The good news for most of us is you don’t need to be a major tech company such as Uber, Netflix or Airbnb to take advantage of AI-driven DataOps. Tools such as Unravel are making AI-powered data insights practical for organizations everywhere, without requiring them to build bespoke DataOps solutions.

This sponsored article was written on behalf of Unravel.

— Twain Taylor

July 16, 2019
Low-Code Providers Continue Enhancing Their Development Platforms

Enterprise low-code development continues to grow in popularity as software vendors continue to unveil tool enhancements designed to help business users and non-developers build functional applications and business processes.

The help from low-code platforms (and other platforms that are integrating low-code capabilities) couldn’t come at a better time, as enterprises continue to struggle to digitize operations and dig through their application backlogs.

Consider open source digital process automation platform Bonitasoft and its recent increased support for low-code development and what the company promises to be accelerated user interface development for more customized user experiences. The platform (including a new Bonita Cloud edition) helps to enhance collaboration between professional developers and so-called citizen developers, according to the company.

Recently, low-code development platform provider OutSystems announced it is adding new artificial intelligence (AI) and machine learning capabilities into its platform. The company said the enhancements provide customers the ability to help automate the creation of self-service portals, respond to text and voice queries and improve customer service.

OutSystems said citizen developers now can more easily build communication channels into their apps and better utilize virtual assistants such as Alexa, Google and Siri, as well as common chatbot platforms. The company claimed developers don’t need any specialized development skills or knowledge in data science to use these features. “These new, high-level components package everything organizations need to implement rich use cases from end to end. As a result, many more companies can easily adopt AI and machine learning to positively impact their business,” said António Alegria, OutSystems’ head of artificial intelligence.

According to the research firm Gartner, by next year, consumers will manage 85% of their relationship with enterprises without having to interact with a human. However, despite the use of AI having tripled over the past four years, two-thirds of organizations have yet to deploy AI in some form, it noted.

According to Research and Markets, the global low-code development platform market size will reach just over $27 billion by 2022, up from $4 billion in 2017 and a 44% annual growth rate.

According to the research firm, it is the increased demand in software automation and innovative applications and the ease of development of web and mobile apps that are driving enterprise demand for low-code development. “Moreover, the cloud-based deployment mode is gaining high traction in the market, as it requires less capital investment, helps decrease the operational and maintenance costs, and reduces the management’s efforts,” the research firm said.

While many analyst market predictions do fall short, as long as low-code platforms reduce costs and help organizations move forward rapidly with their digitization efforts, low-code development is here to stay and will continue to grow.

— George V. Hulme

July 9, 2019
How DevOps Teams Can Skill Up on DataOps

If you’re a DevOps engineer, you probably don’t think of data operations as a key part of your job. You help to write code and deploy and manage applications, while someone else handles data management and analytics, right?

Probably not. Although it might be comforting to imagine that every company has a dedicated team of data specialists on staff who oversee data operations, the reality is that the vast majority of companies employ no data scientists at all. Only around 6% of large enterprises have data scientists on staff, and virtually no small and medium businesses employ data scientists.

What this means is that at most companies, effective data management is the responsibility of any and all IT employees that these companies have on staff, including (especially) the DevOps engineers who bridge together development and IT operations. If the DevOps team doesn’t take responsibility for data operations, it puts at risk the applications that depend on healthy data flows to do their jobs.

That begs the question: How do DevOps engineers (most of whom weren’t trained in the intricacies of data management) learn to support data operations? Keep reading for tips.

Data Operations and Software Delivery

Data operations (DataOps) refers to all of the processes and workflows required to collect, store, manage and analyze data.

Traditionally, DataOps were siloed away from software delivery and management. The people who wrote, deployed and monitored applications were different from those who dealt with an organization’s data. That approach worked well enough in previous decades, when the data that organizations had to manage was less massive and less valuable.

Today, however, data has become too critical a component of effective software delivery to be left in a silo. Many modern applications depend centrally on data. If your app includes a predictions engine or a recommendations feature, it requires data to do its job. Similarly, the monitoring tools that IT teams use to assess application health are driven by data. Data collection and analysis are critical for security operations, too.

Thus, modern application delivery pipelines depend not just on the smooth flow of code from development into production, but also on a healthy flow of data. You can no longer isolate DataOps from application delivery (and even if you could, you probably don’t have a dedicated team of data scientists to handle your data anyway, as noted above).

Adding DataOps to DevOps Software Delivery

How do you actually go about integrating DataOps into an application delivery pipeline in a DevOps organization? Consider the following pointers.

Automate, Automate, Automate

Automation is just as important for effective DataOps, as it is for DevOps software delivery. By embracing tools that automate data collection, management and analysis, IT teams ensure that DataOps flow as smoothly as the software delivery pipeline.

What’s more, data automation tools can help reduce the amount of DataOps work that general IT staff need to perform. That’s a key benefit for DevOps engineers who don’t have extensive DataOps experience.

Embrace AI

Another way of reducing the amount of DataOps work imposed on IT staff who are not trained as data specialists is to leverage AI to help them make decisions within DataOps workflows.

Until just a few years ago, only a few tools used AI in a practical way to power DataOps. That is now changing, and modern data management tools make effective use of AI to reduce the decision-making burden on human engineers.

Focus on the Data that Matters

Within the context of software delivery (and elsewhere), some data have more value than other data. Since you have finite staff and material resources to devote to DataOps, identifying which data is most important for your applications is critical for optimizing your DataOps workflow.

In other words, just because you collect data doesn’t mean you necessarily need to store and/or analyze it.

Be Infrastructure- and System-Agnostic

It can be tempting to rely on the data management tools provided, along with the cloud infrastructure you use or the platform you run on. However, although these tools might be helpful to a limited extent, relying too heavily on them puts you at risk of being locked in and having to rebuild your DataOps from scratch if you migrate to a new infrastructure or platform.

For that reason, choosing DataOps tools that work with any environment is a wise idea.

Educate Your Team

The final pointer for helping your DevOps team do DataOps is simple: You need to educate them about data and what makes it work well. Don’t expect to turn them into data scientists, but they should understand concepts such as data quality, and recognize the bottlenecks that can get in the way of healthy data performance. Although these concepts may seem basic to someone trained in data management, they are usually foreign to DevOps engineers.

Conclusion

To say that we live in a data-driven world is a cliché. But it’s a cliché that is eminently true if you work in software delivery. In many cases, you can’t do DevOps without having healthy DataOps, too. That fact, combined with the lack of dedicated data teams at most companies, means that DevOps engineers must learn to integrate DataOps into their workflows.

This sponsored article was written on behalf of Unravel.

— Chris Tozzi

July 2, 2019
Ensure Your AI is Working for Your Data Team
AI tools hold great promise for DataOps. They can help teams identify the best apps to migrate, identify optimum cloud infrastructure and even validate decisions. But how can an IT team ensure the AI tools it uses are actually advancing its DataOps goals?

A hard pill to swallow is that even the best-designed algorithmic decision-making and AI tools are prone to errors. Fortunately, this is unlike errors in software development, which are largely caused by development. AI errors emerge from poorly defined project goals and the organization’s tendency to treat the technology as if it’s magic. Without solid requirements, model monitoring and regular check-ins with reality, it is almost impossible for AI to deliver.

This is why audits must be a mandate when introducing any AI tool that greatly influences business decisions. However, before the audit is conducted, ensure that the following points are met:

Ensure the AI Tool is ‘Auditable’

This step ensures that organizations bring in the right AI tools to their arsenal.

An AI tool is auditable when it is:
- Transparent: When AI tools are used to enhance the decisions a business makes, stakeholders must be able to peer into the system to ensure the legitimacy of the algorithm decision. Ensure the AI tool you choose has a logging system that makes it transparent as to how the system is making a decision, what factors most influence it and how confident it is in that decision. This allows decision-makers to trust in the system, as well as creates a feedback loop for the developers to take in real-world data and enhance and fine-tune the system to your needs.
- Built with best practices: Just as security engineers follow best practices when securing a network, AI models and tools built with best practices in mind fare better when in use. As a general rule of thumb, all AI models must keep detailed access and process logs, incorporate failback mechanisms and generate regular health reports.
Set Up Markers for Success

Don’t approach an AI audit as though it’s a sandbox question. If teams are clear as to what an AI tool must achieve and how to measure its success, only then will an audit be able to judge the performance of the tool.

It is helpful to set up markers that describe success. This could be the number of cases the AI model has rightly predicted, the time it takes, the number of edge cases it is able to handle and the number of times the AI decision collided with a human decision.

The Audit

Once the AI tool has been integrated into the organization and is in active use for a defined period, audits must occur. They help you answer the following questions:
- Is the algorithm suitably transparent to end-users?
- Is it likely to be used in a socially acceptable way?
- Might it produce undesirable psychological effects or inadvertently exploit natural human frailties?
- Is the algorithm being used for a deceptive purpose?
- Is there evidence of internal bias or incompetence in its design?
- Is it adequately reporting how it arrives at its recommendations and indicating its level of confidence?
(HBR: “Why we need to audit algorithms”)

Assess the Algorithm’s Fitness for its Purpose

From a use-case perspective, AI algorithms in modern software applications tend to fall into two main categories. One is offering insights that help humans make decisions—this is what AI-driven analytics is all about. The other is allowing software applications to undertake automated actions entirely on their own, as AIOps tools increasingly do.

A basic step in auditing your algorithm is to determine which type of use case it is supposed to support, then evaluate how well it achieves it. This is important to do regularly, because AI use cases can drift as applications evolve and the original purpose of an algorithm might change over time. (It’s common for algorithms to be implemented originally as a way to offer insights, but for applications to be updated with the expectation that the algorithms can support automated decision-making.)

Compare and Contrast Algorithmic Decisions Against Human Decisions

Though some of the inner workings of an AI model are a black box to even data scientists, the result of those inner workings is not. An AI audit must set up a process and team that regularly and actively monitors the decisions AI makes and compares them against decisions that a human would make.

This is useful in identifying early on whether the AI tool has internalized biases. With an AI system, input greatly affects learning. For example, if a tool used to forecast purchase orders is incorporated by the organization during peak sales season, the tool will assume that this is the organization’s baseline performance. This causes erratic predictions in the long run.

Active Experimentation

Though an audit should uncover any bugs, biases or security concerns of an AI tool, it also could be used to test system variables. An AI tool is generally built on assumptions of the problem borders and trends. However, assumptions are very fragile in reality. The baselines of reality could change, causing the AI to produce questionable results.

An audit also must be able to revisit AI tools with new assumptions, annually or biannually, to uncover any new parameters that could help improve the accuracy of the tool.

Interviewing Users of the System

Ultimately, an AI tool is put into place to either reduce the workload of humans or enhance it. Daily users have vast knowledge and perspective into not just how the tool works but also its limitations and quirks.

Conclusion

More often that not, the mystique of an AI model is used as an excuse for erratic results and poor performance. By building a robust and transparent audit system, your organizations can ensure your AI tools are actually working for your teams—and if not, it can at least uncover rare scenarios and edge cases and help mitigate them.

This sponsored article was written on behalf of Unravel.

— Swaathi Kakarla
June 27, 2019
Moogsoft Advances AIOps Workflow

Moogsoft is moving to make it easier to create custom workflows around its AIOps platform while expanding the number of integrations provided to include connectors to New Relic Insights, Microsoft Teams and proxy support for polling integrations with IT platforms Zenoss, Zabbix, vCenter, vSphere, Solarwinds, Spectrum and SevOne.

Gabe Coelho-Kostolny, senior product management and product marketing leader at Moogsoft, said a Workflow Engine that has been added to version 7.2 of the Moogsoft AIOps platform makes it possible to trigger actions such as notifications, ticket creation and other automated tasks within Moogsoft and external systems. Those actions can be employed within incident management workflows ad other automated remediation tools, he said.

Moogsoft has also added tools that make it possible to visualize alert clustering algorithms in real time and, if needed, fine-tune them. That capability is intended to provide IT operations teams with more confidence in the recommendations being generated by the AIOps platform, Coelho-Kostolny said.

Finally, Moogsoft is making available an analytics dashboard dubbed Individual Statistics, which enables managers to drill down from the team level to better understand the workload and key performance indicators of each team member.

Coelho-Kostolny noted there is already a high correlation between organizations that have adopted best DevOps practices and those moving to implement AIOps. The complexity of many DevOps environments is in many cases forcing the AIOps issue because all the alerts being generated creates a level of fatigue among human IT administrators that results in them not being able to keep pace with the overall rate of change, he said. The concern now is no longer so much whether human IT administrators will be replaced by artificial intelligence (AI) as much as it is enabling IT teams to stay on top of all the tasks required. The opportunity now, he said, is to make it easier for organizations that have embraced DevOps to also customize the AIOps platform.

Of course, AIOps platforms presuppose that IT organizations have access to massive amounts of accurate data about the IT environment. Collecting that data often winds up being a bigger challenge than most organizations appreciate. In general, Coelho-Kostolny said it’s clear AIOps platform adoption is reaching a tipping point in the enterprise. Organizations, however, need to be careful distinguishing between what is truly artificial intelligence and what may be simply a rules-based engine that has been added to an existing IT service management (ITSM) platform, Coelho-Kostolny said.

Regardless of the AI path chosen, the ongoing shortage of IT expertise with critical skills means organizations will continue to experiment with AI. The real issue is determining to what degree of confidence IT teams will have in those AI platforms. After all, if the assumptions an AI system is relying on to make a recommendation prove faulty, the potential for harm tends to be a lot greater than any mistake a single human IT administrator might ever make on their own.

— Mike Vizard

June 17, 2019