devopsy.kinsta.cloud

Tag: Predictive Analytics

How Continuous Transformation Can Deliver More Value
Continuous transformation in IT operations is more than just an industry buzzphrase. In fact, continuous transformation is a critical survival tool for any cloud and IT solution landscapes today. To achieve continuous transformation, innovation must continue well beyond a migration to the cloud. This continuous innovation during day-to-day IT operations is the most critical aspect to achieving month-over-month best-in-class operational benefits.

Simply “lifting and shifting” systems to the cloud without modernizing precludes most of the key benefits that your systems can potentially deliver over the long term. For example, some of the outcomes of this approach, including bug fixes and other common errors, cause downtime and delays that businesses cannot afford in these highly competitive environments. In addition, unplanned downtime and disruptions impact revenue and can damage brand reputations, both of which detract from a business’s ability to compete and win in their market.

Proactive vs. Reactive

To stay ahead of the pack, companies must shift from a reactive to a very proactive posture. Error prevention is far more effective than fixing errors and resolving issues as they happen. To this end, companies should be leaning on service providers that proactively cover the entire application and infrastructure landscape, end to end, through customized, out-of-the-box solutions. For example, proactivity is rapidly enhanced by including predictive analytics and other fault-finding algorithms in software, such as:
- Analytical anomaly detection.
- Cloud-enhanced and optimized monitoring templates.
- Self-healing automated reaction, from disk extensions, cluster and scaling groups to ensure an issue does not result in downtime, simply a post-event investigation.
A company’s IT needs are also continually changing as business landscapes evolve. For example, being on the cloud and using service level agreement (SLA) measurement has now advanced to become mission-critical in two areas: bottom-up technical SLAs (as per usual) and now, more importantly, top-down business transactional SLAs.

Here are some ways IT could streamline its operations and deliver the most value back to the business:
- Operations Analytics – An operations analytics tool allows for capacity planning, monitoring usage trends and patterns over time. Should deviations from the anticipated usage occur, IT teams can react immediately, implementing remedies before the situation escalates to an incident. This early intervention could be done either manually or automated. Automated scripts, triggered by predictive alarms, can restart services and adjust virtual resource allocation or any other remedial action necessary.
- Predictive Analytics – Future planning is essential to successful business operations. Predictive analytics improves on this by using precursor and antecedent information to predict system failure with an associated confidence level. IT teams know when resources operate outside their anticipated range and how likely they are to fail within the next period. System failure is usually associated with a surge in user activity. This is not easily predicted, driving the need for advanced predictive analytics. Armed with this information, IT teams can carry out preventative maintenance on data centers and associated systems, such as that done for manufacturing machines in industrial applications.
- Automated Self-Healing Systems – By combining artificial intelligence and machine learning, digital innovation has made it possible for business systems to self-heal. Advanced, automated software packages monitor business systems, finding errors and fixing them without human intervention. These systems address performance variations and restore outages as well.
Through self-learning algorithms, the software learns and understands the normal business operations and its reactions to irregularities and errors. Using this information, the software reacts appropriately when it detects errors, automatically restoring the business system to normal operations. These automated, exponential self-healing packs utilize innovative, continual logic learning as part of their standard software package. These systems also resolve issues in a fraction of the time it takes through traditional manual intervention. Typically, the end user doesn’t even realize that an error has occurred, improving the user experience dramatically.

The implementation of self-healing systems is relatively quick and can be done in as little as four weeks. As a result, IT teams will have access to advanced troubleshooting and automated diagnostic, enabling fast, root-cause analysis of any errors that occur—implementing the solution through guided prompts, documents the resolution for future reference. And, if the same error occurs again, the solution will automatically be executed.

Automated self-healing systems offer a range of benefits, including business and IT cost reduction, decreasing unplanned downtime and risk-free digital footprint expansion for continuous transformation. IT professionals can work strategically, delegating mundane and repetitive tasks to automated processes, freeing time and resources to carry out more advanced tasks aligned to business goals.
September 3, 2020
OpsRamp Adds Recommendation Engine to AIOps Platform

OpsRamp added a recommendation engine to its artificial intelligence for IT operations (AIOps) platform that makes use of predictive analytics to suggest potential actions to either improve performance or avert downtime.

The OpsRamp Winter 2020 Release also adds 19 cloud monitoring integrations for Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP), along with dynamic topology maps for Azure and GCP. OpsRamp currently provides integrations with more than 120 different cloud services.

Finally, OpsRamp has added an ability to detect seasonality patterns that would otherwise generate alerts, an Alert Stats widget that makes it easier to keep track of raw alerts, synthetic monitoring and agentless discovery and monitoring of Windows Server instances.

Bhanu Singh, senior vice president of engineering and DevOps at OpsRamp, said as IT continues to become more complex resistance to AI within the ranks of IT staffs is declining. Organizations realize that IT specialists are an expensive resource, so they need to maximize the capabilities of IT generalists as much as possible, he said. OpsQ Recommend Mode adds a predictive analytics module to the company software-as-a-service (SaaS) platform for managing IT that provides explainable and transparent analytical recommendations to help the average IT administrator head off issues that would require a specialist to solve later, said Singh.

That visibility is critical because it still leaves the decision about what to automate in the hands of the IT administrator. However, as OpsRamp continues to learn the nuances of the IT environment, the more confidence IT administrators will have in the recommendations being surfaced, he said.

It takes about 24 to 48 hours for the OpsRamp platform to learn enough about an IT environment to start surfacing recommendations, Singh noted, adding because OpsRamp is a SaaS platform, setting up the platform requires minimal effort.

To some degree an increased reliance on AI among IT professionals is inevitable. AI may never eliminate the need for IT personnel, but many of the mind-numbing rote tasks that IT administrators perform today soon will be automated. The challenge that creates, however, is the IT staff needs to be able to manage IT at a higher level, which means many entry-level jobs in the IT sector may no longer be required. The challenge then becomes finding IT professionals who have the right level of skills, when many of the entry-level jobs for which they have been trained no longer exist.

In the meantime, a race is clearly on to infuse IT management platforms with machine learning algorithms. Startups such as OpsRamp are looking to supplant legacy IT platforms that were designed for a previous era of IT. However, the providers of those platforms are clearly investing heavily in AI as part of their ongoing modernization efforts. Each IT organization will need to decide for themselves to what degree the rise of AI warrants switching management platforms.

Regardless of the platform selected, however, the one thing IT teams can count on is that AI platforms are only going to get smarter as they continue to evolve.

— Mike Vizard

March 2, 2020
4 Challenges OEMs Face When Implementing New Technology

Like many others, the manufacturing industry is moving to a subscription-based business model. This new way of doing business is being referred to as servitization and it is forcing manufacturers to no longer strictly sell new products, but instead sell access to and the outcome those products deliver. This is fundamentally changing the way manufacturers have traditionally done business and is not only encouraging a shift to subscription-based pricing models, but also Products-as-a-Service.

As manufacturers implement the processes and resources necessary to make servitization a reality, implementing new technology will be key to success–laying the foundation for success today and in the future. More specifically, machine learning and predictive analytics will become especially important for predictive maintenance, process optimization and supply chain management. While there are opportunities to capture a significant competitive edge using these technologies, there are also significant challenges specific to the manufacturing industry. Some of these noteworthy challenges include:

Efficient and Secure Data Connectivity

Complex, capital equipment (think construction equipment, agricultural equipment or industrial machinery) is built and designed to last, on average, 30 to 40 years. As a result, these machines must be very reliable. Today, however, the majority of equipment in service has upwards of 20 to 30 years remaining in its lifespan. However, almost none of these products were built or designed for IoT connectivity–creating the challenge of retroactively fitting existing equipment with IoT-enabled parts.

In addition, a majority of this equipment operates in some of the harshest and most rugged environments in the world. Whether it’s an electro-submersible pump 3,000 feet below the surface of the ocean or a mining truck in a remote mine with little to no cellular connectivity, efficiently connecting this type of equipment to collect relevant data is a difficult challenge and oftentimes cost prohibitive.

Even if manufacturers manage to connect this equipment, there is an added challenge of doing it in a way that ensures cybersecurity. Rogue elements that could gain malicious access to this equipment could have serious safety and economic consequences–and could even implicate national security in some instances.

Sparse Data

Newer machine learning algorithms, such as Deep Learning and Recurrent Neural Nets, have been groundbreaking for capabilities like image and voice recognition. However, these algorithms are also very data hungry, requiring millions–if not billions–of similar data points from millions of users.

Because complex industrial equipment is intended to last for decades, failures are generally rare. Think about it: There have been just a handful of actual engine failures during flights for the entire global commercial fleet, which amounts to nearly 30,000 aircraft. Assuming an average of 2.5 engines per plane, the global commercial aircraft engine fleet is less than 100,000. While certainly a large number by itself, it is relatively small when considering these engines vary greatly in design and thus any single engine type that could be modeled together would be even smaller.

Manufacturers of industrial equipment or durable goods also tend to be reluctant to share data, especially if their competition could potentially benefit from it. The challenges of efficient connectivity, rare failures and relatively small numbers of similar equipment in a fleet leads to the sparse data problems in the Industrial IoT (IIoT) realm.

Domain Expertise

While domain expertise to successfully apply machine learning and predictive analytics is required in the consumer world, in the manufacturing space, it is much more critical. Machines and processes are designed using physics and engineering principles. And, the context in which machines operate and behave is very important for accurate training models and analyzing results. Without appropriate context, there becomes a “garbage in, garbage out” problem as sensor data and machines alone do not capture or provide any context. This is where domain experts who can make sense of historical data for accurate model training as well as interpreting results are critical.

The impact of making an incorrect decision can have expensive–and sometimes catastrophic–results that aren’t seen in the consumer-facing domain. For example, re-routing an aircraft mid-flight could cost millions of dollars for an airline. If that decision is made because of bad data and an incorrect prediction, it would be a very costly mistake. For us as consumers, we have some aspect of domain expertise for problems in the consumer world, whereas a trained pool of engineers with the relevant design, operational and maintenance expertise is a very small pool. And this, combined with the problem of baby-boomer retirement where expertise is lost at a significant rate, there is a real challenge to capture the knowledge and best practices required to deliver value.

Data Ownership

Currently in the manufacturing sector, there aren’t industry standards for data security, ownership or accessibility. Currently, many manufacturers are wrestling with who owns and has access to the data. Is it the manufacturer who designs and builds the equipment? Or, is it the third-party entity that rents and operates it? Or, is it the financial entity that may be the legal owner? What about the supplier that provides a component that is part of a larger piece of equipment (e.g. an aircraft engine that works in conjunction with the aircraft, with each component manufactured by a different entity)?

Manufacturers being risk-averse in nature, are being very conservative in interpreting and applying the myriad patchwork of regulations being brought in other industries, including data security standards. This is limiting them in adopting more connected processes and strategies. The industry must accelerate efforts to establish industry-wide standards as well as regulatory frameworks for data ownership, data-sharing, interoperability and testing of systems and end-to-end security to overcome the roadblocks for faster adoption of these technologies and realize the promised benefits.

The shift to servitization will require manufacturers to completely re-think how they operate–new organization structures and skilled resources, new incentive models, new KPIs to measure success and new processes. They will have to become data-driven organizations investing in technologies to connect and track products, collect data and efficiently analyze massive amounts of operational and service data, using technologies like IoT, machine learning and predictive analytics. This will strain manufacturers’ existing organizations and IT infrastructures, necessitating investment in highly scalable, cloud-based solutions to lay the foundation for a successful future. Manufacturers that embrace these changes will be the winners, while others will struggle to stay relevant. In fact, the ones that can successfully adapt to these paradigm shifts will be able to gain significant competitive advantage and market share.

— Vivek Shah

August 23, 2019
Machine Learning and Predictive Analytics Are Reshaping Manufacturing
Regardless of industry or vertical, companies everywhere are encountering a new generation of customers with ever-evolving expectations. These new demands are forcing brands to redefine the way they do business.

Today’s customers overwhelmingly favor the simplicity of subscription models, in which they pay a flat monthly fee for access to a product in the form of a service. For example, consumers no longer buy or rent DVDs; they subscribe to Netflix. And more prefer to use the services of Uber or Lyft over traditional car ownership. As these digital-age consumers enter the industrial workforce in large numbers, they are causing business models to be redefined, leading to an increasing trend toward subscription-based services in manufacturing industries.

This shift toward a subscription-based business model in the manufacturing industry is being referred to as servitization, wherein manufacturers no longer strictly sell new products but instead sell access to and the outcome those products deliver. This new way of delivering services is forcing manufacturers to shift to subscription-based pricing models, where product-as-a-service is the norm. Manufacturers of complex industrial equipment, because they operate in a B2B environment, can attribute this shift to another major cause: Their customers are also dealing with complex operating paradigms, forcing them to improve productivity and capital utilization. In these scenarios, customers are looking to simplify their businesses and increasingly favor suppliers that can help minimize their risks in operating a profitable business.

The full realization of servitization, which could take up to 15 years, especially impacts manufacturers’ after-sales service organizations (the service delivered after the initial sale of a product). In today’s current reactive, break-fix service model, the responsibility to maintain and ensure equipment availability falls on the customer. In a servitization model, however, manufacturers own the responsibility of maintenance and repairs and need to focus on maximizing product uptime, since revenue can only be earned when their products are available to generate output in the field.

Machine Learning and Predictive Analytics in Manufacturing

The use cases of machine learning and predictive analytics are as varied as the industries within manufacturing. However, there are a few common use cases that apply to most manufacturing verticals, typically grouped under terms such as smart manufacturing, industry 4.0 or industrial internet of things (IIoT).
1. Predictive Maintenance: Predictive maintenance is the most well-understood and varied use case in most manufacturing industries. Here, data from process monitoring sensors such as temperatures, pressures, flows, vibrations and more are captured in real-time and used in pattern recognition software to detect the earliest symptoms of wear and tear predictive of eventual functional failures. Early detection and prediction can help prevent failures or at least plan for eventual corrective actions leading to minimized downtime. Downtime—especially unplanned downtime—can be a very expensive event, possibly leading to millions of dollars in losses. Some analysts estimate unplanned downtime in certain industries is worth $20 billion.
2. Process Optimization: Process optimization, in which existing processes are updated and optimized based on historical data, is a critical use case, especially in industries such as power generation, oil and gas refining, petrochemicals and chemicals. In this instance, sensor data feed machine learning algorithms for yield and quality optimization of output components for different combinations and quality of input raw material feedstocks. This also helps with energy efficiency, thus improving sustainability and profitability for these process manufacturers. In the global airline fleet, for example, a 1% fuel savings would save $30 billion over the next 15 years.
3. Supply Chain and Inventory Management: High levels of raw material, work-in-process and finished goods (e.g., replacement service parts) inventory are one of the highest contributors to inefficient capital utilization for discrete manufacturing industries. Using machine learning to improve raw material and demand forecasts while meeting dynamically changing production goals helps improve capital utilization and supports lean and just-in-time manufacturing production goals.
In today’s customer-driven world, manufacturers no longer can rely on selling expensive spare parts and services, since these become costs of supporting a single-price subscription. This shift will require manufacturers to completely re-think how they operate—new organization structures and skilled resources, new incentive models, new KPIs to measure success and new processes replacing ones developed over decades and centuries. They will have to become data-driven organizations, investing in technologies to connect and track products, collect data and efficiently analyze these massive amounts of operational and service data, using technologies such as IoT, machine learning and predictive analytics. This will strain manufacturers’ existing organizations and IT infrastructures, necessitating investment in highly scalable cloud-based solutions to lay the foundation for a successful future.

Manufacturers that embrace these changes will be the winners, while others will struggle to stay relevant. In fact, the ones who can successfully adapt to these paradigm shifts will be able to gain a significant competitive advantage over their competition.

— Vivek Shah
July 18, 2019
XebiaLabs Injects Predictive Analytics Into DevOps Platform

XebiaLabs is moving to take the guesswork out of DevOps following the launch of a set of tools that leverage machine learning algorithms to predict when and how the release of an application is most likely to go awry.

The Risk Prediction Module being added to the XebiaLabs DevOps platform comes on the heels of XebiaLabs’ move to make its XL JetPack software for automating application deployments available on the Amazon Web Services (AWS) Marketplace. XL JetPack is an automation framework based on YAML files and a declarative programming framework that enable DevOps teams to push applications into production on a cloud platform in as little as 15 minutes.

XebiaLabs CEO Derek Langone said that while DevOps advancements have been made in terms of achieving continuous integration, the goal of achieving continuous delivery has proven to be more elusive. As automation frameworks such as XL JetPack become more accessible, however, IT operations teams will be better able to keep pace with the rate at which organizations want to be to deploy and update applications, he said.

It’s becoming increasingly apparent that a combination of automation frameworks and machine learning algorithms will enable DevOps teams to make a significant leap forward in terms of application release lifecycle management. Because of all the nuances of the different platforms that IT organizations are trying to leverage, it’s become a major challenge to roll out an application manually. As the number of applications that need to be released at roughly the same time increases, managing the release cycle process is moving beyond the capabilities of an IT team made up of mere mortals.

The Risk Prediction Model from XebiaLabs promises to increase the odds of success by applying predictive analytics infused with machine learning and other proprietary XebiaLabs algorithms to identify DevOps bottlenecks before they are encountered. Capabilities provided by that module include alerts that warn the team when a release is likely to be delayed or to fail before the release pipeline starts running; a Risk Forecast view that summarizes predicted delays and failures for every task in the release process; an overview of statistics for similar releases to provide historical context; and forensics tools that uncover flaky automated tests, problematic build setups, long-running deployments and time-consuming manual tasks. That Risk Prediction Module is meant to complement the risk scoring capability that XebiaLabs already includes in its platform.

Arguably, the biggest barrier to DevOps adoption today continues to be the level of expertise required to succeed. But as the combination of predictive analytics infused with machine learning algorithms and algorithms that can discern instantly how changes to the IT environment will impact applications become more common, the level of expertise required to master DevOps processes should decline. As that transition occurs, the number of organizations embracing DevOps more broadly should increase.

In the meantime, DevOps teams would be well-advised to consider to what degree they might next want to rely on predictive analytics to prescriptively automate processes with eye toward eliminating the need for as much human intervention as possible.

— Mike Vizard

April 17, 2019
4 Predictive Analytics Challenges Facing SRE Teams
“Predictive analytics” refers to the many techniques used to analyze data to make predictions about the future, such as data mining, statistics, modeling, machine learning and artificial intelligence. Being able to predict technical and business outcomes is one of the main promises of big data. Indeed, many businesses have invested heavily in infrastructure to anticipate key performance indicators such as demand, pricing or maintenance. The investment required is substantial but, when it works, it yields a very positive ROI.

In this post, we’ll examine why implementing predictive analytics in a site reliability engineering (SRE) context requires a significant investment in both setting up the right data sets and applying the right logic to that data.

SRE should in theory be a prime target area for predictive analytics. Why? Because companies have already widely adopted monitoring tools over the past decade. In doing so, they’ve amassed extensive system health and behavior data that is ripe for analysis. SRE teams have also increased their ability to make sense of this data as data science tools have become more accessible. Basic data science aptitude is increasingly become a desired skill when hiring engineers.

Additionally, the SRE team is responsible for a mission-critical KPI: system availability. This single metric is directly tied to customer SLAs in digitally dependent businesses. SRE teams have a strong incentive to implement predictive analytics. Why? Having the ability to prevent downtime by addressing potential issues before they hit the production environment is hugely valuable. More system availability translates into more revenue, higher customer satisfaction and lower costs of problem mitigation.

However, the reality is that an SREs job is still mostly focused on the diagnostic, mitigation and “fixing” parts, as opposed to more proactive tasks. This is in big part because predictive analytics applied to SRE/DevOps has proven difficult to implement and perform in a disciplined manner. Therefore, it is still an aspirational goal at most companies.

SRE Challenges in Predictive Analytics

Up next, let’s explore the four challenges that need to be overcome to implement high-quality predictive analytics practice in an SRE context, including:
- Data collection
- Data quality
- Data volume
- Model usability
Challenge 1: Data Collection

First, the working data sets need to fit what you are trying to predict. In this case, it is challenging to get the right data set because the SRE team’s mission is very broad; specifically, ensuring the reliability of the entire system. The difficulty lies in the fact that there is no such thing as “full system” data readily available. All the relevant data is distributed across siloed, or overlapping repositories and monitoring tools. Centralizing the data requires either a full system agent (which takes a lot of work to set up) or through APIs, which requires mastering a multitude of API integrations. An additional obstacle in getting the right data lies in the inconsistent nature of events, time series data and logs. These disparate data types can’t be analyzed together unless they can be transformed into a uniform data set.

Challenge 2: Data Quality

SRE environments emit lots of data and some won’t make sense for the purpose of the analysis. Whichever data collection method is used, the information tracked often reflects some level of bias on the part of whoever set up the tracking mechanism in the first place. This leads to capturing both irrelevant data or inadvertently leaving out important information. In other words, the data is often both noisy and lacking at the same time.

The key to successful modeling lies in the selection of prediction variables with data that is obtained before the event predicted happens (in this case, a failure). Defining the variables that lead to the determination of a subsystem failure is critical and requires specific domain knowledge. It is difficult to define and extract a pure data set sanitized of noise. The more data you leverage, the higher the chances that you are actually “feeding the beast” with irrelevant information. Too much irrelevant data leads to inaccurate models and false positives.

Challenge 3: Data Volume

Here you can have two different types of challenges: too little data and too much.

Not enough data

You typically need at least 100 instances of what you are trying to predict and probably at least 100 where it didn’t work to train a model. This assumes the data set is clean and directly relevant to a very narrow, specifically defined problem. In the case of SRE, the scope of issues that can affect a system is such that the data set required is very substantial. Likely, it’s several months worth of a wide variety of system behavior measurements. For example, if you collect data only through passive collection, think about the lag time between t-zero (when you start collecting data) and the time when you would have results. During that time, you have a bootstrapping problem—your model can’t be trained to predict accurately what you want.

Too much data

There are two practical points to make around the notion of “too much data.”
- Lack of standardization: Processing large volumes of data for analysis remains a difficult engineering challenge. There is no standardization for data pipeline; every method has a limit in terms of how many inputs it can ingest on a per-second basis, and ingest rates often end up becoming bottlenecks in large enterprise environments.
- Volume does not equal accuracy: There is a point beyond which you get diminishing returns: statistical theorems show that after a certain point, feeding more data into a predictive analytics model will not provide more accurate results.
Overall, getting the quality and quantity of the training data right is the best investment you can make in building your predictive capabilities.

Challenge 4: Model Usability

From a pure “mechanics” standpoint, the analytical concepts and methods are there to reflect the complex thinking required to predict the behavior of an enterprise’s combination of technology, information and infrastructure. Predictive software relies heavily on advanced algorithms and methodologies such as logic regressions, time series analysis and decision trees. The problem is that the workings of the kinds of systems that SREs deal with are very hard to understand in the first place, let alone predict. It is not uncommon even for SREs to not entirely understand how a system works if they were not the ones who initially set it up. In addition, every company’s system is different.

When data scientists design predictive models without years of experience in an SRE capacity in multiple environments—in other words, without leveraging subject matter expertise—it rarely works perfectly. Often the models they create lack usability, even after the validation process to confirm its quality. A modeler will split the available data into a training set where the model is run to generate predictions and compare performance against the actual outcomes. That will require multiple iterations to generate a model that has high overall predictive accuracy, and 9 out of 10 times the result is overfitting, or perfect for the training data. A subject matter expert in the domain of technical operations can cut the cycles of iteration and get to a predictive model that works with an acceptable level of risk.

In short, implementing predictive analytics in an SRE context takes a significant investment both in setting up the right data set and applying the right logic. Stay tuned for a future blog post that will provide some pointers on how to achieve the best prediction results.

About the Author / JP Emelie Marcos

JP Emelie Marcos is the co-founder and CEO of SignifAI. Previously, JP served as the Chief Operating Officer of Tango, a leading mobile consumer application that reached 400 million regular users in five years. JP’s career spans 20+ years leading teams in technology startups and large companies as an investor, GM and CEO.
August 23, 2017
Managing Complexity in Dev/Test Infrastructure Using Machine Learning

Businesses are increasingly developing software solutions that can drive innovation and digital transformation. As the number of developers increases in the organization, the ratio of developers to IT also increases over time. The net result: more dynamic and complex dev/test infrastructure requirements and slower support from IT.

Developers and testers often cannot get their hands on infrastructure environments in a timely manner to accelerate delivery of software applications to their customers. They have to deal with lack of timely support from IT and long waits for hardware resources and for provisioning and configuring of the environments.

How does engineering IT manage this complexity and deliver more agility to their engineering stakeholders?

Provide Self-Service Creation of Dev/Test Sandbox Clouds

Engineering IT teams can increase the velocity of development and testing if they enable self-service ability for dev teams to quickly provision their own lab infrastructure and application stacks at a click of a button.

Self-service, API-driven infrastructure is a fundamental requirement for enabling developers to write code (which can be done using their favorite programming language) and make RESTful API calls to the underlying programmable infrastructure. This allows them to dynamically and automatically manage initial deployments and configurations as well as manage ongoing, automated dynamic provisioning of infrastructure, autoscaling, monitoring and alerting.

All this automation removes the confusion and error-prone manual steps for the entire application delivery process, including development, testing, staging and production deployments. This, in turn, accelerates software delivery and increases quality.

Additionally, if self-service is available as a SaaS-based delivery model, it is easy to add more features and workflows very quickly without having IT doing a major upgrade.

Breaking these silos improves collaboration between teams, accelerates velocity of software development, improves infrastructure utilization, increases overall operational efficiency and reduces costs.

Eliminating Silos in the Dev/Test Infrastructure

Silos are common in companies where various specialist teams (storage, networking, security) form fiefdoms around their respective functional areas. Silos impede velocity because they lead to complexity of operations, lack of consistency in the environment, and lack of automation.

Automating across silos turns into to an exercise of custom scripts and lot of “glue and duct tape,” which makes maintenance and change management complex, slow and error-prone.

Breaking these silos improves collaboration between teams, accelerates velocity of software development, improves infrastructure utilization, increases overall operational efficiency and reduces costs.

A hyperconverged cloud design with a software-centric, scale-out architecture tightly integrates compute, storage, networking and virtualization resources and other technologies from scratch in a commodity hardware box supported by a single vendor. Also, companies can keep costs under control by leveraging scale-out cloud designs that make it easy to start small, grow based on demand, and stay close to the right size and customer demands.

Additionally, development teams should have the ability to quickly deploy/clone/share complex multi-tiered application stacks between various teams and between development and testing, breaking the silos within the development organization.

Automate Operations, Monitoring and Patching

Engineering IT teams need to have complete visibility and control of their entire stack from the infrastructure up to applications. They need intelligent software to monitor the hardware and software stack, manage large-scale clusters and automatically handle routine but time-consuming and complex operations such as failure handling, patching, security updates and software upgrades.

Running a service with traditional sysadmin teams who execute the above activities manually becomes expensive—especially if they operate in server, storage, networking and security silos—as the dev/test environments become more dynamic and the demand for more environments and projects grows.

However, it is possible to use an intelligent private cloud platform that leverages hyperconverged scale-out designs, machine learning software and a SaaS-based operational console to reduce complexity and increase the agility of dev/test teams.

Cloud-based monitoring and advanced analytics dramatically reduce the need for experts in different parts of the infrastructure, scale linearly as the size of the operation increases and cut operational complexity by 90 percent.

Manage Resource Management Using Machine Learning

Applying machine learning to infrastructure management means intelligent software could learn about operational patterns, anticipate capacity needs, raise alerts about security anomalies, self-monitor and self-heal your environment in the face of failures, intelligently apply security patches and automatically upgrade hardware and software systems without any downtime. The next generation of infrastructure management will be driven by advances in machine learning and artificial intelligence, where the infrastructure is able to basically “drive itself” with minimal user intervention.

This will help engineering IT teams optimize resource usage and capacity based on current and future dev/test demand and also be able to better handle the availability and performance they can deliver to their engineering teams. A lot of efficient resource management comes down to capacity planning, utilization monitoring, right-sizing of workloads, demand forecasting and detecting zombie VMs and unused resources.

Demand forecasting and capacity planning can be viewed as ensuring there is sufficient capacity and redundancy to serve projected future demand with the required availability. Capacity planning should consider organic growth, which stems from natural service adoption and usage by dev/test teams. Having intelligent predictive analytics and machine learning can help greatly with accurate forecasting, alerting and providing lead time for acquiring additional capacity.

Better insights into how the infrastructure is performing can also help in fine-tuning performance of end user workloads. For example, an intelligent system that is monitoring a workload for storage performance might recommend using SSDs instead of spindles to increase the IOPS and improve workload responsiveness.

An intelligent private cloud platform that uses hyperconverged scale-out designs, machine learning software, and a SaaS-based operational console can reduce complexity and increase the agility of dev/test teams.

About the Author / Kamesh Pemmaraju

Kamesh Pemmaraju leads product management, product marketing, and technology partnerships at ZeroStack. Kamesh is a seasoned product management/marketing leader with a strong engineering and consulting background. In his most recent product roles at Mirantis and Dell, he focused on delivering private and hybrid cloud solutions to enterprises. He holds a Master of Science in Computer Science from Indian Institute of Science. Connect with him on LinkedIn and Twitter.

June 14, 2017
Embedding Predictive Analytics

For an enterprise with a deep DevOps maturity model, the initial “wow” factor of automation has worn off. At this point, the sexier concepts involve proving continuous improvement. As an example, our first baseline of metrics might have taken the pre-DevOps timings of a typical build and deployment of a given application. We can measure this time against our post-implementation of DevOps automation and show a significant savings in time. This is opportunity time, meaning an employee must be ready to turn around the decision cycle faster to take advantage of the time saved (not be playing solitaire, for example). But once DevOps automation is old hat, the difference between our initial post-DevOps typical build and deployment time and our current one still using DevOps may not be all that significant. But is there more hidden there than meets the eye?

Establishing Context & Baselines

At this point context may be a more meaningful measurement. As an example, if my first version of DevOps automation for a deployment contained 35 steps and my current one contains 160 steps it would stand to reason I am “doing more” now in my deployments than I used to do. These additional steps may involve executing more tests than I originally thought to do. They may be interacting with monitoring systems (turning them on and off as appropriate), which reduces “noise” in the alerts my monitoring teams have to respond to. In short, over time, my deployment processes may become “longer” because I am increasing the capabilities or the sophistication of what I do during these deployments. While from a pure timing perspective this may make them longer, it also makes them more powerful and, so, is worth measuring. To know the delta, however, I must have retained older versions of automation to compare against, not just throw away the old every time I update with new steps.

So let us assume the context of my automation is now fairly stable. I do the same things, more or less, the same way every time I deploy my application. At this point, measuring the functionality of what I do becomes more meaningful. As an example, my deployment automation may complete four main functional tasks to complete, I deploy the code (the app), I deploy the dataset, I deploy the config files and, finally, I run a series of tests on the above. While I may not ever incorporate too many new kinds of things to do in this automation, understanding how long each function takes to complete is relevant to proving continuous improvement. For instance, if my code takes roughly the same amount of time each deployment, as does my config files, but all of the sudden my dataset begins to grow in each deployment (from normal usage), my overall deployments may be getting slower. This is not due to hardware inefficiency or network inefficiency; it is simply due to a larger dataset that keeps growing.

If on the other hand, it is my test set that is getting progressively longer to execute, that is an indicator of test script inefficiency (given same code, same config and same dataset). If the number of tests being executed has not increased, but the length of time it takes to run them has, there likely is inefficiency in “how” the testing is being constructed or executed. Understanding the context for what is going on in a deployment or similar kinds of automation must occur before introducing predictive analytics can occur.

Historical Trends Extended

Predictively evaluating automation is nothing more than taking the data from a historical perspective and extending the trend lines into the immediate future. If for example, my deployment automation is getting four seconds longer each month for the last 12 months, I can reasonably infer the trend will continue, and I can predict the length of time it takes to execute a deployment in the coming months. Understanding “why” this occurs requires a contextual investigation, but predicting its occurrence does not. This may seem inconsequential as we talk about the length of deployment processes of a given app, particularly if my portfolio of apps is small. But if the trend holds true for a portfolio of thousands of applications, the impacts become more consequential. Or if the phenomenon is directly connected to a particular cloud provider and not seen elsewhere, again, the consequences become meaningful.

Where having “ground-up” predictive analytics (build and/or deployment automation) becomes much more useful is in the construction of release events. Often, release managers are asked to supply an estimated time duration for a given event to factor into the construction of a new release. If a vendor were to integrate more predictive analytical tooling into the release orchestration (RO) toolsets, “random” or “best guess” timing estimates could be replaced by extending historical data into the timing estimation fields. The RO tooling could provide a default timing estimation based on examining historical trends for a period defined by the user. Estimation defaults could always be replaced by a user (assuming discreet knowledge of other impacting events), but at least the new overall lengths of releases then could become significantly more accurate.

This has special significance for manual tasks. Let’s say our database admin team traditionally takes 40 minutes to perform a given release event update, but now is running significantly longer than that. Having the RO software aware of the historical trends could alert the release manager, as well as the database team manager, during the event to check out why this is occurring. For manual tasks in particular it may be hard for anyone besides a predictive analytics module to determine what timings “should” be, as compared with what they are in a given release.

Impacts to the Bottom Line

The real questions are, Why should a customer care about this kind of feature set? Why should we be looking at the vendors of our tooling to incorporate forward-thinking architectures such as predictive analytics into the tooling they sell us? Because the efficiency it can bring to our DevOps set of services lowers the costs of administration and increases the speed at which we can deliver innovation in general. Over time, trending combined with context can statistically prove continuous improvement of our DevOps services, which is the promise we made to the executive teams that sponsored and made the investment in DevOps to begin with.

Metrics analysis can occur (usually with a dedicated investment on our part) without a predictive view of the impacts. But this changes the perspective of our analysis to hindsight thinking instead of forward thinking. In essence, we then are simply “reacting” to what has occurred instead of tailoring our environment to alter the picture of future predicted events to be better than the software predicts it to be. We start tweaking the variables to instantly impact future predictions, instead of only examining them in a post mortem.

This perspective is yet another cultural catalyst for change your business needs. When I talk about changing how an organization “thinks” about innovation, this is one of the key objectives. We take an enterprise from knowing little about the costs of innovation, to having an extensive knowledge, to having a forward view of what is—or what should be, possible. All of this is possible despite the loss of waterfall controls and in the midst of an agile construction model. All it takes is the integration of forward-thinking architectures into the products we select. If our vendor does not offer them, we begin to enlighten our vendors on the value they would bring to us in cost reduction and to them in market capture. If you need help in these areas, feel free to contact me, I am looking at the moment 🙂 .

To continue the conversation, feel free to contact me.

June 1, 2016