devopsy.kinsta.cloud

Tag: IT management

  • Rackn Extends Reach of IT-as-Code Framework

    Rackn Extends Reach of IT-as-Code Framework

    Rackn announced the Digital Rebar platform it created to automate the provisioning of bare-metal infrastructure now can be applied simultaneously across multiple physical sites.

    Company CEO Rob Hirschfeld said Digital Rebar 4.3 makes it possible to manage IT infrastructure as code across highly distributed computing environments. Previously, IT organizations would have needed additional versions of Digital Rebar for each physical data center.

    Hirschfeld also noted IT teams can also set up mirrors of data centers, now that Digital Rebar can be employed across multiple sites.

    IT teams that adopt Digital Rebar can define workflows through which, for example, instances of virtual machines or Kubernetes can automatically be deployed on hundreds of thousands of servers in about an hour, said Hirschfeld.
    Accomplishing that same task using manual processes would take most IT teams six months, he noted, adding IT teams can even use Digital Rebar to deploy instances Kubernetes on virtual machines at scale.

    Hirschfeld noted many IT teams assume they need to have a layer of virtualization in place before they can start automating the management of their IT environment. Digital Rebar provides a framework around a common set of application programming interfaces (APIs) through which servers, storage and networking resources can be provisioned consistently. In addition, IT teams can set up a to manage IT infrastructure on an ongoing basis, including firmware and operating system updates and cluster configurations.

    Digital Rebar doesn’t necessarily replace the need for frameworks such as OpenStack or VMware vSphere. However, it does provide a simpler mechanism for installing them. Alternatively, IT teams can employ Digital Rebar to deploy software such as Kubernetes on a bare-metal server.

    Hirschfeld said Digital Rebar is designed to be deployed on-premises or in a public cloud rather than be accessed as a software-as-a-service (SaaS) application. That approach means IT teams operating in highly regulated industries can still take advantage of the framework, he said.

    While the concept of IT as code has been around for a while, IT teams that are being asked to now manage IT at an unprecedented scale are turning to IT automation frameworks in larger numbers. Hirschfeld said Digital Rebar is now designed to replace other frameworks as much as it is to provide a complementary framework for specifically managing bare-metal infrastructure. In fact, Digital Rebar can be access vis Terraform or Ansible using a plug-in that Rackn makes available.

    Rackn also plans to extend the reach of Digital Rebar out to edge computing platforms as well as endpoints based on, for example, Raspberry Pi platforms, he added.

    Most IT teams will eventually wind up employing multiple IT automation frameworks in place of thousands of custom scripts that many IT professionals have developed themselves. The issue with most of those scripts is they don’t tend to scale especially well. In addition, they are usually poorly documented so when the author of the script leaves the organization the remaining IT staff wind up having to write another custom script because no one is quite sure how the script in place works. Whatever the path forward, the one thing that is certain is the days of being able to get by manually provisioning platforms is finally coming to a merciful end.

    — Mike Vizard

  • Edge Computing and ITOps: Opportunities and Challenges

    Edge Computing and ITOps: Opportunities and Challenges

    Edge computing is growing quickly, but IT operations need to evolve as well to effectively monitor and manage new devices, sensors, applications and data. Growing enterprise edge ecosystems should integrate and become part of centrally managed ITOM systems.

    Edge computing is hard to define and is running high on the hype scale. But, research and surveys continue to indicate that this trend of processing data where it’s collected for better latency, cost savings and real-time analysis is an innovation with legs.

    There will be 75 billion IoT devices by 2025, according to Statista. According to Spiceworks’ 2019 State of IT report, 32% of large enterprises with more than 5,000 employees are using edge computing, and an additional 33% plan to adopt it by 2020. Tied to the growth of edge computing is the advent of 5G wireless: 51 operators globally will start 5G services by 2020, according to Deloitte Global research from 2019.

    The major cloud companies are also investing in the edge. The AWS Local Zones service allows single-digit latency connecting to computing resources in a metro environment, while Microsoft offers the Azure Stack Edge appliance and Google Cloud IoT is a “complete set of tools to connect, process, store and analyze data both at the edge and in the cloud.”

    It’s safe to say that edge computing is becoming mainstream, and CIOs and their ITOps leaders should plan appropriately for it in 2020 and beyond.

    Benefits of the Edge for ITOps

    We’ve read plenty about the business benefits from edge computing: oil rig operators need to see critical sensor data immediately to prevent a disaster, marketers want to push instant coupons to shoppers while in the store, video security monitoring can catch a thief in the act and medical device alerts that ensure patient safety. Edge computing may save IT money on cloud and network bandwidth costs as data volumes keep exploding and the need to store every data point becomes harder to justify.

    There are also implications for IT management and operations. Local processing of high volume data could provide faster insights to manage local devices and maintain high-quality business services when seconds make a difference—such as in the event of a critical server performance issue threatening the e-commerce site.

    Today, ITOps teams are inundated with data from thousands of on-premise and cloud infrastructure components and an increasingly distributed device footprint. The truth is, only an estimated 1% of monitoring data is useful, meaning that it provides indications of behavior anomaly or predictions about forthcoming change events. With edge monitoring, we can potentially program edge-based systems to process and send only that small sliver of actionable data to the central IT operations management system (ITOM), rather than transmitting terabytes of irrelevant data daily to the cloud or an on-premise server where it consumes storage and compute power. 

    The job of filtering out the highly-contextual data on the edge, where business occurs, can support real-time decisions for successfully running ITOps at speed and scale—regardless of what combination of on-premise, public cloud or private cloud infrastructure is in place. At the same time, ITOps will need to be a leader when it comes to minimizing the risk of edge technology from a performance, security and privacy perspective. However, as detailed below, we are in the early stages of determining how to make this work in practice.

    ITOps Realities for Edge Computing 

    Edge-Specific Security Needs Are Still Unknown: Edge devices are often small and infrequently designed with security in mind. More than 70% of edge devices don’t mandate authentication for third-party APIs, and more than 60% don’t encrypt data natively. So the attack surface in IoT and edge environments is now larger, and less secure. This is particularly worrisome when considering edge devices that collect personally identifiable information such as email, phone numbers, health data or financial formation such as credit card data. ITOps will need to work closely with security and legal teams to map out the company-specific risk, governance and compliance requirements around managing edge data.

    Edge Monitoring Tools Are Immature: Companies need platforms that can instantly monitor and analyze edge-generated data. In the connectivity of tomorrow, billions of connected devices will be communicating machine-to-machine, and the addition or subtraction of connected devices will be possible at an unprecedented scale. In this environment, the ability to manage large volumes of connected devices and the information being exchanged between them will be critical. 5G acts as the unifying technology, bringing flow of information and the density of scale. We will see an influx of innovation in edge monitoring in the coming years.

    New Environments Call for New Rules: As organizations move more data and application assets to edge computing environments, IT will need to devise new policies and thresholds for central processing and alerting of all this data. Applying AI-based automation is essential here, as manual efforts will have zero chance of keeping up with the volume of data filtering, analysis and response. We are entering the age of nano satellites, vis-à-vis SpaceX and OneWeb. These edge devices will transform the future of agriculture, energy, mining, transportation and finance due to their capabilities for sending insightful data in real-time to customers, wherever they are at any moment. ITOps will have its work cut out to understand and properly manage this evolving edge infrastructure.

    DevOps Processes Will Become Even More Paramount: If you haven’t already realized that DevOps is taking over software development and IT management, just wait for when edge goes mainstream. There will be no other way to manage change and deployments of edge technology without the agile, continuous integration and continuous delivery methodology of DevOps. It will be imperative for ITOps to adopt DevOps practices and tools to manage, monitor and deploy edge resources.

     

     

     

    ITOps is at a crossroads, determining how much of the past is still relative and how much they will need to change to adapt to a distributed, hybrid cloud world that will soon include edge as a fundamental pillar of their digital strategy. Security, machine intelligence and DevOps will be crucial areas of expertise for ITOps teams looking to help drive better business value and customer experiences from the edge.

    — Bhanu Singh

  • LogicMonitor Acquires Unomaly to Advance AIOps

    LogicMonitor Acquires Unomaly to Advance AIOps

    LogicMonitor this week revealed it has acquired Unomaly, a provider of tools infused with machine learning algorithms that make it easier to analyze large volumes of log data.

    Tej Redkar, chief product officer for LogicMonitor, said the acquisition of Unomaly will advance a previously announced AIOps initiative through which LogicMonitor will embed advanced analytics and automation capabilities into its core IT monitoring platform. Rather than requiring DevOps teams to acquire a separate platform to take advantage of AIOps, LogicMonitor contends AIOps is simply an element of the next logical iteration of an IT monitoring platform.

    Unomaly will help LogicMonitor achieve that goal by identifying patterns in streaming log data that are indicative of anomalies that warrant further investigation, said Redkar. Unomaly works by first establishing a baseline for the IT environment. It then records a profile for each log event that includes the event’s structure, sources, frequency patterns and intervals. Similar profiles are then merged into one aggregated profile, with dynamic parameters to reduce the amount of data that needs to be analyzed by several orders of magnitude. If an anomalous event occurs within a rolling 60-second window on a single log source, Unomaly will bring those events together to identify and score a situation based on the type and number of anomalies.
    Redkar noted this approach not only provides more context about the events but it also greatly reduces the amount of log data that needs to be stored. Unomaly is also designed to accept data in any format.

    As IT becomes increasingly more complex, organizations of all sizes need more observability. However, because all the dependencies within the IT environment are becoming too complex to track manually, IT organizations will need to rely more on machine and deep learning algorithms to observe the IT environment.

    Naturally, that increased reliance on algorithms will alter the role of IT operations teams. Most IT operations teams will find themselves supervising algorithms that are surfacing issues and making recommendations, said Redkar. Rather than eliminating the need for IT operations expertise, AIOps will be employed mainly to manage IT environments operating at a significantly higher scale, he noted.

    It will take some time for this transition to play out as issues such as the explainability of AI models are worked out. Some of the lower-level functions within an IT operations environment undoubtedly will be eliminated, so job roles will change. However, many of those functions will be rote tasks that many IT professionals don’t especially like to perform.

    IT vendors are, of course, involved in something akin to an algorithms arms race to infuse AI into their management platforms. In some cases, they will develop these capabilities themselves. In other cases, they will acquire startups to gain access to AIOps platforms. Most IT vendors will be mixing and matching a variety of AI technologies they developed themselves and acquired. Regardless of the path pursued, it’s no longer a question of whether AIOps will be applied to managing IT, but rather when.

    — Mike Vizard

  • Opsani Applies AI to Rein in Cloud Costs

    Opsani Applies AI to Rein in Cloud Costs

    Opsani has added support for cloud computing environments to its namesake software-as-a-service (SaaS) platform that leverages machine learning algorithms and other forms of artificial intelligence (AI) to optimize the IT environment.

    Company CEO Ross Schibler said Opsani AI not only proactively tunes resources such as CPU and memory, but it also can optimize middleware configuration variables such as Java virtual machine (JVM) type and pool sizes; kernel parameters such as page sizes and jumbo packet sizes; and application parameters such as thread pools, cache timeouts and write delays.

    Given the size and dynamic nature of most cloud computing environments, Schibler said it’s not possible for IT administrators to manually optimize cloud computing environments to match the specific needs of each class of application workload being deployed. The Opsani platform makes it possible to leverage AI to optimize settings for the lowest cost or best performance every second, he said.

    Opsani claims existing customers are seeing more than 200% increases in performance per dollar while saving up to 80% on their cloud spend.

    In addition, Opsani has made available plugins for a variety of DevOps tools including GitHub, Terraform, Jenkins, Spinnaker, Wavefront, DataDog, SignalFX, Prometheus, Splunk and New Relic, on all the major cloud computing platforms.

    Schibler said Opsani can be applied to existing monolithic applications as well as emerging microservices-based applications based on containers running on Kubernetes clusters. The goal is to make it possible for IT teams to rein in cloud costs at a time when most cloud resources are being employed inefficiently, he said.

    Individual developers are deploying applications on virtual machines in the cloud with little to no appreciation for total cost, noted Schibler. Before most IT organizations realize it, the cost of running applications on a public cloud has far exceeded budget allocations. IT organizations not only need tools to rein in those costs, but Schibler noted Opsani also makes it possible to model those costs before an application is deployed or upgraded.

    The adoption of public clouds and DevOps often go hand in hand. The challenge organizations are encountering more often is that as the number of applications running in a public cloud computing environment increases, they often wind up overprovisioning resources. Over time, the need to optimize the consumption of cloud resources becomes a more pressing financial concern. However, most IT organizations don’t have the resources or expertise needed to manually tune each instance of cloud workload. At the same time, the typical IT organization doesn’t have the ability to build its own AI engine. Opsani is making the case for a SaaS platform that provides that capability without requiring an IT organization to build its own AI models from scratch.

    Naturally, many IT professionals are dubious of AI because there may not be enough transparency into the AI models being employed. However, because Opsani is a SaaS platform, the risk of experimenting with AI to optimize cloud workloads is relatively slight. Of course, there’s no absolute requirement to employ AI at all. However, as financial pressure on IT organizations continues to mount, there doesn’t seem to be much of an alternative either.

    — Mike Vizard

  • APM for All: Simplifying the Complexity of Cloud

    APM for All: Simplifying the Complexity of Cloud

    As companies adopt hybrid and even multi-cloud strategies, their IT departments are beginning to use dynamic technologies to build new application features on top of existing back-end business systems. Most organizations now have a blended, multi-cloud architecture where public clouds are integrated with the existing on-premises, private or hybrid cloud infrastructure and applications.

    This means, for all the agility, flexibility and scalability the cloud affords us daily, the cloud is also a source of increased complexity for technology professionals required to assure the availability and performance of the application environment. This, in turn, is expected to have a substantial impact on overall management. Gartner predicts that by 2020, “75% of enterprises will experience visible business disruptions due to infrastructure and operations (I&O) skills gaps.”

    It’s vital, then, to first understand the underlying technologies supporting application availability and performance throughout your entire IT environment. Next, you must understand how your technologies are dynamically interacting with each other in both development and production. And you need to know this not just at the various levels of the application stack itself—across all infrastructure as well as services, function calls and code—but for various types of applications themselves: monolithic, service-oriented architectures (SOA) and microservices-based.

    That’s why, whether on-prem, hybrid or cloud-native—and whether you’re running AWS, Azure or Google Cloud (IaaS or PaaS)—you need comprehensive visibility into your complete infrastructure and application stack. Application performance management (APM) gives you this visibility into your IT environment by simplifying, and more importantly, normalizing, app stack management. 

    The Case For APM

    APM has significant business value. In today’s environments, an IT team typically no longer has only one application per virtual machine—but rather dozens of virtual machines per physical host (whether on-premises, private or public cloud), with hundreds or thousands of containerized microservices per virtual machine. Additionally, these business environments with hundreds (or many thousands) of containers are continually being reconfigured and brought in and out of production underneath customer-facing applications. If something goes wrong, the digital operations can collapse. And they all need to play nice together since the underlying resources are shared.

    At the same time, the game has changed. The red, yellow and green glow of infrastructure dashboards now has less bearing on whether you’re really delivering high-quality user experiences. Instead, success metrics are now based on evolving real-world, end-user experiences. You are customer-facing, because clients interact with you through your applications and websites. If your site is slow, you lose money and it negatively impacts the business brand.

    To fix the problem, you need specifics. For example, is the transactional performance slow because you don’t have enough compute? Or is it slow because the application code is bad? These factors are interrelated. In our dynamic, shared-and-virtualized everything environments, it’s challenging to understand where the true causes of your problems are.

    When Amazon went down for an hour during Prime Day last year, experts estimated the company lost 100 million dollars in sales. Imagine the impact of a potential outage during the holiday season. Of course, those are Amazon-sized numbers. But scale down those Godzilla figures to your size, and it still hurts—either through lost revenue or damaged brand reputation.

    With APM, you can see all the metrics and find out how the application’s performing over time. You can see it degrade, giving you time to figure out why it’s slowing—before it stopped getting you your raw materials on time. You can understand down to the line of code, or database query, exactly what, where and why it’s slow.

    But what are we talking about when we talk about APM? Gartner has pointed out the historical confusion on terms. There’s also a matter of differing capabilities among APM tools. Some are geared toward monolithic applications; others for Java and the web (and other applications) and still more are tuned for microservices. Beyond that, there are full-stack APM solutions that can measure it all.  

    Normalizing IT Management With APM

    APM strategies and toolsets aren’t all the same. By my count, there are three iterations of APM implementations. First, there’s APM dating from when all tech was monolithic, of which the best example is the old CA Wily Introscope. Then came a second generation, which covered areas such as SOA, Java and the web—a good example here would be AppDynamics and the old Dynatrace. After that comes a version of APM tuned specifically for microservices and containerized application environments, such as the new Dynatrace and Instana.

    In some cases, the actual APM technologies themselves haven’t kept up to date with the underlying infrastructure changes. That’s especially true for microservices. There are some legacy APM solutions that don’t support microservices in containerized application environments. On the flip side, some new companies in the APM space only operate in the microservices space. They are container-based, and don’t support old monolithic code. They also don’t support n–tier/SOA application environments.

    Most customers, however, have mixed environments. Almost no one is operating with 100% microservices in containers operationally managed by Kubernetes.

    By my reckoning, modern APM systems should monitor it all, because everything is mixed and likely to stay that way, if history is any indication. Companies still rely upon massive databases of proprietary information—databases of business records. They’re just building all-new applications in front of them.

    If you posit that it’s now more important than ever to ensure availability and performance as experienced by the user, then you need APM across all those three categories of applications and their environments, including public cloud providers such as AWS and Azure. You need full-stack APM.

    Getting Started

    Before you do anything else, first remember your due diligence. Ask around and seek out first-hand knowledge about the APM solution vendor. Don’t just trust them to give you objective advice on how good they are; there’s a great deal of hype in the APM market that can often cloud our understanding of a certain solution’s actual function. So, where do you start? Typically, it’s best to start your thinking as if you were the end user coming into your application and infrastructure stack. This means monitoring, measuring and understanding the real performance of your web-based or custom applications. You should start with the customer (external-facing) side and then work your way back in through the stack from a monitoring perspective. Next, monitor, measure and understand the performance metrics of the application itself. Here you’ll find critical capabilities, such as distributed transaction tracing and code profiling, to be critical to truly understand how all the application components relate from a performance perspective. Finally, while it’s essential to also know how your infrastructure components are performing, it’s critical to do so in full application performance context. This application-context infrastructure monitoring must encompass both performance instrumentation as well as all logs associated with your application and related infrastructure.

    Start small for quick success and to build a foundation from which to make APM more broadly successful for your organization. Start with one, or at a most a handful, of applications needing the greatest attention. Prove value and success and then ramp up adoption from there.

    In all cases, remember—unless you’re solely relying on serverless computing, with only microservice-based applications in containerized environments, you’ll need visibility across the three application architecture types: monolithic, SOA and microservices. Otherwise, you’re not getting all the information you need.

    It’s always been the goal of IT throughout all technology revolutions—whether it was mainframe, time-sharing, client-server or distributed, and whether it’s hybrid or full-on public cloud—to get a single pane of glass for operational management. True, full-stack APM delivers that view.

    — David Wagner

  • ServiceNow Makes the DevOps Connection

    ServiceNow Makes the DevOps Connection

    ServiceNow today announced it is extending the reach of its IT operations management platform to integrate with a variety of DevOps platforms.

    Richard Hawes, director of product marketing for ServiceNow, said ServiceNow DevOps will provide IT operations teams relying on an ITIL-based framework accessed as a software-as-a-service (SaaS) application visibility into DevOps processes and events occurring in on-premises environments or public clouds.

    At present, ServiceNow DevOps is available for financial services organizations based in the Northeast region of the United States and the UK. The service gradually will be rolled out to additional ServiceNow customers over the course of 2020. ServiceNow DevOps initially integrates with Jira Software, GitHub, BitBucket and Jenkins, with more integrations expected to be rolled out periodically.

    Hawes said ServiceNow recognizes that IT organizations are building and deploying applications at rates that are faster than existing ITIL-based approaches can track. However, ITIL-based frameworks provide the highly structured set of processes around which the bulk of IT environments are managed. ServiceNow DevOps enables IT teams to automate change management tasks such as planning, development, testing, deployment and operations via a single dashboard surfacing a common set of metrics, he said. He added in many cases, those preset rules and policies will eliminate manual processes that often conspire to slow the rate at which applications can be deployed and updated within any IT environment, said Hawes.

    No developers need to be involved in those processes because IT operations teams can set rules and policies, and developers will see their code running in production environments in a matter of minutes regardless of what tools they employ, said Hawes. That approach also will make it easier to scale DevOps across an enterprise IT organization at a time when adoption of DevOps is still uneven at best, he noted.

    At the same time, that level of integration should also make it easier for IT teams to audit DevOps processes using a common framework to capture events such as when code is checked into a DevOps platform, he said. That capability is a critical requirement for any company trying to apply best DevOps processes with the context of a highly regulated industry such as financial services.

    Hawes said the majority of organizations that have adopted ServiceNow to manage IT have also adopted a DevOps platform. Overall, Forrester Research reports 56% of global infrastructure decision-makers now report that their organizations are implementing, have implemented or are actively expanding their DevOps initiatives. ServiceNow doesn’t expect DevOps processes to supplant ITIL-based frameworks as much as it anticipates both approaches co-existing as they continue to evolve, he said.

    With the release of the latest version of the ITIL framework, it’s apparent that the proponents of ITIL have come to recognize that IT needs to become more agile to meet the needs of digital business. The challenge they face is finding ways to enable IT teams to become more agile without sacrificing the gains in stability achieved over the course of the last two decades. With the launch of ServiceNow DevOps, the effort to find a middle ground between those two extremes can now begin in earnest.

    — Mike Vizard

  • Virtual Instruments Aims to Meld DevOps, ITSM Worlds

    Virtual Instruments Aims to Meld DevOps, ITSM Worlds

    Virtual Instruments today announced additions to an IT management platform that leverage machine learning algorithms to identify application drift and dynamically right-size virtual machines.

    Tim Van Ash, senior vice president of products for Virtual Instruments, said version 6.2 of the company’s VirtualWisdom platform is part of an ongoing campaign to meld the traditional world of IT service management with best DevOps practices using an AIOps platform. As part of that effort, Virtual Instruments last month acquired Metricly, a provider of tools for monitoring cloud costs in real-time.

    The latest update to that platform also adds tools to monitors, reports, forecasts and alarms against the capacity consumption rate of any application, while a capacity forecast alarm now allows IT teams to specify how many days, weeks or months they wish to be notified prior to potential capacity exhaustion occurring on any one node or cluster.

    Finally, Virtual Wisdom 6.2 provides access to updated VirtualWisdom Proactive Dashboards that provide access to Smart Charts, through which the AIOps platform now supports annotation, linking and smart naming in addition to topology views that are specific to a set of IT infrastructure or an application.

    Van Ash said the Virtual Instruments approach to AIOps is that it allows IT teams to respond to changing application conditions in real-time and set granular limits on how much capacity can be consumed by any particular application. For example, IT teams can ensure that specific applications are given a Platinum-level preference that prevents any other application from consuming resources allocated to that application, he said.

    In general, Van Ash said Virtual Instruments is pursuing an application-centric approach to AIOps that will make it easier for IT organizations to optimize IT infrastructure capacity dynamically. Rather being overly focused on utilization rates and tracking events, AIOps platforms need to be able to address the entire IT life cycle as it relates to the value that IT environment has to the business, he said, noting today many DevOps teams are building and deploying applications with little regard or actual control over the service levels those applications need to achieve and maintain. The goal, he said, is to provide a single platform through which DevOps and IT operational teams steeped in legacy IT service management (ITSM) models such as ServiceNow can collaboratively achieve a common set of goals.

    A day of reckoning is coming for proponents of DevOps and IT teams that have built sprawling IT environments employing ITIL-based processes. Those ITIL-based processes are not especially flexible, but they have been employed successfully to roll out applications at scale consistently. Advancements in AIOps should go a long way to bring the two schools of IT management together by making it easier to bend the available IT infrastructure resources to the will of multiple application developers trying to share a common pool of resources.

    In the meantime, no one should expect AI to solve their problems magically overnight. It takes a while for algorithms to solve all their IT headaches. However, the days when the relationship between developers and IT operations teams could be described as mostly contentious soon may be coming to a merciful end.

    — Mike Vizard

  • AI for IT Operations: All It’s Cracked Up To Be?

    AI for IT Operations: All It’s Cracked Up To Be?

    Artificial intelligence for IT operations (AIOps) is becoming a critical tool for organizations attempting to bring automation quickly to network management and IT problem resolution. However, many organizations are still unsure how to deploy AIOps for the most value and to net the full, promised results. Perhaps the challenges come from not looking beyond data silos, or perhaps the problems can be attributed to a lack of understanding AIOps’s full potential in the enterprise.

    Enzo Signore, chief marketing officer at FixStream, lends credence to the problem of siloed information. “Despite continued investments in system resources, IT cannot holistically see what’s in their physical and virtual infrastructure,” he said. “And without a clear view of the entire IT environment and the ability to make sense of the mountain of data being served up by various system tools, Ops teams can’t figure out what’s wrong or how to fix it.”

    In essence, for AIOps to be successful, IT must be able to access that siloed data and then be able to process that data to derive actionable insights, exactly what AIOps is meant to do. Yet, complexity still seems to be one of the major challenges of bringing a holistic approach to managing networks and resolving problems. 

    “IT teams are bogged down by the sheer complexity of today’s hybrid, dynamic IT environments,” Signore noted. “Along with steep learning curves and lightning-fast changes to new technology, IT departments must cope with soaring consumer expectations, supporting legacy systems, lack of skilled talent, disparate tool sets and opaque physical and virtual IT infrastructures. These factors complicate IT staff’s ability to oversee operations and optimize system performance.”

    Naturally, companies such as FixStream have a major stake in the AIOps game; the company offers numerous AI solutions that promise to modernize IT operations and bring AI into the realm of IT operations. However, understanding what AIOps is all about is one of the first hurdles to overcome when contemplating the technology. Gartner defines AIOps as platforms that enhance IT operations through greater insights by combining big data, machine learning and visualization. “IT leaders should initiate AIOps deployment to refine performance analysis today, and as an augment to IT service management and automation within the next two to five years,” advised Pankaj Prasad, senior principal analyst at Gartner.

    Gartner defined some best practices that businesses should adopt, which include: 

    • Deploy AIOps by adopting an incremental approach that starts with historical data and progress to the use of streaming data, aligned with a continuously improving IT operations maturity.
    • Select platforms that enable comprehensive insight into past and present states of IT systems by identifying AIOps platforms that are capable of ingesting and providing access to text and metric data.
    • Deepen the IT operations team’s analytical skills by selecting tools that support the ability to incrementally deploy the four phases of IT operations-oriented machine learning—descriptive, diagnostic, proactive capabilities and root cause analysis, to help avoid high-severity outages.

    Best practices aside, FixStream’s Signore opined that there are some other challenges that must be addressed. “Without the foundational capabilities of accurate discovery, contextualization and full-stack correlation, AIOps can become severely limited,” he said. “The lack of those features have become the big elephant in the room that everyone is not addressing, and it’s holding back the advancement of AIOps and machine learning for many companies.”

    Simply put, when AIOps is not fully deployed, important insights may go unnoticed and leads to data remaining in silos. “Companies are getting all this data, but it’s not being properly processed and correlated, and it’s going to waste,” he said.

    It is those types of deployment issues that are lending credence to AIOps not being as effective as promised. “AIOps is a multi-layered platform whose capabilities should include efficient data collection at big data scale, correlation across the collected data, machine learning, analytics and visualization,” said Bishnu Nayak, FixStream CTO.

    Businesses discouraged by the initial ROI of AIOps should take a long, hard look at their implementation strategy and take into consideration what their data sources are and how they are applying analytics, Nayak said. “Done right, an AIOps implementation can lead to a decreased mean time to remediation and better proactive problem-solving. By increasing system availability and responsiveness, AIOps can enhance your digital business operations and improve profitability.”

    — Frank Ohlhorst

  • VMware Looks to ‘Ruthlessly’ Automate IT

    VMware Looks to ‘Ruthlessly’ Automate IT

    VMware, at VMworld 2018 this week, advanced an ambitious campaign to automate IT management by announcing a deal to acquire CloudHealth Technologies and then previewing a Project Magna initiative that employs machine learning algorithms and other forms of artificial intelligence (AI) to create what VMware describes as self-driven data centers.

    VMware CEO Pat Gelsinger told conference attendees that, as part of achieving that goal in the era of the cloud, every aspect of IT now needs to be automated.

    “The rule of the cloud is to ruthlessly automate everything,” he said.

    Project Magna represents an effort to centralize AI research and development to enable IT organizations to reallocate resources to building applications by eliminating many of the manual tasks that conspire to make IT inflexible.

    At the same time, increased levels of automation will serve to reduce the cost of labor associated with managing IT, which remains the single highest cost within most IT budgets. Without increased levels of automation, those costs inevitably will rise as IT organizations embrace not only public clouds but also emerging new platforms such as Kubernetes.

    Gelsinger this week made it clear VMware views being able to automate the management of Kubernetes as an opportunity to expand the reach and scope of its management offerings.

    The addition of CloudHealth Technologies to the VMware portfolio adds a cloud operations platform that spans Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP). Once the acquisition is formally closed, VMware has pledged to integrate its cloud automation services, Secure State and Wavefront cloud services with the CloudHealth platform. The CloudHealth platform will be rebranded VMware Cloud Operations. Those moves come on the heels of a significant expansion of the portfolio of DevOps tools the company also announced this week.

    VMware also revealed its is working on the Dimension project with Dell EMC and Lenovo, which will create a hyperconverged architecture optimized for deployment at the network edge. Project Dimension will make use of VMware NSX network virtualization software and the software-defined wide area networking (SD-WAN) platform gained by the company’s acquisition of VeloCloud to connect servers running at the network edge. That initiative complements the existing Pulse project for managing internet of things (IoT) devices, for which VMware announced REST application programming interface (API) support and which will be available as an on-premises or a software-as-a-service (SaaS) application.

    VMware this week also unveiled updates to updates to its vRealize automation portfolio that enable organizations to rightsize workflows and automate host-based placement of those workloads. The core vRealize Automation platform also sports a new user interface and integration with open source Ansible Tower automation software and integration with Pivotal Container Service (PKS), an instance of Kubernetes. Version 7.5 of vRealize Automation 7.5 also features enhanced AWS and Azure support.

    It’s unclear to what degree increased automation will transform how IT is managed within the context of modern DevOps processes. But the one thing that is clear is the economics of IT are being altered. Via automation VMware is sharply reducing the need for IT specialists in favor of “super” administrators capable of managing compute, storage, networking and security at unprecedented levels of scale in a much more integrated fashion. How much of that will be accomplished by AI is still anyone’s guess. But it’s already clear the job of the average IT administrator will soon be transformed utterly.

    — Mike Vizard

  • Rethinking the APM Paradigm: A New Vision for Enterprise Application Monitoring

    Rethinking the APM Paradigm: A New Vision for Enterprise Application Monitoring

    Traditional APM systems today can’t measure up to tomorrow’s processes

    IT today is inordinately complex; continually pushing the boundaries and comprehension for even the most sophisticated minds. And while modern IT is unlocking new, innovative experiences and capabilities, the advent of cloud computing and its distributed, ephemeral nature has also made it extremely difficult for IT managers to keep apprised of their infrastructure stacks.

    This reality has taken shape so swiftly and with such a vengeance that Gartner estimates by 2021, enterprises will be able to monitor only 20 percent of their applications accurately. In a digital world, that is like a department head saying he or she knows what happens in only 20 percent of your factories or storefronts. Twenty percent, regardless of the industry or circumstance, is both unacceptable and a grave risk to business continuity and success.

    APM Helps, But Not Much

    Enterprises need to know what the other 80 percent is doing. Unfortunately, the traditional application performance management (APM) tools on which enterprises typically rely are no longer up to this task for three reasons.

    First, the modern application suite looks very different today than it did just a short time ago. The monolithic and custom ERPs, CRMs and other enterprise systems that were the hallmark of big businesses a decade ago are giving way to cloud-based, off-the-shelf, as-a-service solutions constantly updated with iterative DevOps and agile development techniques.

    Second, most traditional APM tools rely on embedded monitoring agents installed on the monitored system, or they need code-level insertions tactics to monitor. Either way, that type of burden is unsustainable at today’s enterprise scale and doesn’t work when applications are launched in containerized cloud environments. It also fails to account for the shift to more off-the-shelf or as-a-service solutions created without APM data outputs.

    Finally, the very infrastructure enterprise operates on looks nothing like it did even five years ago. Most enterprises have moved (or are in the process of moving) to cloud, multi-cloud, or at least hybrid infrastructure. The advent of cloud and the ephemeral nature of microservices and containers means that application topology looks very different at 8 a.m. when only early-bird employees are in the office, then it does at 10 a.m. when the remaining employees arrive. That unpredictability is a nightmare for APM tools, which, at best, now offer only a keyhole view of the entire IT enterprise; performance management now relies as much on the infrastructure as it does the application.

    So, What Needs to Change?

    Traditional APM tools are battleships in an asymmetric warfare age: highly specialized and designed for a single mission, but at the cost of the flexibility to adapt to modern environments as needed. We need carrier groups, not battleship armadas.

    The ideal APM solution for modern enterprises is an “agent-less” or “agent everywhere” approach. A modern solution must holistically consider how one application interacts with other applications. For instance, how does online ordering generate data and feed into warehouse processing tickets? And it must understand how those applications interact with the underlying infrastructure. For example, where are databases or applications stored and how reliable is that infrastructure?

    Current APM solutions miss the mark entirely, scratching only the most superficial layer of today’s technology stack when instead, they need to understand the relations from the deepest layers of infrastructure out to the end users’ experiences. Such an approach will allow enterprises to see into the other 80 percent of applications and deliver the insight and automation needed to drive robust and innovative digital experiences.

    However, knowing you need visibility and having it are two very different things. Fortunately, innovation within AIOps through agent and agentless technologies promises to deliver what conventional APM can no longer.

    By making visibility into application layers easier and deploying machine learning-enabled topological mapping between infrastructure and applications, AIOps is allowing for more detailed insights into the health of mission-critical business services than ever before. It allows enterprises to regain the visibility they’ve lost to an ephemeral world and be able to understand, organize and innovate with the agility they need to compete in today’s business environments.

    It is time to leave traditional APM behind and move forward into an AIOps enabled world. Neglecting to do so puts the business at ever-greater risk.

    — Erik Rudin