Tag: AIOps

  • How Systemic Approaches to ITOps Impact the Business Culture

    How Systemic Approaches to ITOps Impact the Business Culture

    We are at an inflection point in the rise of big data. Enterprise IT infrastructure is producing data at a scale like the world has never seen before. IT operations (ITOps) management has accelerated past the notion of data storage and processing (vis a vis the Hadoop model) and into the world of analytics-as-a-service. We now need systems to drive insights and manage change on-demand. In fact, we need a new operating model for IT in the face of these new effects.

    Can a New System Save Us?

    The key to transforming into a responsive, data-driven enterprise—while avoiding a flood of unnecessary processes and over-staffing—is to adapt automation to intelligently consolidate information flows. At the same time, the dynamic nature of today’s IT environment requires IT teams to become literate in a completely new approach to technology, one that spans all functions of the business. Consequently, risks that previously stayed inside organizational units now leak across domains, influencing decision-making for the entire company. These factors will try to pull the organization in multiple directions at once; only a unified data management system with the intelligence to extract operational insights can drive positive business change.

    Culture Has Got to Change, Too

    Agile processes such as DevOps and AIOps offer mechanisms to control the flow of data and turn it into actionable results. For example, continuous application development and deployment gathers data about functional use, user sentiment, efficiency and various other metrics to inform the overall process and improve the product. Such rapid dev cycles let organizations identify and fix issues more quickly and with fewer resources, ultimately delivering greater business value.

    The resulting streamlined collaboration among IT, development and customer teams improves capabilities but also introduces dependencies that must be managed effectively. It changes the culture to adapt to the system, and thereby, the system of service delivery will fundamentally affect how organizations are structured, built and run. Eventually, there will be increased digitization of every human process, and that means human impact slowly will be replaced by AI impact. This doesn’t mean that machines are coming to take our jobs; instead, it means the modern enterprise IT organization must address a new future. How will it look?

    Retraining and Reskilling Is a Fact of Life: In a world of dynamic IT, requirements are no longer static, and teams need to be prepared. Organizational units must be able to work in different ways with new technology, with new skills that include robotic emotional intelligence, cloud-native literacy and infinite adaptability.

    System operators become stewards of the business because system operations directly impact business outcomes. It won’t be enough for workers to merely deploy, configure and manage IT while isolated from the goals of the company, because those goals are inextricably linked to how objectives are achieved through innovation. Employees will need to develop new skills for functioning in a dynamic system. Training staff in agile methodologies, service-oriented software design and six sigma-style approaches for process improvement will become critical for success.

    Automation Is Everything: Automation by policy won’t be enough; even runbooks will become obsolete. Data and results are always changing, so simple automation will not drive meaningful change. AI can be used to drive automation policies and frameworks, with the potential to reduce errors and integrate disparate systems, which previously required multiple points of oversight.

    The trend toward using AI for data understanding is driving intelligent services into other parts of the business, as well. Rather than just for correlating system and operational data, AI is being adapted to business process optimization. Moving forward, IT operations can begin to run themselves, led by deep data analytics. Alerting and response processes automatically will use their own feedback to update the system’s intelligence, identify emergent trends and take actions to deliver improved operational results.

    Risk Becomes Universal: Increasing connectivity and generalized frameworks (such as DevOps and site reliability engineering) mean that risk is no longer siloed to one department or function. It’s spread everywhere, and cascading effects become commonplace.

    In general, successful engineering outcomes rely on coordinated processes and policies that dictate operational needs, even after deployment. Introducing connectivity between formerly disparate systems can create instability that potentially affects operations in other unintended areas. Thus, broad risks arise if different systems can’t interoperate once unified into a single IT ecosystem. Addressing requirements under a generalized framework helps prevent gaps between application implementations.

    The System Drags the Company Forward: Insights-driven action will transform digital business well in advance of executive priorities. Change will happen before anyone is ready. Indeed, with so much complexity, so many tools and processes and so many competing business demands, IT changes now impact culture in entirely new ways.

    Business impacts will come from system intelligence rather than manual processes. Efficiency in operations management is derived from the entire system, including the people themselves. What was previously the IT culture is rapidly being replaced with a combination of self-adapting processes and workers who focus more on value than on plumbing.

    Transforming the enterprise from silos into integrated platforms will propel the business forward. The resulting system will also drive cultural change at an accelerated rate, perhaps faster than the worker community is prepared to accept, but informed by real data instead of executive intuition.

    — Bhanu Singh

  • Moogsoft Advances AIOps Workflow

    Moogsoft Advances AIOps Workflow

    Moogsoft is moving to make it easier to create custom workflows around its AIOps platform while expanding the number of integrations provided to include connectors to New Relic Insights, Microsoft Teams and proxy support for polling integrations with IT platforms Zenoss, Zabbix, vCenter, vSphere, Solarwinds, Spectrum and SevOne.

    Gabe Coelho-Kostolny, senior product management and product marketing leader at Moogsoft, said a Workflow Engine that has been added to version 7.2 of the Moogsoft AIOps platform makes it possible to trigger actions such as notifications, ticket creation and other automated tasks within Moogsoft and external systems. Those actions can be employed within incident management workflows ad other automated remediation tools, he said.

    Moogsoft has also added tools that make it possible to visualize alert clustering algorithms in real time and, if needed, fine-tune them. That capability is intended to provide IT operations teams with more confidence in the recommendations being generated by the AIOps platform, Coelho-Kostolny said.

    Finally, Moogsoft is making available an analytics dashboard dubbed Individual Statistics, which enables managers to drill down from the team level to better understand the workload and key performance indicators of each team member.

    Coelho-Kostolny noted there is already a high correlation between organizations that have adopted best DevOps practices and those moving to implement AIOps. The complexity of many DevOps environments is in many cases forcing the AIOps issue because all the alerts being generated creates a level of fatigue among human IT administrators that results in them not being able to keep pace with the overall rate of change, he said. The concern now is no longer so much whether human IT administrators will be replaced by artificial intelligence (AI) as much as it is enabling IT teams to stay on top of all the tasks required. The opportunity now, he said, is to make it easier for organizations that have embraced DevOps to also customize the AIOps platform.

    Of course, AIOps platforms presuppose that IT organizations have access to massive amounts of accurate data about the IT environment. Collecting that data often winds up being a bigger challenge than most organizations appreciate. In general, Coelho-Kostolny said it’s clear AIOps platform adoption is reaching a tipping point in the enterprise. Organizations, however, need to be careful distinguishing between what is truly artificial intelligence and what may be simply a rules-based engine that has been added to an existing IT service management (ITSM) platform, Coelho-Kostolny said.

    Regardless of the AI path chosen, the ongoing shortage of IT expertise with critical skills means organizations will continue to experiment with AI. The real issue is determining to what degree of confidence IT teams will have in those AI platforms. After all, if the assumptions an AI system is relying on to make a recommendation prove faulty, the potential for harm tends to be a lot greater than any mistake a single human IT administrator might ever make on their own.

    — Mike Vizard

  • Moogsoft AIOps 7.2 Optimizes Ease of IT Operations with New Transparency, Efficiency & Customization Enhancements

    AIOps Platform Pioneer and Leader Transforms Service Assurance with Workflow Engine, other New Features

     

    SAN FRANCISCO – May 29, 2019 – Moogsoft, a pioneer and leading provider of artificial intelligence for IT operations (AIOps), announced today the release of Moogsoft AIOps 7.2, the latest version of its enterprise platform. Release 7.2 features groundbreaking new capabilities that ease the burden of IT Operations and DevOps teams by optimizing service assurance. Significant new transparency, efficiency, and customization enhancements include: a new workflow engine, AI visualizations, performance dashboards, and new tool integrations.

    “Operations teams seek ways to tame the complexity of their IT environments and make sense of frequent alert storms,” said Nancy Gohring, senior analyst for application and infrastructure performance at 451 Research. “Applying sophisticated analytics, including machine learning, is useful for improving the correlation of alerts and delivering better visibility. Automation enabled by this visibility will have a positive effect on incident response time and increase overall ops team productivity.”

    David Bell, Managing Director of Cybersecurity & Technology Risk Management at American Airlines, says, “With the AIOps system, we have cut the average time it takes to resolve technical snafus by 50 percent. The Moogsoft platform, used at the company’s 24/7 IT Global Situation Center, analyzes 5 million IT error notices and warnings a month, or about 2 every second, as it scans for IT hiccups that could disable a luggage-scanning system, delay a flight, or trip up some other core function.” (source: Wall Street Journal, 14 May 2019)

    New Moogsoft AIOps 7.2 Features

    New Workflow Engine Manages Workloads, Automates Ticketing & Notifications

    Moogsoft AIOps 7.2’s new Workflow Engine provides IT Ops teams the ability to visually create sophisticated custom workflows using a simple but powerful user interface. A rich set of workflow options can trigger actions both within Moogsoft and to external systems for actions such as notifications, ticket creation, and other automated tasks. The Workflow Engine simplifies conditional event processing with enrichment of event alert data, enabling automation of incident management workflows as well as integration with automated remediation tools.

    Situation Visualization Increases Transparency, Understanding of How Algorithms Work

    Situation Visualization provides powerful new visual tools for understanding the operation of Moogsoft’s alert clustering algorithms and, if needed, for fine-tuning them. Similarity clusters are presented as radar charts for each Situation. They provide a window into how the system’s automated decision making works. Users can understand at a glance the matching criteria for those events that have been correlated together into a single Situation. Together with Probable Root Cause, Topology, and other visualizations, Moogsoft’s Situation Room offers real-time situational awareness to IT Ops and DevOps teams. 

    Customization Features Conform to Customers’ Unique Organizational Needs

    Moogsoft AIOps 7.2 introduces a number of new features that personalize and configure the platform for a customer’s unique environment and organizational requirements. These comprise:

    • Situation Room Headers. The information presented in Situation Room headers can be easily customized to improve operational efficiency. Team members can understand the Situation at a glance and decide on next steps.
    • Individual Statistics. A new analytics dashboard called Individual Statistics allows managers to drill down from the team level to better understand the workload and key performance indicators of each individual team member. This insight allows team leaders and all members to optimize work distributions and overall operational effectiveness.
    • New Tool Integrations. Moogsoft AIOps platform continues to expand its broad suite of out-of-the-box integrations for faster time to value. New integrations include connectors to New Relic Insights, Microsoft Teams, and proxy support for all polling integrations (e.g. Zenoss, Zabbix, vCenter, vSphere, Solarwinds, Spectrum, and SevOne).

    “AIOps is gaining momentum streamlining IT Operations as well as DevOps,” explains Phil Tee, Chairman and CEO of Moogsoft. “We’ve built the AIOps market from the beginning, pioneered the way with over 50 patents, and now help over 120 of the largest corporations transform their IT service assurance. Today we’re delivering the next-generation platform to democratize the use of AIOps at all organizations. Our goal is to make Moogsoft the solution of choice for all enterprises – large and small – for agile, proactive event resolution. To this end, release 7.2 empowers enterprises to avoid outages, meet service level agreements, and accelerate digital transformation.”

    For more information about Moogsoft’s AIOps 7.2, visit Moogsoft.com, read our Blog or follow us on Twitter and LinkedIn.

    About Moogsoft

    Moogsoft is a pioneer and leading provider of AIOps solutions that help IT teams work faster and smarter. With patented AI analyzing billions of events daily across the world’s most complex IT environments, the Moogsoft AIOps platform helps the world’s top enterprises avoid outages, automate service assurance, and accelerate digital transformation initiatives. Founded in 2011, Moogsoft has more than 120 customers worldwide including SAP SuccessFactors, American Airlines, Fannie Mae, Yahoo!, and HCL Technologies. It has established strategic partnerships with leading managed service providers and outsourcing organizations including AWS, Cisco, HCL Technologies, TCS and Wipro.

     

    Moogsoft® and the Moogsoft logo are proprietary trademarks of Moogsoft Inc. All other products or names may be trademarks of their respective companies.

     

  • DAOPS Foundation Hosts Global DevOps Forum

    DAOPS Foundation Hosts Global DevOps Forum

    DAOPS Foundation recently hosted its “Global DevOps Development Forum” in Santa Clara, California. The meetup, which was held May 13, was hosted by Jeff Liu, a DevOps expert from Google, and drew DevOps experts from Tencent, JFrog and JD.

    The topics of this meeting were the history of DevOps and the introduction of DevOps capability maturity model. Liu shared the theme of DevOps’ development and current situation: Before the advent of DevOps, the lack of communication between development and operations teams always made it difficult to solve problems when they happened.

    Since 2007, a movement to combine development and IT operations has occurred in Europe, and people are increasingly aware of the importance of combining the two. With the development of the movement, the scope of this concept is also expanding. It is no longer simply the communication between developers and IT operations personnel, but has been extended to the entire development life cycle, from the beginning of the process to the automation through deploying solutions in incremental build systems. The term “continuous delivery” is now used, especially for dynamic applications deployed on the internet. DevOps capability has become the focus of many enterprises’ transformation; we need to continue to move forward in the repeated failures and find out the suitable transformation path for ourselves.

    DAOPS MeetupDuring the meeting, Liu invited experts from JFrog shared their understanding of DevOps and expectations for the future.

    The meeting was also the opening ceremony of DAOPS Foundation. Liu and Kohsuke Kawaguchi attended the meeting as governing board members.  Alan Shimel, editor in chief of devopsy.kinsta.cloud and another board member, was unable to reach the scene for personal reasons, but sent a welcome via video recording. As partners in future course development of DAOPS Foundation, Jayne Groll, CEO of DevOps Institute, also sent her founding wishes.

    About DAOPS Foundation

    As an international organization, DAOPS Foundation hopes to gather experts from home and abroad to work together to improve the IT engineering efficiency (R&D efficiency) of enterprises in different industries around the world and to assist enterprise transformation. This is also called Advanced Software Engineering Model (ASEM), which based on DevOps framework. DAOPS Foundation introduces ASEM.

    Future tasks of DAOPS Foundation:

    1. International DevOps Capability Maturity Model and ASEM output
    2. Open Source Product Promote
    3. GNSEC-Global Next Software Engineering Conference
    4. Lesson based on DevOps Capability Maturity Model and ASEM
    5. Enterprise DevOps Promotion Program
    6. Technical Publication

    DAOPS Foundation conference attendance:

    1. Participate in Cloud Expo-Hong Kong, May
    2. GNSEC Taipei Station, June

    DAOPS welcomes your participation as co-founders or individuals. If you are passionate about DevOps or AIOps and are enthusiastic about the future development of DevOps and AIOps, we sincerely invite you to join our team to contribute to the future development of DevOps and AIOps.

    Any questions, please contact DAOPS Foundation via email or look for us on Twitter.

    — DAOPS Foundation

  • DevOps Chat: We Are Devo, With Walter Scott

    DevOps Chat: We Are Devo, With Walter Scott

    If you have massive amounts of data, Devo wants to help. The Madrid-based company specializes in the handling and processing of massive amounts of data.

    In this DevOps Chat we spoke with Devo CEO Walter Scott about the use cases for Devo in a data-crazy world. Walter is a seasoned CEO in the tech sector. He knows a problem when he sees it and he thinks Devo is the answer. Have a listen to why and check out Devo at devo.com.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation. Enjoy!

    Transcript

    Alan Shimel: Hey everyone, this is Alan Shimel, devopsy.kinsta.cloud. You’re listening to another DevOps Chat. Really happy to have a new company, a new friend, joining us on DevOps Chat today. I have Walter Scott, CEO of Devo. Walter, welcome to DevOps Chat.

    Walter Scott: Thanks, Alan.

    Shimel: So, Walter, you and I met recently in Iceland, of all places, at a Insight IGNITE event that really kinda focused on DevOps, cloud, cybersecurity, if you will, the topics, but it was my first exposure to Devo. I don’t know if our audience is familiar with Devo. Why don’t you–give us the foundation, man. “We are Devo.” What are you guys? What do you do?

    Scott: Well, Devo is a data analytics platform that’s the next gen of technology, that provides a data fabric to allow our customers to take all of the noise that exists in the machines and petabytes, and petabytes of data coming in, for them to really start getting signals so that they can start their automation or their AI or their machine learning process, to make their companies more efficient; to be able to have them act more quickly; as well as to be able to get the people who use our platform, to get them the time back that they spend in managing the underlying data platform and allowing them to add the value to drive IT, to drive that value back to the business.

    Shimel: Excellent. And, you know, Walter, one of the things that impressed me with Devo was this wasn’t your sorta typical Silicon Valley story—a couple of wiz kids come up with a PowerPoint business plan and get funded via VCs, pre-revenue, pre-product even. Devo—the underlying technology here, the company’s been around a while and it’s not a Silicon Valley company, right?

    Scott: That is correct.

    Shimel: Give us a little background on that.

    Scott: Well, you know, over my six stints of being a CEO, I’ve never really run an American-based company. There’s great IT being built all over the world. I’ve done them from Malta to Scotland, Germany to Armenia to now with us here in Madrid, is where the company started. And we had a not Silicon Valley startup, but it was five guys—or four guys and a girl or woman—and started off the product in a research key, to really build something special and change the market because the bank that they worked for had to completely change the way they do business.

    And over the course of three years in research and we finally launched a product in 2014 and have been three-figure type growth every single year, since the date of the company being founded, ’cause it’s such a large market. And it’s really accelerating because of what we’re seeing happening with businesses and trying to get ahead of everything from getting arbitrage out of supply chain to solving their security problems, just to start with basic log management and monitoring to consolidate tools and increase agility.

    Shimel: Sure. So, Walter, you were—as you alluded to, this is your sixth or seventh startup where you’ve come in as CEO and had some tremendous success. And, look, not everyone is a success, right? That’s the mantra in business, but you keep plugging. When we look at Devo—and you and I spoke a little bit about this in Iceland, but so much of what we see as, quote unquote, “AI” today is really machine learning. And so much of machine learning is based, really, on data, right? You’re only as good as your data. In order to find patterns, you have to have enough data to analyze it and make actionable kinda calls out of it.

    But this is not a new problem and there’ve been other solutions. What makes Devo’s solution—I don’t know if you use the term “next-gen” or “new-gen” or something like that, but what makes Devo’s solution different?

    Scott: Well, you know, this has been going around a long, long time and you’ve heard me say, in the absence of data, the machine can’t learn. And I go back, all the way back, to the Walmart days in the ’80s, when they built—I would argue … them as the first person to really take the competitive advantage, coming from their data warehouse, right? And —

    Shimel: I agree.

    Scott: —They actually had the line, they could tell your daughter’s pregnant before you could by looking at the data. Well, today, that’s become necessary, but, over the last 10 years, it’s really—what’s happened with the cloud is being able to get that data manageable. I don’t like the term “AI.” I like more “augmented intelligence” because we’re not at the place—or even narrow AI ’cause I think it can solve very specific business problems that people might have. We have one customer that needed to actually build some ML and machine learning to detect what a bot was, were hitting their site, because, when they launched a new product, the bots would come in and sell it all in the secondary market. They wanted that brand loyalty to go to the end users.

    Or we have another customer that has a bunch of IoT devices integrated into their sporting equipment, where they’re trying to figure out if somebody’s gonna get hurt and using machine learning to be able to define what is that case, in terms of repetitive hits, but they need that data over long scales. And just the data science is actually making that possible. It could be done in the past; the problem was it just wasn’t cost-efficient enough to do it.

    We have customers that are taking literally over a trillion events a day to try to—and building machine learning models to understand which customers may leave their environment, based upon dropped calls or poor support integration. And it’s not just having that one set of data; it’s about having hundreds of different sets of data to help prove those hypotheses, to get those more and more efficient. And the data scientists are spending 70% to 80% of their time getting their arms around the data. We take that problem away from them and allow them to have—to solve that other 80% now, they can put in training their models and getting data to make them more and more efficient.

    Shimel: Excellent. Great answer. Hey, you know, Walter, you and I, this is neither one of our first rodeos. And, when I look at your job, your mission over there, at Devo, one of the things that I guess it’s a double-edged sword ’cause it presents so many opportunities, but, at the same time, there’s an axiom in business—“Put all your wood behind one arrow,” right? There are so many use cases for what you can do here, right? There’s the cybersecurity case, right? Where, “Look, man, we need this data. We need to be logging everything. We need to find if we’re under attack, where the attack’s coming from, what’s happening?,” right? So there’s that. And one can spin up an entire company around just using this kind of technology in cyber.

    You alluded to a few other use cases, right? That had nothing to do with cybersecurity. Better business intelligence. Understanding your customers better. Understanding your product better. There’s only so many targets, though, that you can go after, right? Without a tremendous—you know, you wanna build an IBM kind of force or something or a Dell force, that’s a whole different story. So how do you—Walter, as the CEO, how do you prioritize? What are the best use cases for this technology? What are the right markets for us to pursue?

    Scott: Well, what you say no to is just as important as what you say yes to, in a business.

    Shimel: Yep.

    Scott: And, right now, we are very, very focused in the Americas, as well as in western Europe, right? So we’ve been able to say no to Asia for another year or two. Our value proposition is really about the biggest of the big—the telcos, the banks, the large retailers, people who’re looking at real scale that they’re trying to solve problems with. But the fact is, is where I see the market going over the next three years is more about consolidating the DevOps, the ITOps, the Network Ops, the SecOps. All of those are coming into one. And I’ve heard a couple of people start using the term “fusion center,” which is really helping them drive the business and stop pointing fingers and reducing that mean time to response.

    Well, every security product starts with the logs, right? And every IT monitoring product starts with the logs and the data. And the advantage of us being able to load in all of that relational data, which is about 20% of our data that’s in our systems, across our customer base, is things that is coming out of Xdata or data warehouses or relational or customer application type or weather data or threat hunting data that comes in, and being able to combine all of that. So, for us, it’s the larger customers or companies that have data at scales that we’d see in the global 1,000 and really looking at the people who understand that what Google, and Apple, and Facebook, and Netflix have done is taken a data-first approach to their business.

    And, when we look at customers that are forward seeing—that see forward into the market, they realize there are a whole new set of competitors and being able to ask what their real competitive advantage is in their business and the data that it takes for them to be able to do that, whether that’s a telco to figure out how to launch a 5G network because they have all the 4G, 3G and 2G data, whether it’s in their customer retention rates or net retention of “Can they get more dollars out of their customers each year?” to whether it be their manufacturing process and figuring out that arbitrage in the supply chain.

    So I think we are—because of the ease of use of our product, because it’s so easy to get data in, because we keep data in raw format. So you can have one set of data that’s very easy to get in and use it 100 times, not having to index it or reduce that data because it just makes it more efficient.

    But I see us, in the next couple of years, really focused on the security side of the equation. Looking at people who’re doing the transformation and the weight that it’s gonna take on their DevOps and CloudOps teams and being able to work closely with the network and the DevOps teams to be able to grow their business because the cost of becoming so great on having so many different tools, at different layers of your network, I say that the best and the brightest companies will take control of their own data, consolidate those tools so they have a better competitive advantage.

    Shimel: Good. Good. Good. Walter, I’m gonna throw a term at you and I want your opinion on it, “AIOps.” What do you—this is a term that we, here at devopsy.kinsta.cloud anyway, we see it rising with a bullet. And, like you—to me, there’s AIOps with a capital “A” and a capital “I” and it’s like Robby the Robot stuff that’s gonna make us live like The Jetsons, finally. And then there’s AI with a small “a” and a small “i,” like the real, functional, kinda down in the dirt–maybe it’s not as sexy, but it provides real change for us. AIOps is—man, everyone—what’s your story on AIOps? It reminds me of when cloud came out in 2005. What do you think?

    Scott: So let’s be very clear. AIOps is a term that’s been coined by Gartner. Out of the last 100 conversations I’ve had with CIOs, I’ve only heard it referenced once or twice. I think it’s more the vendors that are getting on it. And, when you look at it, log management is a big part of AIOps. You know, user behavior analytics is part of AIOps. The anomaly detection is part of AIOps, right? There’s seven or eight different components to that. I don’t think any vendor right now, in the world, really has defined themselves as an AIOps vendor. I think we’ve probably got three years, through help of M&A and deciding what that is, ’cause incident response or the service management is also there, so I think it’s gotta evolve.

    But, of the seven or eight different components, the monitoring, the dealing with events, the user behavior, the anomaly detection and the log management is there, which is where we focus. And I think the enterprise log management, I thought, five, six years ago, was solved, when we solved who all those vendors are, but now, with the complexity of the cloud, it isn’t solved at the scale customers need to do with it. We had solved router management; now, with SD-WAN, it’s a whole new market.

    Shimel: Yeah. You’re right. I mean, and it’s funny that things that–it’s kinda like the measles, right? We thought we cured measles and here we are back with measles. Walter, we’re both CEOs of companies and I don’t know how many people listening out there are CEOs—many of our listeners are kinda getting their fingernails dirty, practitioners as well as managers at large enterprise—what keeps you up at night about Devo? What do you worry most about, Walter?

    Scott: Well, I think part is we have to manage tens or hundreds of petabytes of data for our customers, right? And I think we have a very large team internally for security, to make sure that the data that we have is protected and we work very closely with Amazon and Google and Azure to make sure that we address that. And that’s something that you’re never gonna sleep well ’cause you have to be passionate about making sure that you protect your customers.

    We were on the phone literally an hour ago with a manufacturer that makes toys for children, and that was the question that’s keeping him up–he says, “I have 10-year-old boys and girls logging into my website. I have to protect that data, making sure in the things that we do with being able to obfuscate data at the row, in the column level, without having to duplicate it and our algorithms that are in there.”

    But my job, as a CEO, is to help people build careers. I’ve been very fortunate in my career and have had some very large exits in part of the business, but it’s making sure that the team around us continues to make sure that we handle all of the things, in dealing with our business and our main offices in Madrid. We also have a big office in Cambridge, Massachusetts, because I love the guys and women coming out of MIT, where we do a lot of hiring. But it’s mostly about the team and then making sure that we continue to have what I call a industry-abnormal or an anomaly in terms of net retention. And our goal is to help our customers continue to grow because it benefits us and wanting to make sure that we make it easier for them to be able to get more value because our mission is about making IT so that they can provide data to the business that is actionable.

    And my belief is is the CISO or the CIO that has control of that data, if they push it out to the business and say, “Hey, did you know that, on this particular case or when this happens, we’re losing customers or we’re gaining customers?” That data is 80% more likely to be leveraged than it is if the customer asks the business. So—

    Shimel: Nice.

    Scott:–I think it’s empowering the IT to accelerate value to the business and most of my nights are, well, we see this new customer is bringing in this type of a data, bringing in our data science teams to help think about use cases that we can drive more value from.

    Shimel: Excellent. So, Walter, as I mentioned to you before we started recording today, the good news and bad news about this audio format is it goes so darn quickly. And we blew past our time. So what we’ll do, though, is we’ll—secret’s out: you live down in Florida, too. We’re gonna need to get you down here and into our studio and let’s do a whole video on this. We’ll continue our conversation in a video format. But, for now, we’re gonna call a—have to pull the plug on this one. Walter Scott, CEO of Devo, thanks for being our guest today on DevOps Chat.

    Scott: Great, Alan. Thanks.

    Shimel: All right. Hey, this is Alan Shimel, devopsy.kinsta.cloud, and you’ve just listened to another DevOps Chat.

    — Alan Shimel

  • New Relic Doubles Down on DevOps

    New Relic Doubles Down on DevOps

    New Relic today unfurled an extension of its monitoring platform specifically designed for DevOps teams tasked with managing complex IT environments.

    Nadya Duke Boone, vice president of product management for New Relic, said New Relic One, available to existing customers at no additional cost, makes it possible for DevOps teams to now visualize all the data that exists across multiple New Relic accounts their organization has created in a single set of dashboards.

    New Relic One also makes it possible to create an “entity” to track their relationships and dependencies to provide more context to the metrics surfaced by New Relic. That context is critical anytime a major incident occurs because these days so much of the revenue any organization generates is determined by the availability of their applications, said Duke Boone.

    To achieve that goal, New Relic One provides Service Maps spanning entities; search and discovery tools based on universal tag filtering; dashboards through which queries can be launched using visual tools; an ability to more easily extend visualizations; and a new home page designed to make it easy to drill down into various dashboards.

    Other capabilities included in New Relic One include Monitoring for the AWS Lambda serverless computing framework, a Kubernetes cluster explorer and distributed tracing global search.

    New Relic also revealed today it plans to deliver both a log monitoring solution and AIOps offering within New Relic One by the end of the fiscal year. New Relic earlier this year acquired SignifAI to gain access to AIOps platform.

    Duke Boone said all this is now possible because of the investments New Relic has made in developing a persistent store for monitoring platform, which she said represents a strategic initiative on the part of New Relic to expand the scope of the services and capabilities it provides.

    Downtime in the age of digital business directly equates to lost revenues. Organizations that face that challenge typically are at the leading edge of adopting best DevOps practices. That challenge these organizations face today is they are drowning in complexity, a situation that likely will only worsen as organizations deploy cloud-native applications based on containers, she said, adding New Relic One not only serves to make it easier to manage all that complexity, it also makes it simpler for teams within an organization to share best DevOps practices.

    Of course, one of the issues that restrains organizations from adopting DevOps practices in the first place is a reluctance to make IT environments any more complex than they already are. Organizations are demanding that IT organizations become agile enough to support a wide range of digital processes. The challenge they all typically face is the current reliance on highly structured processes that are intended to minimize potential errors. As well-intentioned as that might be, however, they often result in IT organizations being unable to adjust rapidly changing business requirements. The good news is as DevOps tools become more sophisticated, the skepticism of many IT professionals have toward DevOps will continue to quickly diminish.

    — Mike Vizard

  • AIOps: Supporting Reliability at DevOps Speeds

    AIOps: Supporting Reliability at DevOps Speeds

    AIOps looms large as a way to help push the DevOps envelope. How can organizations prepare?

    As organizations journey down the path of DevOps maturation, sustainable IT operations and IT service management remains a challenge for many. Even advanced organizations that have managed to speed up deployment rates and improve software quality struggle to maintain the resilience of the infrastructure that supports those applications. To support DevOps speeds, a growing number of organizations are turning to AIOps—the use of artificial intelligence (AI) and machine learning in IT ops—to speed up analysis of IT problems and better automate incident handling.

    A new study out this week from OpsRamp shows that ops pros are able to reduce mean-time-to-resolution of incidents by as much as 50% through the use of AIOps. The use of AIOps is still infrequent, but analysts say its prevalence will grow quickly. Gartner says about 5% of organizations were using AIOps tools last year, but that by 2023 the deployment rate will shoot up to 23%.

    But as DevOps teams lean on AIOps tools, management will need to harness a new set of skills to get the most out of the types of operational automation AIOps will afford. In the coming years, experts believe AIOps will change the complexion of the IT ops workforce. Fewer people will be needed in the network operations center (NOC) to manually sift through alerts and respond immediately to incidents. More people will be needed to curate data, scrub it and train the algorithms that will do the heavy lifting of event correlation and determining the root cause of problems.

    As organizations and individuals plan out how they’re going to future-proof themselves for the changes in IT ops wrought by AIOps, experts say they should increasingly see the need for the following three roles in IT organizations.

    AIOps Architects

    Expect to see more job postings in the near future looking for AIOps architects and automation path designers, said Will Cappelli, CTO of Moogsoft and the former Gartner analyst who’s known for coining the term AIOps.

    “(AIOps architect is) essentially a software development job involving the construction of meta-algorithms that guide the flow of control of the AIOps systems,” Cappelli said.

    Potentially falling in this umbrella could be roles such as machine learning and AI designers that focus on designing the algorithms themselves, and interaction designers that focus on designing the automated workflows. According to Brendan Caulfield, co-founder of ServerCentral Turing Group, these roles already exist in other business capacities. With AIOps, these design jobs will start to be applied specifically to IT operations as well.

    To prepare, veteran IT ops experts should start thinking about how they can bolster their skills, and management need to recruit and train accordingly.

    “Mathematics and an end-to-end understanding of how modern IT systems behave will be critical,” Cappelli said.

    ITOps Data Scientists

    AI depends upon good, clean data and sound modeling to work effectively.

    “The most sophisticated and expensive AI tools are useless if a company isn’t able to feed them with a high volume of high-quality data, in a format the system can digest,” said Rajesh Kalidindi, CEO of Leva Data.

    This was corroborated by the OpsRamp report released this week that showed maintaining data accuracy is the No. 1 challenge cited by organizations deploying AIOps today. Many organizations will need ITOps specific data scientists to attend to this problem.

    “Building high-volume, high-quality data sets will be increasingly important, and it requires a lot of human effort,” Kalidindi said. “AI has a long way to go before catching up with human reasoning (let alone surpassing it), and it can’t make any progress without people developing data models for AI to work with.”

    Augmented Intelligence Analysts

    According to Caulfield, algorithm training and support by domain experts in operations will become increasingly important at most organizations.

    “No matter how ‘smart’ an algorithm is, or how well its designed, there will be exceptions that are unique to a business that will need oversight and tuning,” he said.

    Kalidindi concurred that human intervention will be needed for a long time to augment and validate the insights and actions driven by AIOps, noting that this will lead to the rise of a new role for ops analysts.

    “Especially in the early days of machine learning, humans are needed to judge whether an algorithm is leading to the right judgment,” he said. “For example, if we were to apply AI to analyzing the problems the IT Help Desk encounters, a human would be needed to judge whether the algorithm made an appropriate recommendation. If not, IT and data science needs to work with the human’s recommendation to better train the AI model on how to handle that problem in the future.”

    One thing that most experts agree about is that the rise of AIOps will not diminish the need for IT pros. In fact, they say that domain expertise will be more important than ever. However, it will need to be bolstered by additional knowledge of data science, how AI works, and systems architecture.

    “They need to retool themselves from admin-oriented activities to scripting/reporting-oriented tasks, and they need to be more application aware than ever before as infrastructure becomes a commodity,” said Ravi Raghavenderrao, vice president of service delivery at NetEnrich, of IT ops professionals in an AI-enabled future.

    To those who have adjusted to DevOps transformations, this should all sound very familiar.

    — Ericka Chickowski

  • VMware Doubles Down on IT Automation

    VMware Doubles Down on IT Automation

    VMware this week moved to inject application and operating system monitoring capabilities into the VMware vRealize cloud management platform to make it easier to employ dashboards to map the relationships between applications and the underlying IT infrastructure they run on.

    In addition, VMware enhanced its capacity optimization tools to enable IT teams to run “what if” scenarios based on the amount of IT infrastructure available both on-premises and in the cloud. The company also embedded integrated compliance and automated drift remediation capabilities into VMware vSphere.

    Finally, VMware added support for VMware NSX Data Center network virtualization software to the VMware vRealize Automation platform, and instances of Kubernetes clusters can now be seen by VMware vRealize Network Insight tools.

    Taruna Gandhi, director of product marketing for the VMware Cloud Management business unit, said these efforts are part of a continuing initiative to drive intent-based computing that leverages automation across a software-defined data center (SDDC).

    VMware views application and operating system monitoring capabilities as being critical tools for making informed decisions about those intents, said Gandhi. However, these tools are not intended to replace the need for classes of management technologies such as application performance management (APM) platforms, she said. Instead, VMware envisions using dashboards to more efficiently troubleshoot issues based on metrics gathered by Telegraf agents embedded in VMware vRealize. In effect, IT teams will be able to identify anomalies faster versus wasting time in “war rooms” trying to prove the innocence of any element of an IT stack.

    VMware is pursuing a dual strategy where DevOps and automation are concerned. The VMware vRealize framework can be invoked either through application programming interfaces (APIs) or a graphical user interface (GUI), which tends to appeal more to traditional VMware administrators who don’t have extensive programming skills but still need to respond to rapidly changing IT operation environments.

    Thanks to automation, operational costs are falling considerably, according to VMware. A report from Forrester Consulting commissioned by VMware found organizations that have adopted vRealize have seen a 30% productivity improvement and a 93% reduction in unplanned work. That same report also noted that, thanks to capacity optimization, VMware customers have also seen a 20% reduction in hardware cost savings, a 10% reduction in storage costs, and a 50% reduction in database software licensing costs.

    In most cases, VMware is employing machine learning algorithms to automate IT operations whenever possible within its cloud management suite. But Gandhi said VMware is hesitant to embrace the term “AIOps” until it becomes clearer how employing machine learning algorithms represents a new class of IT management tools rather than an ongoing set of enhancements to existing management frameworks.

    Whatever the outcome, it’s apparent that algorithms are here to stay. How much they will be employed to advance adoption of best DevOps practices across the enterprise, however, remains to be seen.

    — Mike Vizard

  • AppDynamics Applies AIOps to Automate IT

    AppDynamics Applies AIOps to Automate IT

    AppDynamics, a unit of Cisco, today announced it is melding the worlds of AIOps and application performance management (APM) with the launch of Cognition Engine.

    At the same time, AppDynamics is making good on a promise to integrate metrics drawn from Cisco Application Centric Infrastructure (ACI), the software-defined networking (SDN) that Cisco is embedding in all its networking equipment.

    Finally, AppDynamics also announced Serverless Agent for AWS Lambda, which marks the first time the company’s APM platform has been extended to include the ability to capture metrics from every single invocation of a serverless computing framework.

    Matt Chotin, senior director of technology strategy at AppDynamics, said Cognition Engine represents an ambitious effort based on machine learning algorithms and predictive analytics to create a central nervous system for IT spanning applications, infrastructure and the network. The days when applications and infrastructure were managed in isolation are now coming to an end, he said.

    Available at no extra charge to AppDynamics customers, Cognition Engine infuses the Business Transaction data model underpinning the AppDynamics APM services with machine learning algorithms to run diagnostics and automate root-cause analysis in a way that reduces resolution times from minutes to seconds. The machine learning algorithms evolve constantly as data is gathered continuously using streaming analytics to ingest, processes and analyze millions of records per second. The Cognition Engine automatically understands how metrics correlate and instantly knows when there’s been a break in correlation among datasets.

    Cognition Engine also automatically isolates metrics that deviate from normal behavior to identify the top suspects of root cause for any application issue. As an open platform, that information then can be shared with orchestration and automation systems embedded within third-party incident response, intelligent workload placement and cloud cost optimization tools and platforms.

    These types of capabilities are now becoming an absolute requirement because IT in general is becoming too large and dynamic to manage relying on manual processes, said Chotin, adding what’s needed is a shift toward an AIOps mindset that automates the management of routine IT tasks at scale.

    AIOps is, of course, not so much a thing that can be purchased as much as it is a collection of technologies spanning machine learning algorithms, predictive analytics and other technologies that are employed in unison to automate IT operations. It usually takes a while for algorithms to learn a specific IT environment. But once they do, they bring with them the benefits of a more sophisticated approach to managing IT at scale (and they never take a sick day or decide to take another job). Of course, AIOps is not likely to replace the need for IT staff entirely. But roles and functions within IT organizations most certainly will evolve in the months and years ahead.

    In the meantime, organizations will increasingly vet AI models before deploying them in production environments. While AIOps will advance the state of IT as it is known today, the most worrisome thing about any form of current AI technology is being catastrophically wrong at a level of scale that is nearly impossible to recover from.

    — Mike Vizard

  • AIOps: How to Solve Performance Issues in Complex Data Pipelines

    AIOps: How to Solve Performance Issues in Complex Data Pipelines

    AIOps takes center stage to topple barriers as modern data application growth accelerates

    There seems to be a chasm between DevOps and big data teams in the enterprise, and the bridge over that chasm is long and treacherous.

    In fact, data analysts who are charged with managing and making sense of the growing volumes of data in most organizations traditionally have been excluded from the strategy-making and operations functions of DevOps personnel. Yet, data analysts are critical to the success of software development and at every stage of the software delivery pipeline.

    DevOps Under Siege as Modern Data Applications Proliferate

    Let’s take a deeper look. We know that DevOps personnel are on the front lines in ensuring that all systems, both new and existing, perform as expected in a production environment. But, even today, DevOps teams struggle in dealing with technical challenges when faced with complex data pipelines and the underlying big data stack that supports them, including:

    • Root cause analysis — determining if an application issue is due to application code, config errors or resource contention.
    • Correcting inefficiency in code.
    • Improperly configured auto-scale clusters.
    • Lack of actionable insight into how clusters were tuned at both a cluster and job level.
    • Difficult access to support engineers and promotion of DevOps collaboration.

    And these are just the tip of the iceberg.

    The pressure is on. On average, companies with data-driven environments garner up to 50 percent higher market value than their competitors, according to IDC, and big data is a contributing factor. Thus, big data implementations have become a top priority for businesses of all sizes, and on multiple fronts: revenue acceleration, creation of new products and contributing to better, quicker decision-making. But the unfortunate truth is that, as big data projects move from pilot to production, they invariably encounter serious performance problems that frustrate both data analysts and IT operations.

    What’s more, DevOps and big data personnel face a mutual problem: lack of centralized, top-down visibility into their end to end data pipelines and applications. And, as a result, they face challenges in pinpointing the source of problems—namely, that problems can become visible anywhere in the stack, but their root cause could be elsewhere in in the stack. And, these problems result from root causes as diverse as bad code, inefficient data partitioning, mismatched system configuration settings, best-guess resource allocation and infrastructure issues. Furthermore, a big data stack running multiple applications faces resource contention and prioritization issues that can reduce both individual application performance and overall system performance.  And streaming applications create their own challenges;: for example, if a streaming application starts to lag in processing data in real-time, diagnosing the root cause takes considerable time and effort.

    The industry is faced with a reality where the problems are outrunning the solutions. In the case at hand, organizations have relied on brute force and expensive custom solutions for making analytics-based recommendations more user-friendly to improve individual application and full-stack performance.

    AIOps Takes Aim

    AIOps, or artificial intelligence for IT operations, is gaining steam in the enterprise, with many new use cases appearing as data volumes grow and systems become more complicated. Gartner expects AI to pervade every aspect of managing IT, including application performance management (APM), IT infrastructure monitoring, cloud platform monitoring, performance management, capacity management, the delivery tool chain (CI/CD) and IT service management.

    In fact, AIOps is on a tear: Just three years ago, it was only a concept. Today, Gartner predicts that by 2019, 25 percent of global enterprises will have implemented an AIOps platform to support two or more major IT operations.

    But even as it continues to evolve, AIOps today can deliver automated recommendations to solve expensive, challenging and time-consuming problems with big data deployments and across entire complex stacks. Without AIOps, there would be simply too many variables to correlate manually.

    AIOps makes APM for modern data application possible. It does so by helping to eliminate painstaking, trial-and-error configuration, troubleshooting and remediation workflows that have long plagued big data operations teams. And, for the first time ever, dataOps and DevOps teams can troubleshoot, support SLAs, remediate issues and manage large-scale deployments in an agile and dependable manner.

    It’s a new environment for the modern data center. As part of an APM solution, AIOps is hard at work helping automate and guarantee the performance of big data applications and addressing the primary challenges that DevOps faces with the unrestrained growth of big data.

    It could still be a bumpy ride—organizations of all kinds face unprecedented increases in data volumes. But now, working together, DevOps and AIOps are better-equipped to contend with the challenges.

    — Bala Venkatrao