devopsy.kinsta.cloud

Tag: devops

New Relic Doubles Down on DevOps

New Relic today unfurled an extension of its monitoring platform specifically designed for DevOps teams tasked with managing complex IT environments.

Nadya Duke Boone, vice president of product management for New Relic, said New Relic One, available to existing customers at no additional cost, makes it possible for DevOps teams to now visualize all the data that exists across multiple New Relic accounts their organization has created in a single set of dashboards.

New Relic One also makes it possible to create an “entity” to track their relationships and dependencies to provide more context to the metrics surfaced by New Relic. That context is critical anytime a major incident occurs because these days so much of the revenue any organization generates is determined by the availability of their applications, said Duke Boone.

To achieve that goal, New Relic One provides Service Maps spanning entities; search and discovery tools based on universal tag filtering; dashboards through which queries can be launched using visual tools; an ability to more easily extend visualizations; and a new home page designed to make it easy to drill down into various dashboards.

Other capabilities included in New Relic One include Monitoring for the AWS Lambda serverless computing framework, a Kubernetes cluster explorer and distributed tracing global search.

New Relic also revealed today it plans to deliver both a log monitoring solution and AIOps offering within New Relic One by the end of the fiscal year. New Relic earlier this year acquired SignifAI to gain access to AIOps platform.

Duke Boone said all this is now possible because of the investments New Relic has made in developing a persistent store for monitoring platform, which she said represents a strategic initiative on the part of New Relic to expand the scope of the services and capabilities it provides.

Downtime in the age of digital business directly equates to lost revenues. Organizations that face that challenge typically are at the leading edge of adopting best DevOps practices. That challenge these organizations face today is they are drowning in complexity, a situation that likely will only worsen as organizations deploy cloud-native applications based on containers, she said, adding New Relic One not only serves to make it easier to manage all that complexity, it also makes it simpler for teams within an organization to share best DevOps practices.

Of course, one of the issues that restrains organizations from adopting DevOps practices in the first place is a reluctance to make IT environments any more complex than they already are. Organizations are demanding that IT organizations become agile enough to support a wide range of digital processes. The challenge they all typically face is the current reliance on highly structured processes that are intended to minimize potential errors. As well-intentioned as that might be, however, they often result in IT organizations being unable to adjust rapidly changing business requirements. The good news is as DevOps tools become more sophisticated, the skepticism of many IT professionals have toward DevOps will continue to quickly diminish.

— Mike Vizard

May 14, 2019
Microsoft Gains DevOps Momentum

Microsoft, at its recent Microsoft Build 2019 conference, launched a series of efforts intended to make it easier for organizations to embrace best DevOps practices, including making it possible to address both continuous integration and continuous delivery within the same YAML file using Azure Pipelines.

Microsoft also announced it will extend Azure Pipelines to provide tighter integration between GitHub, which Microsoft acquired last year, and Kubernetes. Via that integration, Azure Pipelines can be used to automate everything from creating container images and packaging them as Helm charts to generating Kubernetes deployment artifacts and then deploying the workload.

Continuing that Kubernetes theme, Microsoft also is launching the Kubernetes-based Event Driven Autoscaling (KEDA) project, which makes it possible to run Azure functions on top of Kubernetes clusters. Developed in collaboration with Red Hat, KEDA is available in developer preview for the Red Hat OpenShift platform based on Kubernetes.

In addition, Microsoft has extended the Azure IoT Edge runtime to Kubernetes to enable internet of things (IoT) applications to be deployed on Kubernetes and Azure SQL DB Edge, which is packaged as a container and makes it possible to deploy a database on ARM or x86 systems deployed at the network edge.

Also during the conference, Microsoft launched MLOps, which applies DevOps principles to deploying artificial intelligence (AI) models using the GitOps workflow; Azure SQL Database Serverless, which allows organizations to consume a database service on a per-second billing basis on the Azure cloud; and ML.net, a framework for building AI models.

Microsoft also announced it is now making available a full Linux subsystem within Windows 10 and the ability to link IoT devices to the Azure cloud without writing code.

Scott Guthrie, executive vice president for AI and cloud at Microsoft, told conference attendees Microsoft is squarely focused on making it easier to build applications that can be deployed in hybrid cloud computing environments anchored around the Microsoft Azure cloud.

Microsoft historically has focused on making it much easier to consume emerging technologies at the point in time they start to appeal to the broadest number of customers. As DevOps best practices continue to find mainstream adoption, that same philosophy is being applied to cloud services. Microsoft may not have invented the concept of CI/CD, but it is working to make it easier for organizations that have struggled to implement continuous delivery, for example, to master that challenge. The same concept applies to Kubernetes, which some argue is still too complex for the average IT organization to master and deploy.

It remains to be seen how much Microsoft’s efforts, most of which are only available as previews, will result in more applications being deployed on Azure. Because an application is built on a public cloud doesn’t always mean it will be deployed on that cloud. It does, however, increase the chances of that occurring. In fact, providing a better DevOps experience across a hybrid cloud computing environment is clearly key to Microsoft’s strategy as it seeks to gain ground on Amazon Web Services (AWS).

— Mike Vizard

May 13, 2019
The Terrain of DevOps is Strewn with Pitfalls

As Gene Kim, one of the leading thinkers in DevOps, once said, “DevOps is not about what you do, but what your outcomes are.” What he’s telling us is that DevOps is about enabling a rapid flow of work that transitions from development to testing to operations, but the bottom line is that it’s about achieving results. Unfortunately, this too often gets lost in modern-day DevOps practices.

In this article, I will discuss five common pitfalls that occur in DevOps if you don’t pay attention to the endgame and how your organization can avoid falling victim to them.

DevOps and Automation: Measuring the Wrong Outcomes

Some people believe that, at its core, DevOps is about trying to automate everything. But ask yourself this: Once you automate, are you really delivering a better outcome?

The answer is not always yes. While metrics such as deployment frequency or mean time to repair (MTTR) are interesting DevOps indicators, they are no more interesting to determining if your team is delivering actual value than measuring the number of lines of code submitted or counting the number of keystrokes by the development team. You measure outcomes by determining a baseline from where you start and compare that with where you end.

We need to understand that DevOps is about much more than automation; it’s about the entire pipeline and how your team moves from ideation to delivery to production. Even large companies with world-class DevOps teams don’t always clearly see the problem and can struggle managing the value stream and putting together multiple pipelines. An anonymous but real-life example of this is an organization I know that had been doing DevOps for 18 months, yet hadn’t delivered anything into production in that span. Are they really doing DevOps? DevOps is a process, but if it produces zero value, it’s not delivering on its promise.

The key is focusing on delivering value and measuring value delivery into production. However, before you can begin measuring value, you have to define what it means to each individual deliverable. Common metrics include the number of story points, actual revenue from product comparison and customer satisfaction. With your clearly defined metrics in place, you can measure value as it gets delivered to the customer.

DevOps and Fragile Environments

One Achilles’ heel in the DevOps process is the pre-production environments where code gets deployed and tested. Automating a delivery pipeline relies completely on code being built and deployed to various environments. Fragile environments mean that the developers and testers are wasting time dealing with problems originating from the environment and not the code itself.

While we wish all environments could be on-demand; the fact is, most are not. Another DevOps expert, Randy Bias, has an analogy about pets versus cattle in describing servers, and the same analogy can be used for environments. If environments are not standardized and repeatable, you have to be willing to move on from it like a farmer moves on from livestock; you can’t hold onto sentimental attachments like we have with our pets. What’s most important is that in no case can an environment be singular in nature.

One element of the solution is to create code and automate, but the key is building up environments that are complete. Every environment needs proper builds, proper data and proper tests validating the environment. Additionally, the automation must have the stability to perform successfully every time. If you have piece of automation that fails even just 5% of the time, that’s unacceptable.

The only way automation works in an environment is if you can trust the environment. When the environment is unstable, it’s like playing Russian roulette: No one knows exactly where the danger lies. It could be in the environment, the code, the data, the middleware … a number of places. When a problem arises, developers are often the last line of defense, but dealing with an issue such as this pulls them away from getting real work done.

Disconnected Teams

On this point, I’d like to share a warning: Disconnected toolchains lead to disconnected teams. The planning, development and testing teams can lack cohesion, and the release and operations teams often struggle to be on the same page. Each of these groups have their own tools and management, but they need to be able to communicate with each other effectively to make things work smoothly. In an ideal world, planning initiatives would flow into features for the development and testing team, which would flow into release pipelines and operational changes. Every team still can speak its own language, but the tools need to be interconnected and unified across the teams.

Mismanaging Hybrid

The term “hybrid” has many different uses in the software development space. Hybrid methodology refers to combining waterfall, agile and DevOps practices. Hybrid environments means your environment is housed in multiple locations such as the cloud, on-premises, virtual or code. There’s also hybrid geography, hybrid architecture, hybrid style—the list goes on.

The key to transitioning to DevOps is properly managing the different types of hybrid. If you can’t manage hybrid delivery, you can’t manage your entire delivery and you can’t improve your delivery holistically. Organizations must learn to manage in a world of hybrid delivery and be able to handle cloud and on-prem, as well as teams all over the planet.

Fixing Non-Constraints

The Theory of Constraints is about pinpointing the bottlenecks in your process and fixing them. Any work that’s not focused on eliminating the constraint will not result in improvement and could be considered a waste of time.

It’s important to remember that if you haven’t fixed the bottleneck, you haven’t improved anything. At the same time, if you’re automating the non-constraints, you’re wasting efforts by trying to improve what doesn’t need improving.

As we’ve pointed out, DevOps does not equal automation. If an organization has messy processes and tries to automate them, they’ll be left with an automated mess. Therefore, organizations will see the best results from automating their constraints and just as important, not automating the non-constraints.

— Jeffrey Keyes

May 13, 2019
Nutanix Eases Development of IoT Apps

Nutanix, a vendor of hyperconverged infrastructure solutions, has released a new developer environment that allows developers to try and test IoT and AI apps in the cloud. Starting May 7, developers can sign up for a free 10-day trial of this experience.

The developer environment is built under Nutanix’s Xi IoT Platform and gives developers access to a rapid development environment for creating internet of things and artificial intelligence apps in the cloud. The Rapid Development Environment for IoT includes features such as Xi IoT Cloud Instance, Xi IoT Sensor App and Xi IoT App Library and supports the ability to easily deploy IoT software in the cloud or at the edge, according to the company. The environment has been tuned specifically for IoT and AI, eliminating many of the configuration chores developers had to perform using other cloud-based development environments, and enables single-click deployment via a management console.

For developers targeting production-edge environments, the new environment brings forth the ability to develop for the edge or for the cloud, bringing application portability into the development and testing process. By being able to use a common set of APIs, cloud and edge development becomes relatively seamless and deployment friction is vastly reduced, according to the company.

Beyond ensuring compatibility across platform services and APIs, the environment enables development and production environments to use the same components, so recoding is not needed as part of the transition from dev to ops. That helps to simplify lifecycle management as well as fit agile practices better into the dev process.

Nutanix also offers an extensive library of applications that can be used to ease development chores. The Xi IoT App Library offers individual components (message bus services, time series databases, etc.) and full solutions that developers can use to address various business problems.

Nutanix is framing the new delivery mechanism as a new experience to get more developers—from novice to advanced—creating AI and IoT scenarios.

— Frank Ohlhorst

May 10, 2019
5 Guaranteed Ways to Kill DevOps Developer Productivity

“A developer is an organism that turns coffee into code.” I still laugh when I see that quote as it reminds me of my hardcore developer days, when my cube always seemed to be directly next to the coffee machine and caffeine was management’s way of ensuring we had the energy to develop well into the wee hours of the evening.

What is developer happiness? I like to believe it hasn’t changed dramatically over the years. It’s about giving developers the right tools and the freedom to do what they do best—produce meaningful code. That means reducing distractions so they can meet or beat expectations and timelines, ultimately driving revenue for the organization.

I trust your company is not in the practice of molding reusable human coffee filters. But are you doing everything you can to help your developers be as satisfied and productive as possible, or are you taking the “necessary” steps to ensure they are buried in context-switching hell?

If the latter is your aim, read on to learn about five guaranteed ways you can totally kill developer productivity.

Saddling Developers With ‘Plumbing’ Work

Companies frequently look at developers primarily as technical resources instead of true innovators, so they get saddled with plumbing work. In DevOps and continuous delivery (CD), that’s writing a lot of ad-hoc scripts to string together pipelines, define infrastructure and manually create deployment plans. If your company is on the road to cloud adoption, it’s imperative to get unnecessary scripting under control because things only get more complicated as you start to incorporate the many different cloud platforms and services that are available.

Making Developers Deal With Security After the Fact

A fail-safe way to kill developer productivity is to make them deal with security after a release has left development. Too often, they first build code, QA then tests it and finally someone from security checks it to make sure there are no holes well after the code has been written. If security problems are found after coding is done, the release is halted and must go back to the developer to fix. This often occurs either just before the production release or, worst case, just after. In either case, not only does it take longer to resolve the problem, it also prevents the developer from moving on to the next big thing.

Expecting Them to Chase Down Features and Status Requests

Because development pipelines are often not integrated with the higher-level CD pipeline, those outside of development have no way of seeing the status of features, deployments and releases. Understandably, their only option is to ask developers to provide status information or call a status meeting which, of course, are additional interruptions that divert them from feature development. Developers don’t like it, and it doesn’t exactly help anyone else in the pipeline be more productive, either.

Bogging Developers Down With GUIs

GUIs are a great democratizer in large organizations. They offer a user-friendly way for people across the company, regardless of technical know-how, to access the information they need to be productive. But many developers see them as a waste of time. And while even the most GUI-resistant developer needs them every so often, they are not their preferred way to work.

Taking Away Their Favorite Tools

Standardizing your software delivery process is a must to increase efficiency, but standardizing on tools is not. Taking away their favorite tools is highly recommended only if you’re keen on decreasing developer satisfaction.

What are the Alternatives?

Managers and leaders can help developers by taking as much of these mind-numbing tasks off their plates as possible. They can do that by automating the release process and integrating CI pipelines into the CD pipeline. Doing so allows everyone to plug in to what’s happening in development without having to interrupt developers, and they can continue to use their beloved tools and simply connect them into the release pipelines.

It’s also important to standardize most of your operations work upfront—infrastructure, provisioning, compliance, security. All this advance work will allow developers to consume environments on-demand and just deploy without extensive scripting or re-scripting for changes and maintenance.

Keep in mind, if you shift security left, closer to the development process, you need to automate it. Otherwise, it becomes someone’s manual task—a developer, DevOps engineer or security person—and that will slow down the delivery process. For example, SAST test and SCA scans can be automatically run on every check-in. DAST tests can be automated to run after each build. This way, any security holes are caught as early as possible, while the code is still fresh in the developer’s head.

Finally, reduce the amount of work developers need to do through GUIs. Give them a way to kick off releases from their respective CI pipelines and the ability to define release pipelines—infrastructure, configuration, security and so on—in code.

Coffee Machine or Barista?

Doing these five things—relieving developers of unnecessary scripting, standardizing compliance requirements and security checks upfront, reducing the need for developer-provided status updates, enabling them to work in code and allowing them to continue to use their favorite tools—will make the difference in whether your developers feel like the office coffee machine or a master barista.

— Dan Beauregard

May 10, 2019
Promoting Purpose in Agile Environments

Agile environments increasingly are becoming a popular option for organizations large and small. Although initially created as a solution for small software development teams with short budgets and complex scopes, it’s been adopted (and refashioned) for all industries and team sizes.

Agile Methodologies: Why They Work

The agile approach seems obvious to us now, but back in 2001 when the idea was incepted, it was a revolutionary one: Give employees the room they need to get the job done and let them do it. Throw out extensive documentation and overbearing technical documents for constantly moving, constantly changing, constantly reassessing and growing workplaces.

“In 2001, a group of experienced software developers got together and realized that they were collectively practicing software development differently from the classical waterfall methodology,” wrote Isaac Sacolick for InfoWorld’s “What is Agile Methodologies?”

“This group came up with the Agile Manifesto that documented their shared beliefs in how a modern software development process should operate,” he continued. “They stressed collaboration over documentation, self-organization rather than rigid management practices and the ability to manage to constant change rather than lock yourself to a rigid waterfall development process. From those principles was born the agile methodology for software development.”

How the Framework Works: Higher Purpose

An agile approach pursues the production of top-tier software. Without an agile framework (or something like it), there is often a distinct lack of purpose-driven principles. From an agile framework, companies can create strong cultures.

Why is higher purpose so important? After the great recession hit in 2008, we were left with some very important questions amid a financial disaster, the likes of which hadn’t been seen in 80 years. When the economic foundations of our world were rattled, many firms were forced to answer what their higher purpose really was.

Without the promise of a steady paycheck or a fat margin, there is all too often an absence of purpose. Even when the economy is performing in a spectacular manner, a lack of higher meaning pervades many mediocre organizations.

“Although a higher purpose does not guarantee economic benefits, we have seen impressive results in many organizations,” reported Harvard Business Review writer Robert E. Quinn and Anjan V. Thakor. “And other research—particularly the Gartenberg study, which included 500,000 people across 429 firms and involved 917 firm-year observations from 2006 to 2011—suggests a positive impact on both operating financial performance (return on assets) and forward-looking measures of performance (Tobin’s Q and stock returns) when the purpose is communicated with clarity. So, the purpose is not just a lofty ideal; it has practical implications for your company’s financial health and competitiveness.”

How to Create Purpose With Agile Principles

Joe Robles, the former CEO of USAA, found this utter lack of meaning to be of immense concern for every level of his organization. Concerned about his employees, he visited them to discern where the company culture could improve. He was surprised to find that his call center employees were happy to collaborate on every level.

It became his mission to imbue this type of mission-driven attitude to every employee and everyone in leadership, through major cultural education. How can other organizations emulate USAA’s successful purpose-driven culture? There are a few steps to achieving this goal.

The first step is to imagine the workforce you wish to have. In agile environments, there is a focus on efficiency. There is no unnecessary research, documentation or meetings; there are only results. Successful outcomes, however, only come from motivated, purpose-driven individuals. That means we must ascend above the employer-employee relationship and, instead, swap it with an inspiring mentorship program.

The next step is to make this dream a reality. Craft an authentic message, promote that message, then promote the supplementation of that message with continual learning, training and dialogue.

Conclusion

Agile environments foster meaning and cooperation. Instead of employees that mindlessly listen to the autocratic directives of their superiors, they are part of a task force who shoulder the responsibility of reaching a higher goal together. This means some critical and divergent thinking must be fostered, which can be frightening to traditionalists of all forms.

In agile environments, you will need to start asking questions that help expand how the team thinks and operates. In addition, listening, analysis and reflection skills must be a primary focus for a higher meaning to be divined.

— Anthony Coggine

May 10, 2019
DevOps: What Do Governments Need to Know?

How the government can successfully implement and enforce DevOps practices

The speed at which technology is being produced it outstripping the pace at which better development tools are being introduced, which makes secure development even more challenging. Unfortunately, the negligence of organizations and authorities is intensifying the issue.

For this reason, organizations are adopting DevOps. for the betterment of technology. This way technology will be not only safe but also exciting to use.

The term DevOps is derived from software development (Dev) and information technology operations (Ops), combined to create better results when separate groups work together as one team.

What Can DevOps Do?

DevOps has to manage the IT infrastructure as per the necessity of the recommended software code that is dedicated in a composite multi-tenant environment. There is a requirement to have the resources for provisioning, to get the suitable deployment model and also to authorize and monitor the performance.

When DevOps is discussed, there is a requirement to have constant and incremental adjustments in the code versions so regular testing and deployment are possible. It might not always be feasible for DevOps to do coding from scratch, but they still must understand it.

DevOps needs to connect several elements of coding and desegregate various segments of SQL data management or messaging tools to release the working software, with operating order and production infrastructure. It could be both the developers who started into operations and the system administrators who started into the coding domain. Either way, there is a cross-connection work function that is seeing a massive upside in the style of software that is produced and used in mission-critical applications.

How Can DevOps Be Enforced Successfully?

Now that we have discussed what DevOps do, it’s time to learn the ways in which organizations enforce DevOps successfully.

Higher Authority

It is essential that we refer to higher authorities. By getting support from the top authorities, we can easily manage the work of DevOps.

“DevOps is fundamentally a reorganization.”

–Adrian Cockroft, lead architect, Netflix

Altering any management organization, with its deeply rooted techniques of functioning and complicated network of employees and suppliers, is a challenging job. So, solid backing at the beginning is as crucial as having employees. You cannot start a new technique without employee support; in the same way, you cannot begin a new plan without any support from authorities. You need someone from the top to monitor the work and provide feedback.

Blameless Postmortem

Adopting DevOps also requires every member of the team be comfortable in their learning. Hence, the blameless postmortem, which eliminates finger-pointing when things go wrong—highly recommended for DevOps beginners. Blameless postmortems are even required by some government offices.

Blameless postmortems organize a cross-section of those invested in the project to evaluate why certain outcomes occurred. The postmortems include an examination of the situation and ways to resolve them through various procedures or technology variations. They are an excellent method to initiate constant development.

Increase DevOps Engineers

Another effective way of expanding DevOps and enforcing it in the government is through hiring DevOps engineers. A DevOps engineer is the individual responsible for all the steps during the process of DevOps, and are responsible for bringing both organizations together to work as a team. There are not enough DevOps engineers today and the workload is much more than they can handle.

Start Small and Scale Up

Whenever you start something new, start on a smaller scale. Whether you’re working on a business plan or testing a new recipe, you should always start small and work your way up.

In the same vein, it is important to start small when implementing DevOps. Rather than go all-in at once, create a group of people who will work together as a team. If the results are good enough, then expand your implementations to a wider audience.

Eliminate Restrictions

Last but not least, it is essential to remove all the constraints involved. Eliminate any limitation that may occur during DevOps enforcement. Limitations could vary from generating new constructions or applying code variations to assimilating data foundations. If the constraints are not removed from the beginning, it may cause problems afterward.

Conclusion

With the expansion of equipment and expertise, new practices are introduced in the government. If applied properly, these practices could generate pleasant outcomes. DevOps, which brings together two forces for effective results, is one such practice.

— Zehra Ali

May 8, 2019
Logz.io Unveils Azure Open Source ELK Monitoring Solution

Needless to say, DevOps as a practice is largely fueled by open source. The growth and popularity of open source tools such as Elasticsearch, Kubernetes and others are driven largely by the increased flexibility, innovation and familiarity of options. While this isn’t news for developers (65% of which are active open source contributors), leading cloud providers are taking notice, as they try to increase their appeal to DevOps engineers to finally win the cloud wars.

Currently, AWS is the leading cloud provider. However, with Azure gaining more and more momentum, all three major cloud providers are scrambling to offer their customers the best open source options and empower them to use the tools they know and love in a manner that is convenient and native to their cloud environments.

Going Big on Open Source

Today, Logz.io and Microsoft Azure announced that our ELK-as-a-service will now be offered on Microsoft Azure. Logz.io offers an intelligent machine data analytics platform built on ELK and Grafana, two of the most popular open source monitoring and troubleshooting tools. Our platform enables engineers to easily deploy, run and scale ELK without the hassle and pain of maintaining and managing the stack themselves.

Now, with Logz.io for Azure, users can use open source ELK as a highly integrated experience in Azure. The new offering includes seamless integrations to various Azure services, out-of-the-box Azure dashboards, built-in alerting and crowdsourced machine learning capabilities.

The integration between Logz.io and Azure is not only significant in the context of the cloud wars, but in the culture of Microsoft itself. While the company was previously associated with a more proprietary structure, they have made tremendous strides in recent years toward empowering open source by opening its .NET framework and Azure SDK, as well as their acquisition of GitHub.

According to Julia Liuson, corporate vice president of Microsoft’s developer division: “At Microsoft, we see open source as a major driver of innovation within the engineering community. Since we strive to empower our customers and facilitate their success, it is important to us to meet them where they are by making it easy to access and use open source tools. Our partnership with Logz.io will strengthen Microsoft Azure ties with the developer community, enabling teams to easily build and monitor their products using ELK.”

AWS vs. Google Cloud

Microsoft’s investment in ELK comes just in time to provide an answer to its competitors, AWS and Google Cloud. Just two months ago, AWS announced Open Distro for Elasticsearch, which in their own words “is a value-added distribution of Elasticsearch that is 100% open source (Apache 2.0 license) and supported by AWS.” The new offering includes managed Elasticsearch and Kibana along with features such as built-in alerting, security and more. This was AWS’ most recent appeal to the engineering community, offering one of the most popular monitoring tools as a managed service on AWS.

Following this release, Google Cloud announced its own open source offering featuring integrations with leading companies such as Elastic, MongoDB, Confluent and more. According to Google, the purpose of these integrations was to “provide users with a seamless user experience and the ability to easily leverage these open-source technologies in Google’s cloud.”

Is Open Source the Future?

If what’s happening between cloud vendors is any indication of future trends in the engineering space, it seems imperative that all organizations targeting DevOps teams must consider how they’ll fuel open source and empower its vast user community. At Logz.io, we are proud to be part of this initiative.

Overall, it is great to see the strides large organizations are taking in furthering the interests of the developer community and enabling them to work using the tools, methodologies and strategies that best suit their workflows and ideals.

— Daniel Berman

May 7, 2019
AIOps: Supporting Reliability at DevOps Speeds

AIOps looms large as a way to help push the DevOps envelope. How can organizations prepare?

As organizations journey down the path of DevOps maturation, sustainable IT operations and IT service management remains a challenge for many. Even advanced organizations that have managed to speed up deployment rates and improve software quality struggle to maintain the resilience of the infrastructure that supports those applications. To support DevOps speeds, a growing number of organizations are turning to AIOps—the use of artificial intelligence (AI) and machine learning in IT ops—to speed up analysis of IT problems and better automate incident handling.

A new study out this week from OpsRamp shows that ops pros are able to reduce mean-time-to-resolution of incidents by as much as 50% through the use of AIOps. The use of AIOps is still infrequent, but analysts say its prevalence will grow quickly. Gartner says about 5% of organizations were using AIOps tools last year, but that by 2023 the deployment rate will shoot up to 23%.

But as DevOps teams lean on AIOps tools, management will need to harness a new set of skills to get the most out of the types of operational automation AIOps will afford. In the coming years, experts believe AIOps will change the complexion of the IT ops workforce. Fewer people will be needed in the network operations center (NOC) to manually sift through alerts and respond immediately to incidents. More people will be needed to curate data, scrub it and train the algorithms that will do the heavy lifting of event correlation and determining the root cause of problems.

As organizations and individuals plan out how they’re going to future-proof themselves for the changes in IT ops wrought by AIOps, experts say they should increasingly see the need for the following three roles in IT organizations.

AIOps Architects

Expect to see more job postings in the near future looking for AIOps architects and automation path designers, said Will Cappelli, CTO of Moogsoft and the former Gartner analyst who’s known for coining the term AIOps.

“(AIOps architect is) essentially a software development job involving the construction of meta-algorithms that guide the flow of control of the AIOps systems,” Cappelli said.

Potentially falling in this umbrella could be roles such as machine learning and AI designers that focus on designing the algorithms themselves, and interaction designers that focus on designing the automated workflows. According to Brendan Caulfield, co-founder of ServerCentral Turing Group, these roles already exist in other business capacities. With AIOps, these design jobs will start to be applied specifically to IT operations as well.

To prepare, veteran IT ops experts should start thinking about how they can bolster their skills, and management need to recruit and train accordingly.

“Mathematics and an end-to-end understanding of how modern IT systems behave will be critical,” Cappelli said.

ITOps Data Scientists

AI depends upon good, clean data and sound modeling to work effectively.

“The most sophisticated and expensive AI tools are useless if a company isn’t able to feed them with a high volume of high-quality data, in a format the system can digest,” said Rajesh Kalidindi, CEO of Leva Data.

This was corroborated by the OpsRamp report released this week that showed maintaining data accuracy is the No. 1 challenge cited by organizations deploying AIOps today. Many organizations will need ITOps specific data scientists to attend to this problem.

“Building high-volume, high-quality data sets will be increasingly important, and it requires a lot of human effort,” Kalidindi said. “AI has a long way to go before catching up with human reasoning (let alone surpassing it), and it can’t make any progress without people developing data models for AI to work with.”

Augmented Intelligence Analysts

According to Caulfield, algorithm training and support by domain experts in operations will become increasingly important at most organizations.

“No matter how ‘smart’ an algorithm is, or how well its designed, there will be exceptions that are unique to a business that will need oversight and tuning,” he said.

Kalidindi concurred that human intervention will be needed for a long time to augment and validate the insights and actions driven by AIOps, noting that this will lead to the rise of a new role for ops analysts.

“Especially in the early days of machine learning, humans are needed to judge whether an algorithm is leading to the right judgment,” he said. “For example, if we were to apply AI to analyzing the problems the IT Help Desk encounters, a human would be needed to judge whether the algorithm made an appropriate recommendation. If not, IT and data science needs to work with the human’s recommendation to better train the AI model on how to handle that problem in the future.”

One thing that most experts agree about is that the rise of AIOps will not diminish the need for IT pros. In fact, they say that domain expertise will be more important than ever. However, it will need to be bolstered by additional knowledge of data science, how AI works, and systems architecture.

“They need to retool themselves from admin-oriented activities to scripting/reporting-oriented tasks, and they need to be more application aware than ever before as infrastructure becomes a commodity,” said Ravi Raghavenderrao, vice president of service delivery at NetEnrich, of IT ops professionals in an AI-enabled future.

To those who have adjusted to DevOps transformations, this should all sound very familiar.

— Ericka Chickowski

May 6, 2019
With YAML Support, Puppet Dives Deeper Into DevOps

Puppet this week announced an update to Puppet Enterprise that adds additional DevOps capabilities, including integration with Bolt, an open source agentless task runner that Puppet employs to make IT automation more accessible to IT administrators.

Matt Waxman, vice president of products at Puppet, said adding integration with Bolt will make it easier to integrate DevOps processes that rely on YAML to deploy applications.

Available this month via a Puppet Enterprise 2019.1 update, Bolt provides an agentless approach to automation that enables IT teams to execute existing commands and scripts written in any language within the context of an IT automation framework that is designed to be accessible to the average IT administrator, Waxman said.

At the same time, Puppet is providing the ability to set up pipelines for deploying automation code directly from within Puppet Enterprise. Rather than having to set up a separate continuous integration/continuous deployment (CI/CD) platform, Puppet is embedding the capability it acquired from its purchase of Distelli in 2017. In doing so, Puppet is making it possible to accelerate deployment of automation software in a way that doesn’t require every change to be approved by a master Puppet administrator. Default pipelines guide organizations new to continuous delivery practices through setting up their first pipeline in less than 30 minutes.

Puppet is also including an Impact Analysis capability that enables teams to assess the impact of a proposed Puppet code change before it is deployed.

One of the issues that has held back adoption of DevOps practices has been the simple fact that most IT administrators have limited programming skills. By making available an IT automation framework that provides the ability to manipulate YAML files at a higher level of abstraction, it should become easier of organizations to incorporate IT administrators into any set of DevOps process they care to define, Waxman said.

Puppet via this upgrade is also extending the reach of the agentless Bolt framework to include support for network devices running Cisco NX-OS and Cisco IOS, as well as firewalls from Palo Alto Networks. As network operations continues to be more integrated with DevOps processes, IT organizations increasingly need to be able to apply workflows across compute, storage and networking platforms, Waxman said.

The challenge many IT organizations now face in terms of achieving that goal is each provider of a platform creates their own automation framework, which winds up creating islands of automation across the enterprise. Longer term, Waxman said that, in addition to invoking the application programming interfaces (APIs) that many platforms already expose, Puppet will be working toward integrating domain-specific automation frameworks with its cross-platform framework.

Overall, Waxman said Puppet remains committed to providing a mix of agentless and agent-based approaches to IT automation based on the nature of the IT tasks that need to be automated. It will be up to each IT organization to determine when to employ one of those options. But regardless of the path chosen, Puppet is clearly trying to make its core automation platform a lot more accessible to everyone in IT.

— Mike Vizard

May 3, 2019