devopsy.kinsta.cloud

Tag: automation

  • The Now and Next of Automation

    The Now and Next of Automation

    Today’s never-ending flood of data has many of us drowning in noise. And while humans are still essential for thoughtful analysis and driving action, it’s no longer possible for a human to parse gigabytes of data alone.

    How do you begin to filter out the noise and then understand what to do with it? How can you get to the root cause of an issue so you can act on and resolve it? Most of all, how can you do all of this while being more productive and driving more innovation instead of constantly putting out fires?

    Automation is the obvious answer. For many organizations, automation isn’t just about efficiency – it’s about survival. Automation can make organizations more agile, resilient, and capable.

    Where you sit within an organization or team, however, often dictates how automation can help with your specific goals or alleviate your immediate pain points. Mandi Walls, a DevOps advocate with PagerDuty, offers some advice on how even less experienced teams can use automation to perform simple tasks, solve problems and improve the work environment.

    We hear a lot these days about A.I. and machine learning, or tools like Chat GPT which to some is a harbinger of robots finally replacing humans. Putting the hyperbole aside, where do you see automation today? Is it still a “nice to have” for engineering teams or a “must-have” to do their jobs effectively?

    Automation is definitely a “must-have.” Manual processes are risky; they can inadvertently introduce too many errors, especially in complex environments. The scope and complexity of modern technical ecosystems is just too vast for teams to effectively cope with using only manual methods.

    This is especially true for teams that develop and manage customer-facing services. Having a successful product leads to more customers, which requires more capacity and creates more complexity. To deliver the kind of performance customers expect, teams need to be fast and efficient. That’s where automation comes in.

    What are some specific types of automation that engineers and IT pros should be aware of?

    Some automation has become more or less ubiquitous, such as tools that enable configuring and building large numbers of systems with little human intervention. Other processes might feel less like “big-A” automation–for example, continuous integration/continuous delivery (CI/CD) pipelines are automation.

    Humans also aren’t manually building many software packages anymore. The use of automation in the software testing process has greatly expanded in the last several years, changing the jobs of QA engineers significantly.

    Is self-service automation – aka true autonomy that doesn’t require human intervention – the ideal end state for a digitally mature organization? When is the best time to jump in?

    Organizations that wait for a perfect opportunity to automate their processes risk wasting a lot of time and resources in the interim. Teams can look at any of the tasks that they regularly perform and assess them as candidates for automation.

    Tasks can be assessed in a number of different ways, but for teams just starting out with automation, seeking tasks that are low complexity and low impact is a strong starting point. Even if we look just within the realm of tasks that might be utilized during incident response, teams might have any number of tasks related to information gathering, log preservation, system diagnostics, and other telemetry that responders run for every incident. There’s no reason to require humans to run those well-understood tasks!

    As teams get more comfortable with creating and using automation, they can seek out more complex tasks to automate. And it’s entirely possible that there will be tasks that teams aren’t able to automate fully! Those tasks might be too complex, require a lot of human input or interaction or have too many edge cases for the team to create automation for.

    Are the principles of automation the same in simple versus more complex environments like Kubernetes?

    The principles are essentially the same, but the constraints and tools may be different. In some environments, a simple shell script might be enough to accomplish our goals. In other environments, we might need to make use of APIs or vendor SDKs to do what we want.

    On that topic, is self-service automation in cloud environments the preferred path?

    Actually, self-service is sort of how the cloud really got going! Development teams armed with nothing more than a credit card got the environments they needed without going through lengthy procurement processes.

    Now that cloud is a critical part of many workloads, organizations are putting more rigorous guidelines in place. The cloud platforms are also much more complex, with more service offerings than in the early days, which makes automation more important than ever. Automation helps organizations ensure that their cloud environments meet their security and compliance requirements, helps manage spend and still provides enough flexibility for all teams to get work done. And with self-service automation, developers can request an environment in the cloud to develop their service and have that service meet all the organization’s standards without having to keep up with all the details themselves.

    What about toil? Can that be fully eliminated through automation, or is some toil still necessary?

    I’m not sure I’d say toil is necessary, as much as it tends to be inevitable. There will likely always be some boring, repetitive tasks that are too unpredictable or unstable to automate well.

    That said, teams certainly don’t need more toil; having too much repetitive work tends to lead to less job satisfaction and higher attrition rates, so teams should be looking to minimize as much toil as possible where they can.

    Is there one myth or misconception, either about automation overall or self-service automation in particular, that you’d like to clear up? What’s the reality?

    First of all, don’t worry. You won’t be able to replace your team with a set of shell scripts! Seriously though, automation isn’t about “doing more with less.” It’s about doing more with the same. Automation is an approach to help teams cope with the day-to-day running of complex environments and will need to be managed as part of those environments. Managers need to make sure they are allotting time for the creation and maintenance of automation in the same way they would for other tools and applications.

    Interested in learning more about PagerDuty? Stop by Booth S49 at KubeCon to speak with an expert or see a demo!

  • Automation Relation

    Automation Relation

    One thing that is absolutely true: This latest iteration of IT improvement is about automation. Whether making DevOps more automated, working on any of the various “X-as-code” movements or implementing AI, the idea is to create what one vendor portrayed as a “human-free zone.” A space where automation does everything that needs doing.

    And the one question we all need to be asking now is simple: Where do we want that human-free zone in our organization?

    The answer is inevitably, “In more places than you think,” for a wide variety of reasons. Let’s cover what you should be thinking about when answering this question – because this change is already upon us and we’re doing a lot more without so much as an approval.

    1. Anywhere that compliance is on the line and the action doesn’t risk crashing systems.
    2. Anywhere the process being performed is so rote that any member of the team can do it and everyone thinks of that process as a distraction.
    3. Anywhere that time is critical. Think of incident response to a DDoS or zero-day attack.

    This is a very basic starting point. From here, we want to move into “anywhere we have a static list of steps to perform.” So, the above points would include things like, “If a new employee is in the marketing department, they belong in the Marketing security group. If they are further working on marketing for new products, they belong in the Marketing for product development group.” But eventually, you want to get to, “This app communicates on port 12345, so that port must be accessible, run the script to open that port.” The decision about “Should this port be open?” can then be shifted left to have more involved discussions about options and necessity. Our current world already does some of that; automation will make it standard.

    And that’s what we’re getting. We’re in a world where some standardization is enabling automation, and automation is demanding that we figure out more standardization. And it is all good. There is a lot we do in IT that a trained monkey could do, and we need to let a trained system do it. The more time we have, the more problems we can solve … So let’s keep freeing up that time.

    And keep rocking it. Your time is valuable precisely because you keep the organization running. Freeing up that valuable time just makes you a greater asset to the organization and makes your job less boring as a bonus.

  • Orgs Shift Toward Centralizing Automation Initiatives

    Orgs Shift Toward Centralizing Automation Initiatives

    A survey published today found that 75% of respondents recognized that automation silos hamper their ability to unify cloud operations across functions.

    Conducted by CloudBolt Software, the survey polled 350 senior IT and DevOps leaders at organizations with more than 3,000 employees. On a positive note, the survey found 57% of respondents were starting to pursue a more centralized approach to automation, with a total of 88% acknowledging that having all previous and new automation centralized for the entire organization would ensure compliance, bolster security, enable business continuity, encourage reuse and reduce risk.

    Rather than taking a more holistic approach, well over half of respondents (57%) reported automation efforts are limited to specific silos. On average, organizations are employing 27 different automation tools their organizations have deployed, using one or a few as they see fit, with little to no consistency across teams. More than half (55%) of respondents said they have four or more separate automation teams, with 95% noting that each of those teams operates in their own silo.

    CloudBolt CTO Rick Kilcoyne said that lack of continuity clearly creates inefficiencies and redundancies that conspire to raise the total cost of IT. The challenge is that each team tends to favor one approach to automating a silo with little to no regard for a larger overall strategy to create an automation standard, he said.

    That approach also creates issues whenever a member of one of those teams leaves an organization, he noted. In fact, the survey found 77% of respondents reported that continuity is at risk any time a vital member of the team leaves.

    CloudBolt has been making a case for a federated automation platform that can be invoked either via application programming interfaces (APIs) or a graphical user interface (GUI) that provides access to low-code tools. The goal is to make the platform equally accessible to DevOps teams or IT administrators that tend to lack programming skills, and do so in a way that doesn’t require individual teams to replace their automation platform of choice. Instead, each of the automation platforms can be programmatically invoked via a central framework, said Kilcoyne. That approach eliminates any political debate over the potential need to replace one team’s favored automation platform with another, noted Kilcoyne.

    Regardless of who is accessing the platform, most organizations would be well-advised to develop a center of excellence for automation, he added.

    DevOps teams have always been committed to ruthlessly automating as many IT processes as possible, but in many cases, that has resulted in a plethora of automation tools being employed. In effect, automation may have become too much of a good thing. One approach to regaining control over those platforms is to create a platform engineering team that centralizes the management of DevOps workflows.

    Regardless of approach, Kilcoyne said the most important thing for any team to do before automating an IT workflow is to first understand how to manually set it up. Too many IT teams are trying to apply automation to processes they don’t really understand with mixed results, he added.

    The one thing that is certain, especially with the rise of artificial intelligence (AI), is the management of IT is going to become more automated.

  • How to Supercharge Your Engineering Teams

    How to Supercharge Your Engineering Teams

    With the economy still bleak, data and engineering teams that were already struggling to keep pace with business needs are now facing a potential hiring freeze or worse in 2023. That’s why there’s never been a more important time to maximize the productivity and impact of your teams, but the question is how to identify the big changes that move the needle in performance. Incremental adjustments that save a sliver of time here and there are not hard to come by, but what teams need are the breakthroughs that yield giant leaps in productivity and efficiency. Rather than a 10% improvement, it’s the 10x improvements that drive lasting gains.

    As an engineer and CEO of a software company, I’m constantly looking for ways to boost the productivity of our teams. Our guiding principle is simple: Maximize the impact per hour invested for every individual. To do that, we must ensure that individuals maximize the time spent flexing their superpowers, minimize busy work and reduce dependencies on others that delay work or tie them up in meetings. While counterintuitive at first, we consistently find that collections of well-aligned, independently executing individuals produce a far greater impact than a large team swarming on a project. Put another way; the individual impact goes up when the number of people you depend on to deliver the result goes down.

    I’ve developed an approach over the years that has allowed me to identify several opportunities for step-change advancements in productivity and efficiency, and I’d like to share it here, along with two concrete examples. I will also share an exercise we run with our teams to help identify and execute these opportunities.

    First, here are my two main criteria for identifying the projects and tasks where we can effect changes that will have an outsized impact on the company:

    ● Big resource-consumers: Work that’s essential to the business but consumes a large amount of engineering and other resources is a top candidate for optimization.
    ● Big impact, easy win: Areas that will have a significant impact on company performance if we can do them faster and with higher frequency are also great candidates—even if they’re not consuming a ton of resources.

    Here are two examples that illustrate each of those criteria and where we were able to see massive gains in productivity, quality of service, or both.

    Tackling Big Resource-Consumers

    At Ascend.io, we provide a platform for building and automating data pipelines, which requires us to build connectors into a huge variety of data sources. At one time, building a new connector required multiple engineers to spend two to three weeks writing more than 5,000 lines of code that spanned four microservices in three different languages. As we added more customers and needed to support more data sources, this quickly became unsustainable.

    When we looked at this process, we saw two issues: Engineers were writing custom code for each connector in a way that would be better served by a common framework. Moreover, the process required engineers from multiple teams and disciplines to work together and coordinate their priorities. As soon as you get dependencies that require engineers to coordinate with others and wait on the outcomes of other people’s work, you’re wasting time.

    We addressed this by designing a new development framework that exploits commonalities across the connectors we build. This now requires far less unique code and removes the need for multiple engineers to work together. In most cases, one person can build a new connector in days with 100 lines of code. This made us far more nimble and freed up engineering time to do other valuable work.

    Achieving High Impact

    Like most companies building software, we need to convey the capabilities of our product to new and existing users to ensure they receive maximum benefit from their use of Ascend. With new features and enhancements being released on a weekly basis, there is a lot to keep up with. This is high-impact work, however, and if we can do it in a vastly better way, it drives an outsized impact for Ascend and our customers.

    For us, building these automated product tours typically required a product marketer or product manager to design the customer journey, and one or more engineers on a sustained basis to build the actions into the product. This again creates dependencies that slow progress and require significant manual coding.

    The breakthrough for us was a tool called Intercom, which allows you to build sophisticated product tours without needing to write code. Our product team can now build more interesting and complex product tours for customers, and our engineers can stay focused on building new features instead of the code needed to showcase them. What’s more, they are able to design, test and deploy these tours independent of our product and code deployment cycle, further freeing them to iterate as fast as possible.

    A Thought Experiment

    So how do you actually find these opportunities for 10x improvement? I run the following thought experiment with our teams several times a year to uncover these opportunities:

    Imagine your workload scales 10x overnight and you absolutely must meet this increased need, but without hiring more engineers or working more hours. How do you do it?

    It’s amazing what this exercise can uncover—but there are ground rules and it can take time to see results. Coming up with these big ideas is a skill like any other, and most people are not great at it the first time they try it. Your first iteration may produce no options to pursue—some may have high-impact ideas but with even higher investment requirements, while others may reject the premise outright. That’s normal. So, don’t insist they sit in a room until they dream up that 10x breakthrough, because it will be demoralizing. Instead, plant the seeds by establishing the above framework and then let the ideas start to grow.

    More likely, a few days later, they’ll be out for a hike or chatting with an old co-worker and they’ll suddenly have the germ of an idea. That may still not be the winning solution, but someone else will build on it, and then someone else and, finally, you iterate your way to a breakthrough. Pulling the team back together and embracing the “yes, and” approach can be both energizing and highly productive.

    Brainstorming like this is fast and free, and you only need one great idea to have a material impact. We run this exercise a few times a year and have come up with several big ideas that were game-changers for the company. Besides the examples above, we also moved key parts of our software development from Scala and Go to Python a few years ago, which is a much more efficient language (as measured by developer time) and allows our engineers to build things much faster.

    Software engineering is a complex beast that often spans many teams working on multiple processes in parallel. But that complexity is also your opportunity: Every complex system has room for optimization, especially at companies that moved fast in the beginning to build a product and find market fit. Now is the time to lift your gaze, look around and ask where significant gains can be made. They almost certainly exist, and a 10x improvement could be what allows you to not only survive the turbulent year ahead but emerge with a thriving team and business on the other side.

  • Large Organizations Are Embracing AIOps

    Large Organizations Are Embracing AIOps

    A survey of 265 IT leaders found the primary reason organizations are applying artificial intelligence to IT operations (AIOps) is to improve service and application availability and performance (60%), followed closely by automation of operations (58%) and processes (54%).

    Conducted on behalf of OpsRamp, a provider of an AIOps platform, the survey polled 138 IT leaders working for enterprise IT organizations and 127 representing managed service providers (MSPs). The organizations surveyed generated at least $25 million in revenue and have more than 500 employees. The survey found monitoring/native instrumentation (55%) ranked as the most important feature of an AIOps platform.

    Bill Talbot, chief marketing officer for OpsRamp, said the survey suggested the initial reason organizations appear to be adopting AIOps is to improve monitoring and overall observability as IT teams continue to find simple tasks—such as discovering what is in their IT environments—challenging to achieve.

    Overall, the survey found the greatest IT operations challenge for enterprises in 2023 was automating as many operations as possible (66%), followed by driving greater efficiency and productivity of IT operations teams and processes (61%). Application-to-infrastructure dependency mapping is the top incident management challenge (64%), the survey found.

    The primary operational benefit cited was reducing open incident tickets (65%), followed by reducing mean-time-to-discovery (MTTD) and mean-time-to-remediation (MTTR) tied at 56% and automation of tedious tasks (52%).

    The survey also found 80% of respondents reported it took less than six months to implement an AIOps platform, with more than two-thirds (68%) noting it takes more than six months to hire engineers with the right skillsets for AIOps. Well over 90% of respondents said AIOps is delivering the level of value they expected.

    Despite advances in AIOps, however, two-thirds of respondents said they are concerned about the accuracy of the data being used to train AIOps platforms. A little more than a third (36%) are also concerned about AIOps deployment causing job loss.

    While there has been a lot of initial skepticism concerning AI applications in IT operations at scale, adoption has now crossed the proverbial chasm, said Talbot. IT teams are finding a wide range of static application environments that lend themselves to being automated using machine learning algorithms, he added.

    Less clear, of course, is the degree to which AIOps can be applied to highly dynamic IT environments where changes to code are frequently made. However, the more routine processes that are automated, the more time DevOps professionals will have to manage more challenging issues.

    At this juncture, it’s not so much a question of whether AIOps will be employed to manage DevOps processes as much as it is to what degree. It remains to be seen how generative AI platforms such as ChatGPT might be applied to IT operations, but it’s apparent that AI will automate a wide range of tasks. It’s not likely those capabilities will eliminate the need for humans any time soon, but it does mean that many of the rote tasks that make working in IT tedious may soon disappear.

  • What NetOps Teams Should Know Before Starting Automation Journeys

    What NetOps Teams Should Know Before Starting Automation Journeys

    Automation has captured the human imagination since the earliest days of recorded history. A fanciful example of this comes from Homer’s classic “The Iliad” in the form of a bronze robot called Talos, which was designed to automatically monitor and defend the island of Crete from marauding intruders. The appeal of automation for IT organizations, including network operations (NetOps) teams, may not be so life-and-death but it’s undeniable, nonetheless. More and more NetOps teams are turning to automation to achieve higher network uptime (by reducing manual errors), increase efficiency and reduce costs and help deliver a better and more consistent operational experience for all users.

    Due to the allure of these promised benefits, the temptation for NetOps teams to integrate automation as soon as possible may be strong. However, it is important for these teams to remind themselves that automation is a tool, not the destination itself. Indeed, simply overcoming the technical challenges of deploying automation may not result in the successful outcome teams want, without identifying and defining the intended outcome of what their automation journeys should be.

    This article will explore some practical steps and best practices that NetOps teams should use throughout their own automation journeys.

    Getting the Journey Started on the Right Step

    A well-known maxim in business is that you should visualize success and start with the end in mind when embarking on a major project or initiative. This is a good reminder for NetOps teams as they begin their automation journey. That is, these teams should be able to answer the question, “What is our desired end state when it comes to automation?” before they even evaluate what tools or technologies they may want to consider.

    Teams who have this mindset will have a clearer understanding of their overall mission as well as an enhanced view of the various projects, processes and products they can improve through automation. It is critical for NetOps teams to take the time to put in the work of defining the desired IT experience at the end of the automation journey, even if the task itself may be difficult. For example, this experience may help define how NetOps engineers will interact with an automation tool through its code, its dedicated user interface, or even its generic service portal. As the saying goes, “no pain, no gain.” The teams who invest in the time and effort to plan ahead will be far more likely to maximize the benefits of automation.

    What to Automate?

    In addition to visualizing what success looks like, it is important for NetOps teams to identify what tasks or processes they should be automating. This is important because there is a vast universe of NetOps tasks that can be automated. Taking the time to identify tasks to be automated will generate huge benefits for the NetOps team as it will help them select the correct automation strategy and toolset that will deliver them the results in an efficient way. And while specific projects may be unique to each NetOps team, in general, there are two broad categories of tasks that are ideal for automation:

    1. Highly repetitive tasks. These are manual or semi-automated tasks that are not necessarily hard but can be cumbersome for NetOps teams. Examples of repetitive tasks that can be automated include assigning an interface into a specific VLAN or perhaps updating network security policies or rules.

    2. Highly complex tasks. These can be more strategic and difficult tasks that can be prone to human error due to their scope, scale, and complexity. Automating these tasks may require additional planning and steps to ensure success. Examples of complex tasks that are ideal for automation include service redirection from one device to another or BGP peering with third parties.

    Think Big, Start Small With NetOps

    Like all people, NetOps professionals enjoy the results of a job well done. So, while the vision of their automation journey may be big, it’s important to start with a small, short-term project that can be completed quickly. There are a couple of benefits to this approach:

    1. Quick automation wins will give NetOps teams confidence for future projects.
    2. Projects like this can generate data and feedback that NetOps teams can convert into learnings and insights for the next project.

    This approach can also be applied to bigger, more complex automation projects. Instead of taking on the entire scale of the project at once, NetOps teams can break it down into smaller components. For example, infrastructure-as-code (IaC) is a broad category of automation entailing the managing and provisioning of systems through machine-readable definition files. There are many considerations to large-scale IaC projects such as identifying the use case, e.g., Day 0 or Day 2 operations. The advantages of this approach are the same as with the quick-win scenario: There is a better likeliness of success and more immediate feedback and data to guide the NetOps teams through this entire process.

    Finally, as talented as most NetOps teams are, they are not likely to have all of the automation expertise in-house at any given time. So, it is important that they address skills gaps (automation, programming) through strategic vendor partnerships, internal/external recruitment and additional training for their teams. Again, instead of recruiting or investing en masse, NetOps teams are well-advised to take the time to plan, execute, learn and modify as they go along.

    Summary

    The human fascination with automation is stronger than ever. The industry buzz around one particular form of automation—especially artificial intelligence and machine learning—is indicative of this. NetOps teams have much to gain from investing in the tools and skills required to do automation right.

    Automation of data center network pods, self-contained zones that represent a section of the overall data center, is a good example of a strategic project requiring forethought and planning to execute well. A logical first step in this process might be to anticipate how to manage future changes when automation is in place. Options for this may include code manipulation, Git (open source code management) or CI/CD pipelines, all elements of infrastructure-as-code automation. Or it may involve improving user experience through a specific UI interface or through a self-service portal. The next step then might be to decompose this large configuration into smaller bits starting with the registration of the new switches into the controller. Once this is done, we can now go into port configurations and then eventually move toward larger and more sophisticated elements such as the automation of VLAN assignments and L3 peering.

    For long-term and large-scale automation projects like this, it is especially important to follow the recommended key steps outlined in this article, which are:

    1. Start with the end in mind. It’s critical that the teams take the time and effort to define what success looks like, especially in the form of the IT experience they want to achieve.
    2. Know what you want to automate. There is an endless supply of tasks and processes that can or should be automated. Prioritizing automation projects that NetOps wants to take on is a major milestone in any automation journey.
    3. Think big, start small, learn as you go: Quick automation project wins are key to building confidence and “muscle memory” for future projects. For bigger projects, the best approach is to break them down into more manageable projects that can provide vital data and insights during the journey.

  • Before ChatGPT, AIOps Powered Enterprises for Years

    Before ChatGPT, AIOps Powered Enterprises for Years

    Led by OpenAI and its ChatGPT software, a new era of consumer use cases for artificial intelligence is here. These exciting new tools have made AI more accessible to more users than ever before. No longer is this technology the exclusive domain of wealthy corporations.

    However, while the world is now focused on the potential use cases for AI in consumer applications, it’s important to remember that this technology has been part of the business world for years. Behind the scenes, AI has quietly been powering solutions that companies of all sizes have harnessed to improve business outcomes. These technologies have specifically been used by IT teams in the form of artificial intelligence for IT operations (AIOps) to help modernize their companies and operate more efficiently.

    By leveraging AIOps-powered solutions in use cases like observability, overworked and understaffed technology teams are improving their ability to manage complicated, hybrid and multi-cloud IT environments, automate processes and optimize their businesses. As companies prepare for a potential economic downturn and a reduction in resources, these solutions and AIOps are more important than ever.

    Here are four ways that AIOps is helping businesses overcome today’s IT challenges.

    Providing Unified Visibility Into Complicated Networks

    Companies have accelerated their digital transformation efforts in recent years. This has included migrating workflows to public and private clouds, implementing new productivity tools and adopting next-generation technologies. The average business now relies on geographically dispersed workforces and IT environments that use 2.6 clouds and leverage more than 200 apps!

    The result is that organizations lack visibility into these environments. A recent SolarWinds survey found that 54% of technology professional respondents stated they only have visibility into about half or less of their apps and infrastructure. This lack of visibility can lead to problems that negatively impact the return on investment for projects.

    AIOps helps overcome this challenge by providing comprehensive visibility into an organization’s systems, regardless of whether those environments are hosted in on-premises, hybrid or multi-cloud environments. AIOps automatically collects and integrates insights and information across geographies, apps and infrastructures into one dashboard. This provides technology teams with single-pane-of-glass visibility into the state of IT environments.

    Automating Processes & Remediating Issues

    While the visibility provided by AIOps and observability is integral to managing today’s complex IT environments, the technology goes a step further.

    With AIOps, teams can automate processes including the identification and remediation of issues. Given the stakes, this is critical. Companies must ensure their environments are always available and that their performance meets and exceeds customer demands. Even short service outages can lead to frustrated users and a damaged customer experience.

    This is challenging, as an IT anomaly often results in many alerts and a flood of log messages, making the problem difficult for a human to understand and address. AIOps quickly ingests massive amounts of information, provides rapid root cause analysis, data/event correlation and incident management which allows for fast and automated remediation of issues as they arise.

    Additionally, since this technology is always observing and gathering information on potential incidents, it can predict and prevent future issues proactively. By implementing AIOps, companies are taking a step toward autonomous operations or environments that can manage themselves without human involvement.

    Protecting Revenue

    In the face of an uncertain economy, every organization is taking steps to protect its revenue. With AIOps, enterprise technology companies have a powerful tool in this fight.

    By implementing AIOps and observability, companies can quickly resolve and prevent IT issues before they arise. This ensures that IT environments are operating at maximum efficiency.

    With AIOps, organizations can reduce downtime and prevent outages. This ensures that business-critical applications and infrastructure are available and functioning. These capabilities are more important as even short service outages have reportedly been capable of causing millions of dollars in damages.

    Freeing Up Employee Time for Innovation and Efficiency

    Over the last few years, companies have experienced severe staffing challenges, with many organizations finding it difficult to fill critical technology roles. This has led to smaller teams with more responsibilities and less free time. The result has been overworked technology professionals who spend far too much time on manual and menial tasks that do little to support the business’s bottom line.

    By implementing AIOps, organizations can streamline workflows by automating repetitive and routine tasks. Doing so can cut down on the amount of overwhelming metrics technology professionals are required to analyze and, instead, empower those individuals and teams to again focus on innovating and completing projects that support key business objectives.

  • Six Mainframe DevOps Predictions for 2023

    Six Mainframe DevOps Predictions for 2023

    DevOps, or the automation of application development and hand-off to operations, is more prevalent than ever on the mainframe. As we begin 2023, below are six predictions for what the coming year will bring. The composite themes will undoubtedly be speed, alignment and experience.

    1. Enterprise DevOps Will Break Down Silos Across Core Systems

    DevOps practices will continue to expand beyond cloud and distributed systems to include more core systems than ever before. Core platforms, including the mainframe, are the functional backbone of mission-critical applications that drive the world economy. In a recent IDC survey, 82% of organizations reported taking steps to prioritize using the same application development tools across distributed and mainframe teams. 2023 will continue to see a rise in enterprise DevOps that will bring down the silos between platforms. As core system workloads increase, organizations will see the benefit of aligning platforms such as mainframe and distributed to realize the full benefit of their DevOps initiatives.

    2. Growth in AI/ML as Key Components for Measuring DevOps Performance Metrics

    The value of analytics will increase as organizations emphasize the use of key performance indicator (KPI) data to measure the performance and maturity of DevOps pipelines. In this way, organizations will not only gain more insights into their pipelines, but will also be able to proactively manage the data in real-time. They will be able to adjust loads as needed and create smart pipelines with advanced error handling. Organizations will leverage benchmarks within the analytics to see how they compare with their peers.

    3. Cross-Functional Pipelines Will Create Hybrid Automated Everything

    This will enable the cloud, core systems and data to deliver a more holistic view of “platforms, processes and people.” Silos will finally disappear. Automation will become a key priority in this effort as organizations seek to free up time for developers to innovate and achieve greater velocity, quality and efficiency. A single integrated and connected pipeline will emerge that automates the creation and testing of applications in a single flow. The rationale of prioritizing this alignment between distributed and core systems includes the reuse of best practices, workflow simplification, cost-effectiveness, application integration and streamlined training.

    4. DevOps Will Embrace Platform Engineering and Site Reliability Engineering (SRE)

    Platform engineering is the next evolution of DevOps, taking a DevOps platform and turning it into a self-service product to improve the developer experience. It treats DevOps as a product and allows organizations to create an end-to-end offering that serves developers, operations and platform teams. SRE focuses on the “Ops” side of DevOps, specifically how to automate operations capabilities. This is vital to the DevOps mission since it provides the data that allows organizations to manage what they can measure.

    5. DevSecOps Will Play an Increasingly Central Role

    Security must become a central part of the toolchain and a “first-class citizen” in the pipeline. Static application security testing (SAST), also known as white box testing, along with dynamic application security testing (DAST), also known as black box testing, will be further integrated into processes to enable security teams to scan large loads at rest or in real-time quickly and efficiently.

    6. Git-Centric Enterprise Source Code Management (SCM) Strategy Will Grow

    Git has become the de facto standard for SCM for distributed application workloads. In a recent IDC survey of industry professionals across 11 countries and multiple industries, 87% said they will migrate all or some of their mainframe source code to Git over the next three years. This means Git and Git-native development will become important capabilities for mainframe DevOps companies to support. Organizations will increasingly prefer implementation approaches and tools that offer flexibility in both scope and pace.

    The techniques highlighted in these predictions are, naturally, related. They point to an ever-increasing capacity for efficiency and quality that will enable core and distributed technologies to work better together.

  • Understanding the Infrastructure Automation Maturity Model

    Understanding the Infrastructure Automation Maturity Model

    Nearly every company wants to evolve its digital transformation and increase the pace of software delivery and operations. And infrastructure automation is a new means to achieve these ends. It supports the emerging discipline of platform engineering and can increase scalability and reactivity to unforeseen events, enabling software ecosystems to be more anti-fragile. Increased infrastructure automation also helps free up engineers from focusing too much on managing DevOps when they really should be building out new features.

    But introducing more infrastructure abstraction can be met with certain challenges. This is often because departments have a wide range of cultures, processes and tooling already in place. With varying maturity levels across teams, it can be challenging to gauge where a company is before attempting to introduce new automation across the board. And as further evolution may require significant changes, having a baseline is key to planning for the future.

    I recently chatted with David Williams, senior vice president of market strategy at Quali, about an internal technique they’ve been using at Quali to gauge the maturity of their clients’ infrastructure automation. Their infrastructure automation maturity model breaks down the journey into four unique stages, moving from the least-mature ad-hoc automation to the most-mature autonomous solutions. Below, we’ll review this model and consider what questions need to be asked to ascertain where your team sits on the spectrum. We’ll see how enterprises can leverage such a model to hone in on improving their software operations.

    Four Questions to Determine Infrastructure Maturity

    DevOps is still a wild west in terms of how people approach it, said Williams. So before attempting to mandate sweeping cultural changes, it helps to first determine the base level of DevOps maturity within an organization. Here are four questions that can quickly assess the state of infrastructure automation within an organization.

    1. What Culture Already Exists?

    First, it’s good to evaluate the culture that already exists. How do teams deliver value and how is this value measured? Perhaps throughput or code quality is measured with specific metrics or SLAs. Understanding the DevOps attitude (and aptitude) within the organization will highlight pre-existing capabilities and show how teams collaborate.

    2. How Are People Organized?

    The genetic makeup of teams can dictate much about development and operations. How are teams organized? Are they large, siloed teams? Or are they two-pizza-sized teams? Understanding team organization and communication patterns are necessary to identify bottlenecks.

    3. What Processes Are Already in Place?

    Next, consider the development timelines and procedures already established and in use. Are teams applying SAFe, Lean, Agile or Scrum methodologies? What are these exact processes and how mature are they? Answers to these questions will demonstrate general release cadences and how teams drive productization.

    4. What Infrastructure Automation is Already Being Used?

    Lastly, you’ll need a window into what level of infrastructure automation, if any, is already in use today. How is it delivered? How is it managed? How is the success of this automation measured, and what is the overall outcome? You will likely find that automation is typically quite siloed and teams have differing approaches. This auditing is vital to identify what works well and what doesn’t.

    The Four Levels of the Infrastructure Automation Maturity Model

    Asking the above questions will quickly determine an organization’s “readiness state,” said Williams. But how do these answers map to the overall maturity of the organization’s infrastructure automation? Well, he shared four basic levels of infrastructure automation maturity that are commonly found in practice throughout today’s software development environments, ranging from bespoke scripts to more robust self-defining automation.

    Level 1: Ad-hoc

    The most basic level of infrastructure automation is ad-hoc integrations involving a mix of old and new technology. This level is chaotic and fragmented, with varying processes used throughout the organization. These bespoke integrations and scripts also may not have excellent visibility, limiting the potential for reusability within an organization.

    Level 2: Automation

    Next is having automation that is specifically tied to a certain technology stack. Nowadays, organizations are commonly hybrid multi-cloud, having adopted automation piecemeal with each new stack. This often results in multiple silos of automation stacks that are blind to each other. At this maturity level, one automation might fire off another automated procedure and so on, but engineers aren’t 100% aware of how they influence each other.

    Level 3: Frictionless

    The next level of infrastructure maturity begins to take the reins with more intentional automation adoption. At this stage, organizations embrace an internal developer platform, similar to Spotify’s Backstage, that helps engineers deliver and manage infrastructure more seamlessly. This tooling is more advanced than other levels integrating key commands into the developer’s IDE. Doing so eliminates the need to switch between environments and reduces friction.

    Level 4: Self-Defining

    Finally, the most advanced form of automation involves deeper intelligence within application operations, including the ability to dynamically optimize infrastructure. At this stage, infrastructure is self-defining, meaning production applications can automatically adapt, be self-governed and enable bursting and non-bursting capabilities. Such intelligence is typically proprietary to cloud providers, said Williams, yet companies often desire this intelligence inherent at the application level. Williams describes this stage as the “intelligent allocation of resources in line with application life cycle requirements.”

    Benefits of Self-Defining Infrastructure Automation

    Self-defining infrastructure automation has many benefits for companies, including having the infrastructure in line with application needs, scalability and measuring usage to optimize cloud spending. Many points within the software development life cycle could warrant increased automation, such as infrastructure-as-code (IaC), deployment release engines, configuration, traffic scalability, cloud management, containerization and documentation. But it’s not until you open conversations around infrastructure automation maturity and understand who is responsible for managing it that you can hone in on what areas can be improved.

  • Graduating From DevOps to MLOps? 5 Tools to Help

    Graduating From DevOps to MLOps? 5 Tools to Help

    DevOps is a set of practices that aims to bridge the gap between software development and operations. It aims to improve collaboration and communication between these two teams and to automate the process of software delivery so that changes can be made and deployed more quickly and easily. This can include continuous integration, continuous delivery and infrastructure-as-code. The goal is to enable faster delivery of new features and improvements to customers while also improving the reliability and stability of the software.

    What is MLOps?

    Machine learning operations (MLOps) is the practice of applying DevOps principles and practices to machine learning projects. It involves the collaboration of data scientists, machine learning engineers and operations teams to streamline the process of building, testing, deploying and maintaining machine learning models in a production environment. 

    The goal of MLOps is to increase the speed and reliability of deploying ML models while also improving the overall quality of the models. This can be done by using tools such as version control, continuous integration and automation of the model deployment process.

    Why DevOps is Not Enough: The Benefits of MLOps

    Machine learning models have unique requirements, such as data privacy, version control and model deployment that are not addressed by traditional DevOps practices. MLOps is specifically designed to address these unique requirements and to ensure that machine learning models are deployed and managed in a production environment in a way that is consistent with other software components. 

    There are several benefits to implementing MLOps in a machine learning project, above and beyond the benefits of DevOps:

    • Improved collaboration: MLOps promotes collaboration between data scientists, machine learning engineers and operations teams, which helps to ensure that models are deployed quickly and efficiently.
    • Faster deployment: By automating the model deployment process and using continuous integration and continuous delivery (CI/CD) practices, MLOps can help to speed up the time it takes to deploy a model to production.
    • Improved model quality: By using version control and automated testing, MLOps can help to ensure that models are thoroughly tested and of high quality before they are deployed to production.
    • Better monitoring and maintenance: MLOps can help to improve the monitoring and maintenance of deployed models by providing better visibility into the performance of models and enabling easy rollbacks if necessary.
    • Better model governance: MLOps can help to improve model governance by providing a clear process for the management of models, including versioning, testing, and deployment.

    5 MLOps Tools to Help Transition From DevOps to MLOps

    Making the move from DevOps to MLOps typically requires dedicated tools that can help manage the complexity of machine learning models and datasets. Here are a few tools, some open source and some commercial, that can help.

    MLflow

    MLflow is an open source platform for managing the machine learning life cycle. It aims to make it easier for data scientists and machine learning engineers to develop, deploy and track machine learning models. MLflow provides several capabilities that can help streamline the machine learning development process, such as:

    • Experiment tracking: MLflow allows data scientists to track and compare different versions of their models, including the code, parameters and metrics associated with each run.
    • Model packaging: MLflow allows data scientists to package their models in a format that can be easily deployed to a variety of environments, such as a local machine or a cloud platform.
    • Model deployment: MLflow provides tools for deploying models to a variety of environments, such as a REST API or a container.
    • Project management: MLflow allows data scientists to organize their work into projects, including multiple models and associated files.
    • Model registry: MLflow provides a centralized model registry that allows data scientists to store and manage models.

    Pachyderm

    Pachyderm is an open source platform that provides data versioning and pipeline management for machine learning and big data workloads. It allows data scientists and engineers to manage, version and collaborate on large datasets and create reproducible and reusable data pipelines.

    Pachyderm uses a version control system (VCS) like git to version data, which allows data scientists to track and collaborate on data sets and enables reproducibility and rollbacks. This allows tracking the changes in the data, seeing who made the changes and when they were made and rollback to previous versions if needed.

    It also provides a powerful pipeline management system, which allows engineers to create, test and deploy data processing pipelines. The pipeline management system is designed to work with containerized workloads, which makes it easy to deploy and manage pipelines.

    Pachyderm is built on top of Kubernetes, which allows for easy scalability and can be run on-premises or on any cloud provider. It also supports data processing frameworks like Apache Spark and Apache Flink, as well as machine learning frameworks like TensorFlow and scikit-learn.

    Amazon SageMaker

    Amazon SageMaker is a fully-managed cloud-based machine learning platform. It provides a suite of tools for building, deploying and managing machine learning models. It is designed to make it easy for data scientists and developers to quickly and efficiently build, train and deploy machine learning models in a production environment.

    Some of the key features of Amazon SageMaker include:

    • Model building: Amazon SageMaker provides a variety of tools for building machine learning models, including pre-built algorithms and frameworks for popular libraries such as TensorFlow, Apache MXNet and scikit-learn.
    • Model training: Amazon SageMaker allows data scientists and developers to train machine learning models on large-scale datasets, using distributed training and automatic scaling to improve performance and reduce training time.
    • Model deployment: Amazon SageMaker makes it easy to deploy machine learning models to production, with a variety of options for hosting models, including Amazon SageMaker hosted endpoints and Amazon Elastic Container Service (ECS) and Kubernetes.
    • Model management: Amazon SageMaker provides tools for monitoring, updating, and maintaining deployed machine learning models, including automatic A/B testing, canary deployments and automatic model rollbacks.
    • Experiment management: SageMaker experiment management is a feature that allows data scientists to keep track of the different versions of their models, the parameters and the metrics associated with each run.
    • AutoML: SageMaker AutoML is a feature that allows data scientists to automatically explore different algorithms and hyperparameters to find the best model for a given dataset.

    H2O MLOps

    H2O.ai’s MLOps is a platform that allows data scientists and engineers to collaborate and deploy machine learning models in production. It provides a set of tools for model development, management, and deployment, as well as monitoring and governance capabilities. 

    It aims to streamline the process of taking machine learning models from development to production while maintaining high standards of model quality and performance. Some of the main features include:

    • Model Development: It provides a set of tools for developing and testing machine learning models, including support for popular frameworks such as TensorFlow, PyTorch and H2O.
    • Model Management: It allows users to organize and manage models in a central repository, making it easy to track versions and collaborate with other team members.
    • Deployment: It provides an easy way to deploy machine learning models in a variety of environments, including on-premises, in the cloud or at the edge.
    • Monitoring: It offers monitoring capabilities that allow users to track the performance and quality of deployed models in real-time and automatically detect and diagnose issues.
    • Governance: It provides governance capabilities to help ensure compliance with regulatory requirements, such as data privacy, and to help manage the risks associated with deploying machine learning models in production.

    Neptune.AI

    Neptune.ai is an open source platform that provides a suite of tools for organizing, tracking and reproducing machine learning experiments. It is designed to help data scientists and machine learning engineers to keep track of their work, collaborate more effectively and improve reproducibility.

    Some of the key features of Neptune.AI include:

    • Experiment tracking: Neptune allows data scientists to track and compare different versions of their models, including the code, parameters and metrics associated with each run.
    • Experiment management: Neptune provides a user-friendly web interface for organizing and managing experiments, which makes it easy to keep track of progress and share results with others.
    • Reproducibility: Neptune provides a way to reproduce experiments by tracking the code, data, environment and dependencies used in each run.
    • Collaboration: Neptune allows data scientists to share and collaborate on experiments with others and provides a way to track who made changes and when they were made.
    • Model management: Neptune provides a centralized model management system that allows data scientists to store and manage their models, including versioning, testing and deployment.

    Conclusion

    In conclusion, MLOps is a practice that applies DevOps principles and practices to machine learning projects to streamline the process of building, testing, deploying and maintaining machine learning models in a production environment. The goal of MLOps is to increase the speed and reliability of deploying ML models while also improving the overall quality of the models. 

    There are several tools available that can help organizations to implement MLOps, including MLflow, Pachyderm, Amazon SageMaker, H2O MLOps and Neptune.AI. Each of these tools provides a different set of features and capabilities, but all aim to make it easier for data scientists and machine learning engineers to develop, deploy and track machine learning models.

    Image Source